Rasch fit statistics and sample size considerations for polytomous data

Adam B Smith,Michael Sharpe,Robert Rush,Galina Velikova,Lesley J Fallowfield

doi:10.1186/1471-2288-8-33

Abstract

BackgroundPrevious research on educational data has demonstrated that Rasch fit statistics (mean squares and t-statistics) are highly susceptible to sample size variation for dichotomously scored rating data, although little is known about this relationship for polytomous data. These statistics help inform researchers about how well items fit to a unidimensional latent trait, and are an important adjunct to modern psychometrics. Given the increasing use of Rasch models in health research the purpose of this study was therefore to explore the relationship between fit statistics and sample size for polytomous data.MethodsData were collated from a heterogeneous sample of cancer patients (n = 4072) who had completed both the Patient Health Questionnaire – 9 and the Hospital Anxiety and Depression Scale. Ten samples were drawn with replacement for each of eight sample sizes (n = 25 to n = 3200). The Rating and Partial Credit Models were applied and the mean square and t-fit statistics (infit/outfit) derived for each model.ResultsThe results demonstrated that t-statistics were highly sensitive to sample size, whereas mean square statistics remained relatively stable for polytomous data.ConclusionIt was concluded that mean square statistics were relatively independent of sample size for polytomous data and that misfit to the model could be identified using published recommended ranges.

Highlights

Previous research on educational data has demonstrated that Rasch fit statistics are highly susceptible to sample size variation for dichotomously scored rating data, little is known about this relationship for polytomous data
Fit Statistics – Type I error rate Tables 1, 2, 3, 4 show the fit statistics for each item averaged across sample size and provide an indication of the Type I error rates
For cases where mean square statistics fell within the range 0.7 – 1.3, the t-statistics increased in magnitude as sample size increased, for the t-statistic the Type I error rate was inflated and the probability of identifying misfit where none was identified by the mean square statistics increased with sample size

Summary

Introduction

Previous research on educational data has demonstrated that Rasch fit statistics (mean squares and t-statistics) are highly susceptible to sample size variation for dichotomously scored rating data, little is known about this relationship for polytomous data. These statistics help inform researchers about how well items fit to a unidimensional latent trait, and are an important adjunct to modern psychometrics. Given the increasing use of Rasch models in health research the purpose of this study was to explore the relationship between fit statistics and sample size for polytomous data. The final criterion, which will form the focus of this paper, is item fit, in other words whether individual items in a scale fit the Rasch model

Objectives

Methods

Discussion

Conclusion