Impact of Item Parameter Drift on Rasch Scale Stability in Small Samples over Multiple Administrations

Abstract

ABSTRACTTraditional psychometric guidelines suggest that at least several hundred respondents are needed to obtain accurate parameter estimates under the Rasch model. However, recent research indicates that Rasch equating results in accurate parameter estimates with sample sizes as small as 25. Item parameter drift under the Rasch model has been previously investigated, but not under small sample conditions. The current study simulated data using the Rasch model to investigate the impact of varying proportions, magnitudes, and directions of item drift across multiple administrations with sample sizes of 10, 25 and 50. High proportions of high magnitude drift strongly affected bias, RMSE, and classification accuracy. High proportions of positive drift (items becoming harder across administrations) were particularly problematic for classification accuracy, with accuracy falling to less than 60% in the most extreme conditions. Results indicate that item drift affects Rasch examinee ability estimates similarly under both small and large sample conditions, and even small to moderate degrees of unaddressed item drift can substantially affect the validity of scores and pass-fail decisions.