Combining Mokken Scale Analysis with Rasch measurement theory to explore differences in measurement quality between subgroups

Abstract

Mokken Scale Analysis (MSA) is a nonparametric approach that offers exploratory tools for understanding the nature of item responses while emphasizing invariance requirements. MSA is often discussed as it relates to Rasch measurement theory, which also emphasizes invariance, but uses parametric models. Researchers who have compared and combined MSA and Rasch approaches have generally concluded that these approaches offer complementary perspectives. However, researchers have not considered how the two approaches may provide insight related to examinee subgroups. Using a simulation and illustrative real data analysis, we propose an exploratory approach to integrating MSA and Rasch techniques to understand item fit as it relates to differences in item characteristics between examinee subgroups. Our results suggest that integrating complete-sample and subgroup-level MSA and Rasch model item fit analyses can provide insight into differences in item quality between subgroups. In some cases, misfit at the complete sample level corresponds to subgroup-related differences. On the other hand, fit at the complete sample level does not guarantee consistent measurement properties within subgroups. When subgroups are important for scale interpretation and use, our results suggest that conducting separate analyses within groups and considering their results with respect to those from the full sample may lend useful insight.