Abstract
PurposeIn Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions.MethodsWe conducted two simulation studies in the context of dichotomously scored item responses. We manipulated the type of assumption violation, the proportion of violating items, sample size, and quality. False positive rates and power to detect assumption violations were our main outcome variables. Furthermore, we used the Crit coefficient in a Mokken scale analysis to a set of responses to the General Health Questionnaire (GHQ-12), a self-administered questionnaire for assessing current mental health.ResultsWe found that the false positive rates of Crit were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Overall, in small samples Crit lacked the power to detect misfit, and in larger samples power differed considerably depending on the type of violation and proportion of misfitting items. Furthermore, we also found in our empirical example that even in large samples the Crit index may fail to detect assumption violations.DiscussionEven in large samples, the Crit coefficient showed limited usefulness for detecting moderate and severe violations of monotonicity. Our findings are relevant to researchers and practitioners who use Mokken scaling for scale and questionnaire construction and revision.
Highlights
Mokken scale analysis (MSA; e.g., [9, 12, 19, 20, 22]) is a popular item response theory (IRT) approach for evaluating the psychometric quality of tests and questionnaires in various fields such as psychology, education, health and qualityof-life (QoL), or marketing (e.g. [7, 11, 16, 33, 34])
We focus on the so-called Crit coefficient [15] that summarizes information from the H coefficient and other statistics concerning the violation of model assumptions
We found that Crit for checking the monotonicity assumption in MSA was affected to a large extent by sample size, the number of misfitting items, and the type of violation of monotonicity: For small samples (N = 100), Crit had very low power to detect violations of monotonicity, regardless of the type or amount of violation
Summary
Mokken scale analysis (MSA; e.g., [9, 12, 19, 20, 22]) is a popular item response theory (IRT) approach for evaluating the psychometric quality of tests and questionnaires in various fields such as psychology, education, health and qualityof-life (QoL), or marketing (e.g. [7, 11, 16, 33, 34]). Molenaar and Sijtsma [15] provided some tentative rules of thumb to help researchers interpret the severity of a violation, but these rules of thumb were empirically (i.e., not theoretically) derived from a limited set of real datasets To fill this gap, and to investigate whether Crit can be advocated to be used in practical applications, in the present study we first discuss the formulation of the Crit coefficient in the context of Mokken scale analysis and some of its properties. We discuss the usefulness of the Crit coefficient and of the proposed rules of thumb as a measure of effect size for violations of Mokken scales. As for checking monotonicity, for IIO the Crit coefficient can be calculated for each item, according to Eq 2, and the same rules of thumb apply [15]. An example of how to obtain the quantities in Eq 2 when evaluating IIO is available in the Online Resource
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
