Multilevel Modeling for Testing Whether Items Have Good Discrimination

Abstract

Several indices of item discrimination, such as the index of discrimination, have been used in routine item discrimination analysis. Because no objective criterion is available to judge whether an item has good discrimination, test analysts usually rely on subjective criteria, such as 0.3 for the index of discrimination, to screen bad items. Wang and Hung (2002) developed an objective procedure for testing whether dichotomous items have good discrimination. The present study generalizes their procedure to polytomous items. Moreover, multilevel models are proposed to further increase predictive precision. Parameter estimation and hypothesis testing are addressed. Two empirical examples of dichotomous items and rating scale items are provided to illustrate implication and applications of the proposed objective procedure.