Evaluating the Robustness of Graded Response Model and Classical Test Theory Parameter Estimates to Deviant Items

Abstract

The authors examined the in.uence of deviant scale items (i.e., items failing to adequately assess the construct of interest) on item parameter estimates of focal scale items and person parameter estimates through a comparison of item response theory (IRT) and classical test theory (CTT) methods. The authors used monte carlo methods to fully explore results found in a pilot investigation of jobattitude data. They found that graded response model item discrimination parameter estimates were robust when the number of focal items was larger than the number of deviant items, when the focal items were more discriminating than the deviant items, and when the correlation between the focal and deviant items was positive. IRT discrimination estimates were typically more robust than CTT discrimination estimates. Under extremely unfavorable conditions, however, CTT estimates were more robust than the IRT estimates. For person parameter estimates, the robustness of estimates depended primarily on the correlation of the deviant items with the focal construct. Implications are discussed for researchers employing IRT or CTT methods in scale development.