Evaluation of Factors Affecting the Performance of the S−X2$S-X^{2}$ Item‐Fit Index

Abstract

AbstractOrlando and Thissen (2000) introduced the item‐fit index for testing goodness‐of‐fit with dichotomous item response theory (IRT) models. This study considers and evaluates an alternative approach for computing values and other factors associated with collapsing tables of observed and expected numbers (OE tables), which can affect flagging items. Results suggest that collapsing OE tables requires careful consideration of a trade‐off between power and empirical type I error rate. Concurrent collapsing of score categories would be preferred over separate collapsing for its procedural simplicity, minimal effect of choice of a minimum cell value on empirical type I error rates, and reasonable type I error rates even for the most sparse condition in the study. For separate collapsing, a smaller minimum cell value is recommended as OE tables possess more sparseness (e.g., longer test lengths and smaller sample sizes) if inflated type I error rates are more of a concern in detecting items for misfit based on the index. If it is more important to identify misfit items, the study results recommend using a larger minimum cell value for collapsing.