Assessing Differential Step Functioning in Polytomous Items Using a Common Odds Ratio Estimator

Abstract

Many statistics used in the assessment of differential item functioning (DIF) in polytomous items yield a single item‐level index of measurement invariance that collapses information across all response options of the polytomous item. Utilizing a single item‐level index of DIF can, however, be misleading if the magnitude or direction of the DIF changes across the steps underlying the polytomous response process. A more comprehensive approach to examining measurement invariance in polytomous item formats is to examine invariance at the level of each step of the polytomous item, a framework described in this article as differential step functioning (DSF). This article proposes a nonparametric DSF estimator that is based on the Mantel‐Haenszel common odds ratio estimator (Mantel & Haenszel, 1959), which is frequently implemented in the detection of DIF in dichotomous items. A simulation study demonstrated that when the level of DSF varied in magnitude or sign across the steps underlying the polytomous response options, the DSF‐based approach typically provided a more powerful and accurate test of measurement invariance than did corresponding item‐level DIF estimators.