Effects of the common scale setting in the assessment of differential item functioning.

Abstract

This study compares three methods of detecting differentialitem functioning (DIF), the equal mean difficulty (EMD), all-other-item (AOI), and constant item (CI) methods, in terms of estimation bias and rank order change of ability estimates using a series of simulations and two empirical examples. The CI method generated accurate DIF parameter estimates, whereas the EMD and AOI methods produced biased estimates. Moreover, as the percentage of DIF items in a test increased, the superiority of the CI method over the EMD and AOI methods became more apparent. The superiority of the CI method is independent of the sample size, test length, and item type (dichotomous or polytomous). Two empirical examples, a mathematics test and a hostility questionnaire, demonstrated that these three methods yielded inconsistent DIF detections and produced different ability estimate rankings.