A Monte Carlo Comparison of Parametric and Nonparametric Polytomous DIF Detection Methods

Abstract

Two parametric procedures for detecting differential item functioning (DIF) using the Graded Response Model (GRM; Samejima, 1969)-the GRM-Likelihood Ratio test (GRM-LR; Thissen, Steinberg, & Gerard, 1986) and the GRM-Differential Functioning of Items and Tests (GRM-DFIT; Flowers, Oshima, & Raju, 1999)-were compared with a nonparametric DIF detection procedure, Poly-SIBTEST (Chang, Mazzeo, & Roussos, 1996), in a simulation study. The 3 DIF procedures were examined (a) under conditions in which the GRM provided an exact fit to the data and (b) under conditions of slight model misfit. Small amounts of model misfit were simulated by applying the GRM-based DIF procedures to data generated from alternative item response models. Although all 3 DIF procedures adhered to nominal Type I error rates when data were generated from the GRM, the GRM-LR demonstrated large Type I error inflation under certain conditions when the generating model was not the GRM. GRM-DFIT showed much less Type I error inflation under model misspecification, and Poly-SIBTEST appeared to be unaffected by the generating model. However, of the 3 methods, Poly-SIBTEST showed the least amount of power across all conditions. Implications for the use of parametric versus nonparametric DIF procedures with real test data are considered.