The Influence of Conditioning Scores In Performing DIF Analyses

Abstract

The effect of the conditioning score on the results of differential item functioning (DIF) analyses was exam ined. Most DIF detection procedures match examinees from two groups of interest according to the examinees' test score (e.g., number correct) and then summarize the performance differences across trait levels. DIF has the potential to occur whenever the conditioning criterion cannot account for the multidimensional interaction between items and examinees. Response data were generated from a two-dimensional item response theory model for a 30-item test in which items were measuring uniformly spaced composites of two latent trait param eters, θ1 and θ2. Two different DIF detection methods— the Mantel-Haenszel and simultaneous item bias (SIBTEST) detection procedure—were used for three dif ferent sample size conditions. When the DIF procedures were conditioned on the number-correct score or on a transformation of θ1 or θ2, differential group perfor mance followed hypothesized patterns. When the con ditioning criterion was a function of both θ1 and θ2 (i.e., when the complete latent space was identified), DIF, as theory would suggest, was eliminated for all items. Index terms: construct validity, differential item functioning, item bias, Mantel-Haenszel proce dure, SIBTEST.