Identification of Nonuniform Differential Item Functioning: A Comparison of Mantel-Haenszel and Item Response Theory Analysis Procedures

Abstract

Although practical and one of the most popular methods, the Mantel-Haenszel (MH) procedure was not initially designated for detecting nonuniform differential item functioning (DIF). To improve detection rates for items showing nonuniform DIF, a modification of the standard MH procedure has been proposed by breaking the full sample at approximately the middle of the test score distribution and then rerunning the MH procedure on each subsample separately. The purpose of the present study was to compare this MH variation to a residual procedure based on item response theory analysis to test its capacity for the detection of nonuniform DIF. Different sets of examinee responses were generated to simulate both uniform and nonuniform DIF. Results from this study suggest that although the MH procedure may be effective for detecting nonuniform DIF, and although this capacity may be increased with a simple modification to the MH standard procedure, the residual analysis procedure achieves a higher detection rate with both uniform and nonuniform DIF.