A Comparative Study on using Contingency Table Approaches in Differential Item Functioning Analysis

Abstract

This study provides a demonstration of differential item functioning (DIF) analysis. It made use of test scores of 200 junior high school students on a Chemistry Achievement Test, a measure tested for its psychometric properties. One hundred students came from a public school, while the other 100 were private school examinees; one hundred students were males and the other 100 were females; and 95 students were of low ability and 105 students were of high ability based on their English II grades. Four contingency table approaches, the Chi-Square, Distractor Response Analysis, Logistic Regression and the Mantel-Haenszel Statistic, were applied in the DIF analysis to identify test items indicating bias between examinees matched on school type, gender, and English ability. Thereafter, the results for the four approaches were compared. The findings revealed the presence of items indicating school type-, gender-, and English ability-based DIF. There was a high degree of correspondence between the Logistic Regression and the Mantel-Haenszel Statistic in identifying potentially biased test items.