Item-focused trees approach in differential item functioning (DIF) analysis: a case study of an EFL reading comprehension test

Abstract

Recent years have witnessed an increasing interest in providing new insights into modern language testing method targeting test bias. In this study, item-focused trees (IFT) approach was applied to identify uniform and non-uniform differential item functioning (DIF) of an English as a foreign language (EFL) reading comprehension test. The multistage cluster sampling method was employed to randomly choose a large sample of 4937 students who took the entrance exam of MA program in English studies. The reading comprehension section of the general English test including 20 items was selected for the IFT analysis. Three categorical and continuous DIF source variables including gender and academic background were concomitantly taken into account for the IFT analysis, which is capable of handling more than one variable with both binary and continuous measurement. Then, in the final stage of IFT analysis within a logistic regression framework, uniform and non-uniform DIF was analyzed using DIF tree package of R. The results showed that 10 items had uniform DIF in which 2 items had 2 joint DIF predictor variables (2 splits) and 8 items had only one split. Additionally, 6 splits and 5 non-uniform DIF items were found in non-uniform DIF analysis in which only 1 item had 2 simultaneous DIF source variables. Furthermore, gender and background knowledge had significant relationships with EFL reading comprehension. This study promises practical implications for addressing gender and background knowledge differences in EFL reading comprehension studies on the one hand, and impacting language testing methodology on the other.