Investigating Sources of Differential Item Functioning in International Large-Scale Assessments Using a Confirmatory Approach

Abstract

International large-scale assessments of achievement often have a large degree of differential item functioning (DIF) between countries, which can threaten score equivalence and reduce the validity of inferences based on comparisons of group performances. It is important to understand potential sources of DIF to improve the validity of future assessments; however, previous attempts to identify sources of DIF have had variable results. This study had two purposes. The first was to apply a confirmatory approach (Poly-SIBTEST) to investigate sources of DIF typically found in international large-scale assessments: adaptation effects and cognitive loadings of items. We conducted three pairwise DIF analyses on Spanish and English versions of the Progress in International Reading Literacy Study 2001 Reader booklet. Results confirmed that item cognitive loadings were a source of differential functioning favoring both England and the United States when compared against Colombia; however, adaptation effects did not consistently favor one group or the other. The second purpose of this study was to highlight strengths and limitations of Poly-SIBTEST for conducting substantive analyses of differential functioning sources and also to offer suggestions for future directions on this type of methodological research.