How do different versions of a test instrument function in a single language? A DIF analysis of the PIRLS 2006 German assessments

Abstract

The challenge inherent in cross-national research of providing instruments in different languages measuring the same construct is well known. But even instruments in a single language may be biased towards certain countries or regions due to local linguistic specificities. Consequently, it may be appropriate to use different versions of an instrument in a single language to allow for regional differences. Using data from the Progress in International Reading Literacy Study (PIRLS) 2006, this article examines the consequences of differing German translations of the reading assessment on item difficulty in Austria, Germany, Luxembourg, and in the German-speaking Community of Belgium. A differential item functioning (DIF) analysis indicates that a substantial number of items have significantly different item difficulties. Especially items with differing translations function differently. A closer look at these items shows that there are plausible reasons why one version is easier and another version more difficult.