Measurement Comparability of Reading in the English and French Canadian Populations: Special Case of the 2011 Progress in International Reading Literacy Study

Shawna Goodrich,Kadriye Ercikan

doi:10.3389/feduc.2019.00120

Abstract

The purpose of this study is to examine item equivalence and score comparability of the Progress in International Reading Literacy Study (PIRLS) 2011for the Canadian French and English language groups. Two methods of differential item functioning were conducted to examine item equivalence across thirteen test booklets designed to assess reading literacy in early years of schooling. Four bilingual reviewers with expertise in reading literacy conducted independent, linguistic and cultural reviews to identify both the degree of item equivalence and potential sources of differences between language versions of released items. Results indicate that an average of 25% of items per booklet function differently at the item level. Reviews by experts indicate differences between the two language versions on some items flagged as displaying differential item function (DIF). Some of these were identified to have linguistic differences pointing to differential difficulty levels in the two language versions.

Highlights

Measurement Comparability of Reading in the English and French Canadian Populations: Special Case of the 2011 Progress in International
Results of logistic regression (LR)/ordinal logistic regression (OLR) analysis are shown in Tables 3 and 4 compare the number of items identified as differential item functioning (DIF) by the item response theory (IRT) and LR/OLR methods for all the booklets
The LR/OLR method identified two items, which were not identified as DIF by the Linn and Harnisch (LH)-IRT method

Summary

Introduction

Measurement Comparability of Reading in the English and French Canadian Populations: Special Case of the 2011 Progress in International. Previous research has identified a number of differences between languages that make test adaptation difficult and can result in the incomparability of assessment scores between language groups (Ercikan, 1998; Bonnet, 2002; Arffman, 2007; Grisay and Monseur, 2007; Sireci, 2008; Marotta et al, 2015) These include differences in grammar, meaning, vocabulary, syntax, word usage, and difficulty (Allalouf and Sireci, 1998; Ercikan, 1998; Allalouf et al, 1999; Ercikan et al, 2004a; Arffman, 2007)

Objectives

Methods

Results

Conclusion