The validity of international surveys of reading literacy: The case of the IEA reading literacy study

Abstract

In the last twenty years the interest in international comparative surveys of educational achievement has increased as can be seen from the number of studies and participating nations. As the number of participating countries increases, the range of cultural differences becomes larger, and it becomes increasingly difficult to obtain internationally comparable data. Especially when it is no longer possible to use the same questionnaire in each group. This article describes the requirements that measures of reading literacy must meet to be comparable across nations, and the conditions that affect the usefulness of international comparisons. Before discussing the methodological problems of comparative surveys in Section 4 and 5, we discuss the definition of reading literacy in Section 2, and the possible reasons why comparative educational research has become so popular in Section 3. In Section 6 to 8 we consider the conditions that affect the comparability of measurements of reading literacy in more detail. In the final sections we summarize our findings. As an illustration we consider an international comparative study, which was recently conducted by the International Association for the Evaluation of Educational Achievement (IEA). The results of this study were reported in detail by Elley (1992, 1994), Postlethwaite and Ross, (1992), Lundberg and Linnakyki (1993), Wagemaker et al. (1996), the IEA (1995), and the US Department of Education (1995). We will refer in less detail to the ZntemationaE Adult Literacy Study (IALS), which was conducted in the autumn of 1994 by the Organisation for Economic Co-operation and Development (OECD). The results of this study were published by the OECD in 1995 and 1996.