Using educational effectiveness research to design international comparative studies: turning limitations into new perspectives

Abstract

This paper argues that Educational Effectiveness Research (EER) could be a foundation upon which studies of the International Association of the Evaluation of Education (IEA) could be based, both in terms of their design and in terms of data analysis. First, it is supported that multilevel modelling techniques, some of the most frequently used statistical techniques in the domain of EER, offer a wider spectrum in analysing IEA data. An illustrative example is used to provide support for this argument. Specifically, TIMSS 1999 data concerning student self-perceptions and achievement in both mathematics and science were analysed using single- and multilevel analyses. While single-level analysis resulted in positive correlations at the micro-level and negative correlations at the macro-level, the findings of multilevel analysis did not support such contradictious relationships. Second, the paper argues that the design of international comparative studies could be based on the theoretical models of EER that attempt to explain the variation in student achievement by taking into account variables at different levels. To support this argument a large number of variables taken from the student, teacher and school questionnaires of TIMSS 1999 associated with the comprehensive model of educational effectiveness were added to a multilevel model. Implications for the design of comparative studies are drawn.