The Challenges of Measuring Outside-School-Time Educational Activities: Experiences and Lessons from the Programme for International Student Assessment (PISA)

Abstract

Recent years have brought a major shift in the field of international comparative education with the rise of quantitative assessments of student achievement. Within these studies, outside-school-time (OST) as a supplement and complement to schooling has been included. However, OST is difficult to define and measure; and challenges multiply when comparisons are made across countries and cultures. The Programme for International Student Assessment (PISA) managed by the Organisation for Economic Cooperation and Development (OECD) has sought to confront these challenges. It has had some success, and the 2015 PISA iteration showed advances over its predecessors; but it also showed significant shortcomings. With a methodological thrust, this article examines the challenges of preparing appropriate questions in English, translating those questions into other languages, and preparing meaningful cross-national comparisons. The article has pertinence not only within the specific domain of OST studies but also the wider thrusts of quantification in international comparisons.