What PISA Intends to and Can Possibly Achieve: A Critical Programme Theory Analysis

Abstract

This article advances the enlightened discussion of the nature, logic, and possible effects of the Programme for International Student Assessment (PISA). The purpose is to analyse the assumptions regarding how PISA is to achieve its intended effects, that is, to reconstruct PISA's programme theory (PT) and to probe the validity of its underlying assumptions. The article demonstrates that PISA's PT has low internal validity. However, some PISA assumptions are consistent, for example, the assumption that legitimisation activities justify PISA as a transnational benchmarking system measuring education system performance. PISA exemplifies systemic evaluation governance: all actors in the field are expected to use PISA results to react to and reflect on their own practice, compare themselves with others, and then act accordingly to improve education systems and school practice, though no activities or resources are allocated to change school practice. There is no empirical research into how systemic evaluation governance works in practice that can be used to probe PISA's external validity. PISA's PT is in line with discourse policy, governance theory, and school effectiveness research, but whether and under what conditions and how PISA helps change education systems and school practice are empirical questions waiting to be answered.