Challenges for the Design of International Assessments: Sampling, Measurement, and Causality

Abstract

This paper examines critical design factors that influence data quality in educational research, using international large-scale educational assessments as an example. We will focus on statistical challenges related to sampling, measurement, and causality. While international assessments employ rigorous random sampling techniques, deviations such as exclusions and non-participation can introduce bias and affect representativeness. In terms of measurement, although these assessments excel in core domains, there is a growing call for broader assessment areas, such as environmental literacy and civic education. Additionally, concerns are emerging about the quality of context surveys. Causality remains a central concern, and despite the challenges posed by the cross-sectional design, combining data and applying sophisticated analytical methods can help address causal questions. Recognising the interconnectedness of sampling, measurement, and causality is essential for conducting robust research and informing evidence-based policies and practices.