Analyzing International Large-Scale Assessment Data with a Hierarchical Approach

Abstract

International large-scale assessments in education (ILSAs) follow complex sampling designs and ultimately create hierarchical data structures with students nested in classrooms, classrooms in schools, schools in regions, etc. To describe adequately key issues in education, such as socioeconomic gaps in academic achievement or the relations among school characteristics and student achievement using ILSA data, researchers need to consider the hierarchical data structure in statistical models. Multilevel modeling is one approach to account for such hierarchies and consider variables at different levels of analysis. This chapter provides an overview of the prominent multilevel modeling approaches to analyzing ILSA data, illustrates and discusses their strengths and weaknesses, and highlights the key methodological decisions researchers have to take in this context. The first part reviews the current practices of multilevel modeling in secondary analyses of ILSA data. This rapid systematic review is followed by a second part in which we present, illustrate, and discuss multilevel modeling approaches, including multilevel regression, multilevel structural equation models, and multilevel mixture models. Next to model estimation and fit evaluation, we review key issues associated with the multilevel modeling of ILSA data and focus on handling plausible values, multigroup and incidental multilevel data structures, and weighting. Our chapter provides worked examples showcasing the potential of multilevel modeling for ILSA data analysis.