Getting Better All the Time? An Illustrative Investigation of Multilevel Longitudinal Measurement Invariance Based on School Climate Surveys

Abstract

AbstractSchool climate surveys are central to school improvement and principal evaluation policies. The quality of school climate has been linked both to student achievement and to teacher retention. Oftentimes, policymakers and practitioners are concerned with monitoring change in school climate quality in each academic year. Such applications assume longitudinal factorial invariance—it is presupposed that the surveys are measuring the same things in the same metric at each time point. While there is considerable research examining the validity of inferences based on survey‐derived climate indicators, this research is almost exclusively based on cross‐sectional data. There is little literature describing procedures for gathering evidence of factorial invariance of school climate indicators. This study proposes to adapt existing methods for evaluating factorial invariance in longitudinal designs into multilevel frameworks, and in doing so, articulates a novel method for evaluating longitudinal measurement invariance in school climate research. This technique is illustrated on a widely used school climate survey.