Identifying current practices in modeling multi-year student achievement data: a systematic methodological review

Abstract

ABSTRACT Assessing student achievement over multiple years is complicated by students’ memberships in shifting upper-level nesting structures. These structures are manifested in (1) annual matriculation to different classrooms and (2) mobility between schools. Failure to model these shifting upper-level nesting structures may bias the inferences researchers make about predictors of student achievement. The current study is a systematic methodological review of 59 multi-year elementary education studies that examine student achievement growth. The review focuses on studies from the previous decade with the hope of better understanding current practices in modeling student growth in multiyear educational research, the extent to which researchers account for shifting upper-level nesting structures, and the percent of variability attributed to different levels. Findings indicate that approaches to address shifting upper-level nesting structures are not consistently applied. Further, many researchers fail to recognize shifting upper-level nesting structures as a potential threat to the validity of their findings. Finally, evidence is presented that suggests even when schools are included in a model additional variability is attributed to classrooms beyond what is attributed to schools. Accordingly, it may be necessary to include multiple classroom memberships, where such data exist, in models that seek to understand contributors to student achievement.