Does the Model Matter? Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments

Abstract

Policymakers have demonstrated an interest in using measures of student achievement to inform high-stakes teacher personnel decisions, but the idea of using student outcomes as a teacher performance measure is complex to implement for a variety of reasons, not least of which is the fact that there is no universally agreed upon statistical methodology for translating student achievement into a measure of teacher performance. In this article, we use statewide data from North Carolina to evaluate different methodologies for translating student test achievement into measures of teacher performance at the elementary level. In particular, we focus on the extent to which there are differences in teacher effect estimates generated from different modeling approaches, and to what extent classroom level characteristics predict these differences. We find models that only include lagged achievement scores and student background characteristics are highly correlated with specifications that also include classroom characteristics, while value-added models (VAMs) estimated with school fixed effects have a lower correlation. Teacher effectiveness estimates based on median student growth percentiles are highly correlated with estimates from VAMs that include student background characteristics, despite the fact that the two methods for estimating teacher effectiveness are, at least conceptually, quite different. But even when the correlations between performance estimates generated by different models are quite high for the workforce as a whole, there are still sizable differences in teacher rankings generated by different models that are associated with the composition of students in a teacher's classroom.