Teacher Value-added Estimates Research Articles

Does Helping John Help Sue? Evidence of Spillovers in Education

Does the impact of teachers extend beyond the students in their classroom? Using the natural transitions of students from multiple elementary schools into a single middle school, this paper provides a new method for isolating and quantifying peer spillover effects of teaching and shows that ignoring these spillovers underestimates a teacher’s value by at least 30 percent. Because the spillovers also affect teacher value-added estimates, I develop a method of moments estimator of teacher value-added and show that accounting for the spillovers does not have a large impact on the ranking of teachers in New York City. I conclude by showing that the spillovers occur within groups of students who share the same race and gender, which suggests that social networks play a critical role in disseminating the effect. (JEL H75, I21, J15, J16, J45, Z13)

The Influence of Rapidly Guessed Item Responses on Teacher Value-Added Estimates: Implications for Policy and Practice

While most educators assume that not all students try their best on achievement tests, no current research examines if behaviors associated with low test effort, like rapidly guessing on test items, affect teacher value-added estimates. In this article, we examined the prevalence of rapid guessing to determine if this behavior varied by grade, subject, and teacher, and evaluated if rapid guessing influenced teacher value-added estimates. We observed differences in rapid guessing across grades, subjects, and teachers; however, this behavior did not appear to have a substantive effect on teacher value-added estimates. These findings suggest that rapid guessing occurs frequently enough that educators should be mindful of its effect on the interpretations of student test results. However, based on the value-added specifications used in this research, this type of behavior did not appear to affect estimates of teacher performance.

Two teacher quality measures and the role of context: evidence from Chile

This research examines empirically the relationship between two measures of teacher quality: one based on professional standards and a second one using teacher value-added estimates. It also studies the extent to which teacher observable characteristics, such as teacher training variables, are associated to better performance on either of these measures and whether any of these two assessments is able to effectively measure teacher quality isolated from the effect of the context where teachers work. Context in this article is defined as any variable that is not under the direct control of the teacher but plays an important role on student learning and we believe is captured by school and municipal variables. The study uses hierarchical linear models and information from national and standardized assessments from Chile, specifically from the municipal education sector. Results show a small correlation between the two measures of teacher quality, in the lower end of results from previous studies conducted in the USA, and suggest that there is only a limited relationship between both measures of teacher quality. Teacher initial education type and professional development were statistically associated only to the standard-based measure of teacher quality. Context (both the school and municipal levels) plays an important role in the teacher effect measure, and in the standard-based measures, therefore, we conclude that neither of these measures are context-free. We expect that these results will contribute to the discussion about how to best measure teacher quality and how to evaluate teacher performance both in Chile and other parts of the world.

Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels

ABSTRACTIt is common in the implementation of teacher accountability systems to use empirical Bayes shrinkage to adjust teacher value-added estimates by their level of precision. Because value-added estimates based on fewer students and students with “hard-to-predict” achievement will be less precise, the procedure could have differential impacts on the probability that the teachers of fewer students or students with hard-to-predict achievement will be assigned consequences. This article investigates how shrinkage affects the value-added estimates of teachers of hard-to-predict students. We found that teachers of students with low prior achievement and who receive free lunch tend to have less precise value-added estimates. However, in our sample, shrinkage had no statistically significant effect on the relative probability that teachers of hard-to-predict students received consequences.

Disentangling Disadvantage: Can We Distinguish Good Teaching From Classroom Composition?

This article investigates the use of teacher value-added estimates to assess the distribution of effective teaching across students of varying socioeconomic disadvantage in the presence of classroom composition effects. We examine, via simulations, how accurately commonly used teacher value-added estimators recover the rank correlation between true and estimated teacher effects and a parameter representing the distribution of effective teaching. We consider various scenarios of teacher assignment, within-teacher variability in classroom composition, the importance of classroom composition effects, and the presence of student unobserved heterogeneity. No single model recovers without bias estimates of the distribution parameter in all the scenarios we consider. Models that rank teacher effectiveness most accurately do not necessarily recover distribution parameter estimates with less bias. Since true teacher sorting in real data is seldom known, we recommend that analysts incorporate contextual information into their decisions about model choice and we offer some guidance on how to do so.

Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables

Teacher value-added models (VAMs) must isolate teachers’ contributions to student achievement to be valid. Well-known VAMs use different specifications, however, leaving policymakers with little clear guidance for constructing a valid model. We examine the sensitivity of teacher value-added estimates under different models based on whether they include student and peer background characteristics, and a double-lagged achievement score. We also consider two previously unexplored model variations: (a) replacing classroom peer characteristics with teacher-level averages, and (b) allowing demographics to influence the relationship between current and prior achievement. Using data from a northern state, we find that teacher effectiveness estimates are highly correlated across specifications. However, up to 26% of teachers in the bottom quintile using one specification are ranked higher using another specification. Differences between VAMs have direct implications for which estimates change. In particular, teachers in a district with a large fraction of disadvantaged students receive lower ratings when background characteristics are omitted. Other modeling choices have smaller practical consequences, and none are as important as selecting which assessment to use as the outcome measure.

Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates

Are teachers' impacts on students' test scores (value-added) a good measure of their quality? One reason this question has sparked debate is disagreement about whether value-added (VA) measures provide unbiased estimates of teachers' causal impacts on student achievement. We test for bias in VA using previously unobserved parent characteristics and a quasi-experimental design based on changes in teaching staff. Using school district and tax records for more than one million children, we find that VA models which control for a student's prior test scores provide unbiased forecasts of teachers' impacts on student achievement. (JEL H75, I21, J24, J45)

Value-added Teacher Estimates as Part of Teacher Evaluations: Exploring the Effects of Data and Model Specifications on the Stability of Teacher Value-added Scores

In this study we explored the effects of statistical controls, single versus multiple cohort models, and student sample size on the stability of teacher value-added estimates (VAEs). We estimated VAEs for all 5th grade mathematics teachers in a large urban district by fitting two level mixed models using four cohorts of student data. We found that student sample size had the largest effect on changes in teachers’ relative standing and designation into performance groups, while control variables affected VAEs only minimally. However, we also found that teacher VAEs showed a fair degree of stability; year-to-year correlations ranged between .62 and .66, and changes in teacher effectiveness systematically varied by teacher experience, with beginning teachers showing the largest improvements over the four years under study. Our results suggest that some model specifications are likely to produce teacher value-added scores that can reflect meaningful differences in teachers while we also found that other models might produce VAEs that might be unreliable.

Does the Precision and Stability of Value-Added Estimates of Teacher Performance Depend on the Types of Students They Serve?

This paper investigates how the precision and stability of a teacher's value-added estimate relates to the characteristics of the teacher's students. Using a large administrative data set and a variety of teacher value-added estimators, it finds that the stability over time of teacher value-added estimates can depend on the previous achievement level of a teacher's students.The differences are large in magnitude and statistically significant. The year-to-year stability level of teacher value-added estimates are typically 25% to more than 50% larger for teachers serving initially higher performing students compared to teachers with initially lower performing students. In addition, some differences are detected even when the number of student observations is artificially set to the same level and the data are pooled across two years to compute teacher value-added. Finally, the paper offers a policy simulation which demonstrates that teachers who face students with certain characteristics may be differentially likely to be the recipient of sanctions in a high stakes policy based on value-added estimates and more likely to see their estimates vary from year-to-year due to low stability.

Beyond Levels and Growth: Estimating Teacher Value-Added and its Persistence

The levels and growth achievement functions make extreme and diametrically opposed assumptions about the rate at which teacher inputs persist. I first show that if these assumptions are incorrect, teacher value-added estimates can be significantly biased. I then develop a tractable, cumulative model of student achievement that allows for the joint estimation of unobserved teacher quality and its persistence. The model can accommodate varying persistence rates, student heterogeneity, and time-varying teacher attributes. I implement the proposed methodology using schooling data from North Carolina, and find that only a third of the contemporaneous teacher effect survives into the next grade.