Abstract
ABSTRACTValue-added models (VAMs) of student test scores are used within education because they are supposed to measure school and teacher effectiveness well. Much research has compared VAM estimates for different models, with different measures (e.g., observation ratings), and in experimental designs. VAMs are considered here from the perspective of graphical models and situations are identified that are problematic for VAMs. If the previous test scores are influenced by variables that also influence the true effectiveness of the school/teacher and there are variables that influence both the previous and current test scores, then the estimates of effectiveness can be poor. Those using VAMs should consider the models that may give rise to their data and evaluate their methods for these models before using the results for high-stakes decisions.
Highlights
Estimating causal effects is important within education
Value-added models (VAMs) of student test scores are used within education because they are supposed to measure school and teacher effectiveness well
From Pearl’s rules, it is predicted that the backdoor path VA ← um1 → pre ← um2 → post will be problematic because conditioning on pre unblocks this path: Rule #2 from Pearl (2009, pp. 16–17)
Summary
Estimating causal effects is important within education. This includes, in many countries, using students’ test scores to estimate the relative causal effects of schools and teachers. A statistical model often used to make these estimates is called the value-added model (VAM) by education researchers in the U.S This name is perhaps unfortunate because it presumes that the procedure accurately measures the causal impact of schools and teachers whenever the model is applied. Using test scores to evaluate schools and teachers is one of the most important and hotly debated policy issues in education because the results from VAMs influence high-stakes decisions. There has been much research comparing VAM estimates from (often slightly) different statistical models, comparing VAM estimates with other measures like observational ratings, and comparing VAM estimates within studies that use random assignment (reviews include: Raudenbush and Willms 1995; Goldstein and Spiegelhalter 1996; Raudenbush 2004; Braun 2005; Baker et al 2010; Wainer 2011; Foley and Goldstein 2012; McCaffrey 2012; Castellano and Ho 2013; Haertel 2013; Amrein-Beardsley 2014; Goldhaber et al 2014; Goldstein 2014; Koedel et al 2015)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
