Contamination of Current Accountability Systems

Abstract

WE SEE nothing wrong with the essential notion of accountability in education. As public employees, educators should expect to be held accountable for their use of public funds. Nonetheless, the various state governments and now the U.S. Department of Education have implemented high-stakes achievement testing as the nearly singular approach to accountability. While these accountability efforts vary in a number of ways, virtually all are similarly flawed. We feel that a number of issues related to current accountability schemes need to be addressed. We tend to agree, for instance, with assessment researcher Robert Linn, who received the American Educational Research Association's career achievement award and noted in his acceptance: I am led to conclude that in most cases the instruments and technology have not been up to the demands that have been placed on them by high-stakes accountability. Assessment systems that are useful monitors lose much of their dependability and credibility for that purpose when high stakes are attached to them. The unintended negative effects of high-stakes accountability uses often outweigh the intended positive effects. (1) But we leave issues of the technical inadequacy of current assessment tools to others and turn, instead, to the contamination of accountability data--i.e., test scores--now widely used despite Linn's cautions. Here we argue that, in all our accountability systems, four sources of contamination--summer reading loss, retention in grade, test preparation, and testing accommodations--serve to undermine the reliability of estimates of student reading achievement, a common measure of school effectiveness. Unless accountability policies are substantially revised and current practices modified, estimates of school effectiveness will remain unreliable, and the public and policy makers will continue to be misled. UNDERESTIMATING SCHOOL EFFECTS BECAUSE OF SUMMER READING LOSS Most students in the U.S. attend school for roughly nine months each year, and summer vacation periods represent the longest break in schooling. What every experienced teacher knows about the effect of summer vacation on reading proficiency has also been documented by research. That is, some children backslide over the summer and return to school unable to read as well as they did just two months earlier. This summer setback is observed more often in children from low-income families. Harris Cooper and his colleagues conducted a meta-analysis of recent studies of summer reading loss. They reported, On average, summer vacations created a gap of about 3 months between middle- and lower-class students. (2) In other words, it was primarily poor children who lost ground in reading during the summer. We have linked this loss to the limited access many poor children have to books when school is not in session. (3) The problem here is that most accountability systems use annual achievement testing--most often a spring-to-spring testing plan. Such plans ignore the impact of summer reading loss on the achievement of poor children. And doing so produces lower estimates of school-related reading gains. Consider that teachers in high-poverty schools must, on average, teach until October before their students are performing at the level at which they performed the previous June! Summer reading loss is a major reason for the consistently lower reading achievement in high-poverty schools. Policy makers may thus mistakenly believe that schools that enroll many poor children are less effective in developing reading proficiencies than they actually are. High-poverty schools could be doing a relatively good job of teaching kids to read during the school year, but those same kids then lose a total of two or more years of reading growth across their elementary school career. …