Instructional Insensitivity of Tests: Accountability's Dire Drawback

Abstract

LARGE-SCALE accountability tests have become increasingly important. They influence the deliberations of policy makers and affect the day-by-day behaviors of teachers in their classrooms. The premise underlying the use of these accountability tests is that students' test scores will indicate the quality of instruction those students have received. If students score well on accountability tests, we conclude that those students have been well taught. Conversely, if students score poorly on accountability tests, we believe that those students have been poorly taught. Furthermore, advocates of these tests make two assumptions: 1) that teachers who realize they are going to be judged by their students' test scores will try to do a better instructional job and 2) that higher-level authorities can take action to bolster the quality of instruction in schools or districts where test results indicate ineffective instruction is taking place. For either of these assumptions to make sense, the accountability tests being employed must actually be able to determine the effect of instruction on students' test scores. However, all but a few of the accountability tests now having such a profound impact on our nation's schools are instructionally insensitive. That is, they are patently unsuitable for use in any sensible educational accountability program. INSTRUCTIONAL SENSITIVITY A test's instructional sensitivity represents the degree to which students' performances on that test accurately reflect the quality of the instruction that was provided specifically to promote students' mastery of whatever is being assessed. In other words, an instructionally sensitive test would be capable of distinguishing between strong and weak instruction by allowing us to validly conclude that a set of students' high test scores are meaningfully, but not exclusively, attributable to effective instruction. Similarly, such a test would allow us to accurately infer that a set of students' low test scores are meaningfully, but not exclusively, attributable to ineffective instruction. In contrast, an instructionally insensitive test would not allow us to distinguish accurately between strong and weak instruction. Students' performances on most of the accountability tests currently used are more heavily influenced by the students' socioeconomic status (SES) than by the quality of teachers' instructional efforts. That is, such instructionally insensitive accountability tests tend to measure the SES composition of a school's student body rather than the effectiveness with which the school's students have been taught. Instructionally insensitive tests render untenable the assumptions underlying a test-based strategy for educational accountability. How can the prospect of annual accountability testing ever motivate educators to improve their instruction once they've realized that better instruction will not lead to higher test scores? How can officials accurately intervene to improve instruction on the basis of low test scores if those low scores really aren't a consequence of ineffective instruction? There is ample evidence that, instead of improving instructional quality, ill-conceived accountability programs can seriously diminish it. Teachers too often engage in curricular reductionism and give scant, if any, instructional attention to content not assessed by accountability tests. Too often teachers impose excessive test-preparation drills on their students and thereby extinguish the genuine joy those students should experience as they learn. And too often we hear of teachers or administrators disingenuously portraying students' test scores as improved when, in fact, no actual improvement has taken place. Yet, while the distinction between instructionally sensitive and insensitive accountability tests may be readily understandable and the classroom consequences of using instructionally insensitive accountability tests are all too apparent, it accomplishes little when educators complain, even profusely, about policy makers' reliance on the wrong kinds of accountability tests. …