Raising Achievement of At-Risk Students — Or Not

Abstract

LET ME BEGIN by sharing the conclusions of two recent articles on raising the achievement of at-risk students. Contrary to earlier work, the findings presented in this paper suggest that mandatory graduation tests do not have a positive impact on student Here's the second: This article suggests that we should not assume that low- achieving students will always react negatively to policies that place a strong emphasis on achievement. The majority of students in our sample responded positively to the policy. The former is from Brian Jacob and appeared in the summer 2001 issue of Educational Evaluation and Policy Analysis. The latter is from Melissa Roderick and Mimi Engel and appeared in the fall 2001 issue of the same journal. While these two conclusions from articles in the same journal are not necessarily contradictory, they certainly are not easy to interpret. If motivation is related to achievement and if students respond positively to policies designed to motivate them, as Roderick and Engel contend, then such policies - in the form of high-stakes tests in both articles - should lead to increased achievement. But Jacob didn't find any such increase. Let's look at each article in turn. First, Jacob, a professor at the Kennedy School of Government, Harvard University, is not concerned with the current crop of high-stakes tests. He is analyzing the impact of the craze known as MCT (minimum competency testing) that swept the nation in the late 1970s and 1980s. And for his analysis, Jacob is using the NELS (National Education Longitudinal Study) databases from 1988 and 1992. He divides his sample into states with and without minimum competency tests. Since these tests were presumed to affect lower-achieving students, he also examines the test scores for the bottom 10%, the bottom 20%, and the bottom 50% of students. Surveys of state standards from such organizations as the American Federation of Teachers, the Fordham Foundation, and Education Week typically award highest marks to states with lowest achievement as measured by test scores. (High-scoring states, not surprisingly, don't feel the need for tough standards.) Jacob finds an analog from the past in that states with minimum competency tests had lower achievement on the NELS reading and mathematics tests. Students in states that used the tests showed no more gain on the NELS tests than students in states without tests. After accounting for prior student achievement and a variety of other controls, graduation tests appear to have no effect on 12th-grade math or reading scores, he writes. That is the result for the whole population. And when Jacob breaks the data out by achievement levels, the test states and nontest states also show roughly equal gains. However, while state-mandated graduation tests had no overall effect, there were small (not statistically significant) negative effects on students in the bottom 10% and bottom 20% of the achievement distribution. States that used the tests had somewhat higher dropout rates but also served greater numbers of disadvantaged students. Students in the bottom 10% of achievement were 33% more likely to drop out in states with tests than in nontest states. For the bottom 20%, they were 25% more likely to drop out. However, more than 21% of the students in Jacob's sample did not even take the 12th-grade NELS tests: they had either dropped out by then or had unofficially stopped coming to school. Jacob lists criteria that lead to tests that have powerful incentives: * the tests must have real consequences, * the tests define achievement relative to an external standard, * the tests are organized by discipline and keyed to the content of specific courses, * the tests signal multiple levels of achievement, * the tests cover almost all students, and * the tests assess a major portion of what students are studying or expected to know. …