An Ammunition Dump?

  • Abstract
  • Literature Map
  • Similar Papers
Abstract
Translate article icon Translate Article Star icon
Take notes icon Take Notes

WHEN EDUCATORS open their Fibber McGee's closets of past reforms, a pile of silver bullets will spill out onto the floor. For at least the past 20 years, the sense of public confidence in public education has been fed by an arsenal of such quick, often creative, but largely unsubstantiated remedies for turning schools and kids around. And the pile became much larger this fall. The highest-caliber bullet of all--the No Child Left Behind (NCLB) Act--landed on top of the pile. But it was not the only presumed salvation of the public schools to lose some of its luster recently. Collecting data about these failed silver bullets can often tell us more than we've known before. The first National Assessment of Educational Progress (NAEP) report on reading and math scores that fairly reflects changes under NCLB showed essentially that, so far, this silver bullet is pretty much of a dud. Reading scores were flat, and math scores were up only a bit. Despite statements by the Bush Administration that sought to put a happy face on the findings, the NAEP data showed that progress had been better before NCLB became law than it has been since it was enacted. The data also indicate something that few pundits seem willing to acknowledge. Since the passage of NCLB, about four years ago, changes in the demographics of the student population have accelerated, and schools all over the country are feeling the impact of more English-language learners than ever before. It is to the schools' credit that NAEP reading scores--in the early grades, at least --did not go down. Moreover, the very slight improvement in math scores could be attributed partially to the fact that math test items in the early grades, and even to some extent on the eighth-grade tests, do not require advanced reading skills. (Another possible explanatory factor is that the efforts of the National Council of Teachers of Mathematics have given us some semblance of national math standards and that these standards have been in various stages of implementation for more than 15 years.) The NAEP data also underscore the accident-of-birth unfairness built into state accountability systems. The erratic performance of students on state measures of achievement, compared to their performance on NAEP, demonstrates that students in some states are being held accountable for much higher levels of performance than students in other states. And no state has made much progress, as revealed in the NAEP scores. Some 20 states claim to have made progress on their own eighth-grade reading assessments between 2003 and 2005, but only three showed any progress at even the basic level on NAEP. And, according to the Thomas B. Fordham Foundation, only eight of 25 states that did announce progress on their own fourth-grade reading tests had more than one-third as much progress at the basic level on NAEP. As might be expected, proponents and opponents of charter schools took the very same data and came up with different analyses. The National Alliance for Public Charter Schools found them to be making more progress than other schools; the American Federation of Teachers found them to be making less, even after accounting for student-level racial/economic variables. Another silver bullet to land atop the pile of spent ones is the grandiose array of promises made by Edison Schools, Inc. In the immortal tradition of Edward R. Murrow's series You Are There, I was there when Tennessee entrepreneur Chris Whittle announced that he would launch a network of at least 100 new schools that would rival public schools in terms of technology and student achievement. …

Similar Papers
  • Research Article
  • Cite Count Icon 2
  • 10.1353/aad.2005.0032
Editorial: Progress Is Not Our Most Important Product
  • Jun 1, 2005
  • American Annals of the Deaf
  • Donald F Moores

Most people, when they take the time to think about it, would agree that, all in all, the world is improving and that our lives today are better than they were in the past for our parents and grandparents. We live much longer and enjoy better health. Technology has provided us access to each other and to information that was undreamed of a generation ago. If someone had tried back then to describe what Google provides at a touch of our fingertips, the response would have been disbelief. Diseases such as polio and rubella have been conquered. Research on human genetics will provide benefits across a broad spectrum. Socially, people in general appear to be more accepting of differences and ready to embrace diversity. Given overall trends, we are justified in a certain satisfaction about the human condition in general. However, it is also clear that progress does not move in a constant upward linear fashion, even if the overall trajectory is upward. There are peaks and valleys, starts and stops, breakthroughs and dead ends. The last century was the bloodiest in the history of humankind. The AIDS pandemic is threatening the fabric of some societies. Racial, ethnic, and religious hatreds threaten to explode into violence across the globe. The health of our children may be deteriorating, as evidenced by an upsurge in poor nutrition and obesity. Within the framework of education in general and education of deaf and hard of hearing children in particular, we still do not have any clear indication that the academic achievement of our children is better or worse than it was years ago. The U.S. Department of Education has issued positive reports that the No Child Left Behind (NCLB) regulations have already resulted in documented improvement in academic performance. These claims are not supported by the results of the annual National Assessment of Educational Progress (NAEP), a federally funded program. The NAEP data indicate little change over several years. I tend to give more credence to the NAEP results for one reason. The NAEP administers sets of tests at three different grade levels, with students in the 50 states, the District of Columbia, and other governmental units receiving the same grade-level tests. Analysis can be made across time and across states, and can address numerous other factors. The NCLB regulations do not call for uniformity across states, but allow each state to develop its own rigorous tests at different grade levels. Results are not comparable; it would be like comparing apples and oranges. It is not helpful, for example, to learn that 31% of eighth-graders scored at the proficient level or above on the Rhode Island mathematics test and 72% of eighth graders scored at the proficient level or above on the Virginia mathematics test. The tests have different standards and share no common frame of reference. Readers of some of my previous editorials and other writing may be aware that I believe a major lack of NCLB is that it did not address special education, in particular education of deaf and hard of hearing children, a grave omission in light of the fact that the federal government data report a figure of 6,000,000 school-aged children identified as disabled. The rationale given for this neglect was that the Individuals With Disabilities Education Act (IDEA) was being reauthorized and that it would follow the guidelines established by NCLB. Unfortunately, Congress did not pass the reauthorization, which was originally scheduled for 2002, until late 2004; and it was not signed by President Bush until January 2005, 3 years after he signed NCLB. It should be noted that one characteristic that NCLB and IDEA have in common is that they are both underfunded. When IDEA, then known as the Education of All Handicapped Children Act, was passed, President Ford hesitated to sign it because he believed that it promised more than it would deliver; specifically, federal support was designed to increase over a short period of time to provide 40% of all of the excess cost of educating each disabled child. …

  • Research Article
  • Cite Count Icon 39
  • 10.1177/003172170308500211
The Condition of Public Education
  • Oct 1, 2003
  • Phi Delta Kappan
  • Gerald W Bracey

This year's Bracey Report begins and ends with the item that will continue to dominate education news: No Child Left Behind. TO PRODUCE last year's report required me to sift through 21/2 drawers full of materials. This year, there were 31/2 drawers. Attention is being paid to public education; much of it is not benign. On the education news front, budgets and No Child Left Behind (NCLB) dominated. Because readers certainly know the budget situation in their own states, we begin with NCLB, in two parts. NCLB: The AYP Trap Last year's report said of No Child Left Behind, [It's] a trap, a Trojan Horse . . . choose your metaphor. These days, Americans speak mostly in war images, so I now call NCLB a weapon of mass destruction targeted at the public schools in a campaign of shock and awe, which, given the incredible underreaction of educators, I must conclude is working. I first expressed that sentiment in an article that no one would publish in April 2002. (The article can be found at www.america-tomorrow.com/bracey/EDDRA.) No one seemed to believe it then. They do now. At the April 2003 meeting of the American Educational Research Association (AERA), several employees of the U.S. Department of Education approached me to express dismay over NCLB and over working for an ideologically driven department. These are sad times, indeed. One career staffer declared that NCLB should be renamed: NCL-B.S. As all Kappan readers surely know by now, all schools must test all students every year in grades 3 through 8 in reading and math, with science to be added in 2006-07. Schools must demonstrate adequate yearly progress (AYP). For a school to show AYP, all ethnic groups, all major socioeconomic groups, English-language learners, and special education students must make AYP separately. Ninety-five percent of each group must be tested, and, if any one group fails to make AYP, the school as a whole fails. The official phrase is needs improvement, but headlines across the nation reveal how everyone actually thinks about it: Most Schools in Failing, Los Angeles Daily News, 24 July 2003; State Adds 544 Schools to Failing List -- for Now, Grand Rapids Press, 12 July 2003; It's Pass or Fail; All or Nothing, Raleigh News Observer, 13 July 2003. Schools must continue to make AYP until, by 2014, 100% of a school's students must score In last year's report, I contended that, while the law allowed each state to define proficient, that wouldn't last and that the NAEP (National Assessment of Education Progress) definition of would come to rule. I claimed that this would happen even though the NAEP achievement levels have been rejected as fundamentally flawed by everyone who has ever analyzed them. This includes the General Accounting Office, the Center for Research in Evaluation, Student Standards, and Testing (CRESST), the National Academy of Education, and the National Academy of Sciences. Even so, in his presidential address to AERA, Robert Linn, co-director of CRESST at the University of Colorado and UCLA, made no attempt to provide or use any definition of proficient other than that rendered by NAEP.1 Educational researchers now assume that proficient means NAEP proficient. Secretary of Education Rod Paige has said that he will use the discrepancy between the performance on NAEP and the performance on state tests to shame schools into better performance.2 Certainly those discrepancies will be there for Paige to point to, and in many instances they will be huge. To see why, we need only look at the Princeton Review's comparisons of performance on state tests with performance on NAEP.3 Ironically, the largest discrepancy is for Texas, the home state of Secretary Paige and President Bush. Texas declared 91% of its eighth-graders proficient in math. NAEP says only 24% of Texas students reach that level. Although there are some states where the differences are small and even four states that have tougher requirements than NAEP (Arizona, Louisiana, Missouri, and Maine), the average difference is 23%, and for 22 states, the difference exceeds 30%. …

  • Research Article
  • Cite Count Icon 2
  • 10.1177/003172170708800714
Should NAEP Performance Standards Be Used for Setting Standards for State Assessments?
  • Mar 1, 2007
  • Phi Delta Kappan
  • James Pellegrino

The National Assessment of Educational Progress is considered the gold of assessment. However, Mr. Pellegrino warns, those who are setting proficiency standards for state tests to meet the requirements of NCLB should be wary of using NAEP's achievement levels as a guide. ********** CONFUSION and controversy frequently surround the process of setting performance standards for a state's high-stakes achievement assessments. This is especially problematic because performance data from such tests are used to meet the adequate yearly progress (AYP) provisions of the No Child Left Behind (NCLB) legislation. In light of these pressures, there is a natural tendency for policy makers, the press, and the public to use the National Assessment of Educational Progress (NAEP) for comparisons and guidance. There are two primary reasons why NAEP, rightly or wrongly, is often singled out in this way. First, NAEP is considered the gold of educational assessments, largely because of its history as a national indicator and the quality and care that have gone into its design and development. Second, because NAEP is seen as a high-quality indicator of academic achievement, its performance standards are perceived to have greater rigor and validity than those set for many other assessments, including the achievement tests developed by individual states. NAEP results have been used for comparisons to recent state achievement test results and, appropriately or inappropriately, they will probably be used to evaluate standards-setting processes for those states that have developed new assessments or have expanded the number of grades assessed. Typically, such changes in a state's assessment program are needed to meet the requirement, which took effect in 2006, to test all students in grades 3-8 in both mathematics and reading. To make this discussion a bit more concrete, consider as an example Wyoming's new state test known as PAWS (Proficiency Assessment for Wyoming Students). PAWS is quite different in purpose and design from Wyoming's prior test, known as the WyCAS. Given that performance standards for PAWS need to be established and that those standards are likely to be scrutinized carefully both within Wyoming and at the federal level, it is reasonable to ask whether NAEP provides appropriate standards for Wyoming or any other state in which new standards must be set. To make such an appraisal, one needs to understand three things. First, setting standards is a judgment carried out by reasonable people, and it occurs in a social and political context. As such, it is influenced by multiple factors, including the nature of the assessment itself, current goals and aspirations for the educational enterprise, practical considerations, sources of comparative data such as NAEP, and immediate social consequences. The second thing that bears consideration is the history of and motivation for establishing the high performance standards that have become NAEP's trademark and that serve as the standard for comparison. The third thing to consider is the validity of NAEP's standards--do they mean what people think they mean--and whether those standards are applicable to the interpretation of levels of performance by a state's students on its NCLB high-stakes achievement test. UNDERSTANDING NAEP STANDARDS AS EDUCATION POLICY STATEMENTS Although NAEP has been operational since the late 1960s, the practice of reporting NAEP results in terms of performance standards is little more than 15 years old. And in that short time, the labels Basic, Proficient, and Advanced have become part of the testing landscape, adopted not only by NAEP but also by many state testing programs, including Wyoming's WyCAS and now PAWS. The genesis of reporting performance standards can be traced to the 1989 education summit in Charlottesville, Virginia. The summit participants, including President George H. …

  • Book Chapter
  • 10.1007/978-1-4020-8427-0_16
Predicting Group Membership using National Assessment of Educational Progress (NAEP) Mathematics Data
  • Jan 1, 2009
  • David A Walker + 1 more

Since 1969 in the United States, the federally mandated National Assessment of Educational Progress (NAEP) has been used to assess the condition of student learning at the state and national levels, in particular subject areas, and in specific grades and/or ages (US National Center for Educational Statistics [NCES], n.d.-a). In the 1990s, academic achievement in various disciplines derived from NAEP scores were tracked by state and measured via the percentage of students at or above the levels established in a three-tiered scoring model: basic, proficient, or advanced (Hombo, 2003). In concert with the No Child Left Behind Act of 2001 (NCLB, 2002), the US Congress decreed NAEP as the Nation’s Report Card, to be used to indicate student-achievement score trends in academic areas such as mathematics, reading, science, and history among states in nationally representative samples (US NCES, n.d.-a). Results derived from NAEP data aggregated at the state level pertaining to student achievement—which in the latter years of its nearly 40-year existence have been employed as a means toward ascertaining accountability affiliated with high-stakes testing, such as NCLB—have yielded mixed interpretations and perceptions (see the American Educational Research Association, 2000, definition for a standard interpretation of the term high-stakes testing). For example, Hanushek and Raymond (2006) found that NAEP mathematics scores provided some positive evidence for accountability, whereas Amrein and Berliner (2002) determined the contrary with various state-level NAEP scores in certain academic areas. Darling-Hammond (2007) contended that NAEP, in areas such as mathematics, did not measure higher-order cognitive domains but instead “measures less complex application of knowledge” (p. 319). This lack of NAEP’s application of knowledge via measuring—for instance, problem-solving skills—is linked to multiplechoice-type tests used for accountability purposes under NCLB that do not allow for the assessment of student achievement with the following question: “What can students do with what they have learned?” (p. 319).

  • Research Article
  • Cite Count Icon 16
  • 10.1177/003172170408500804
Responding Effectively to Test-Based Accountability
  • Apr 1, 2004
  • Phi Delta Kappan
  • Laura Hamilton + 1 more

Taking NCLB at its word, Ms. Hamilton and Mr. Stecher look beyond each year's release of test scores and consider how to make the new law work more effectively as a tool for educational improvement. THE NO CHILD Left Behind (NCLB) Act of 2001 has focused the attention of educators, policy makers, and the public on accountability for performance in public education. Yet many of those who will be responsible for improving school performance lack guidance on how to proceed in the brave new world of NCLB accountability. For the most part, state education department and district staff members, principals, teachers, and parents have spent their time trying to understand what the law means and how it will affect them. They have had to react quickly -- and sometimes without much direction -- to those provisions of the law that have already gone into effect. Few have had time to look ahead and make plans for operating effectively in this new environment. This article is intended to offer some strategies that will help educators function better in the world of accountable districts and accountable schools. NCLB mandates the implementation of accountability systems that include standards, assessments, annual progress goals, and incentives. States have options about how some of these components are constituted, but all must be present. Together these components make up what we call a test-based accountability system. The system is designed to improve school outcomes by sending clear signals about expected performance (standards), measuring student achievement against those standards (assessments), comparing performance to increasingly tough targets (annual progress goals), and establishing rewards and sanctions to change behaviors and promote improvement (incentives). NCLB mandates accountability for student outcomes and is designed to give states, districts, and schools flexibility over the educational process. The core of NCLB is an annual feedback loop in which information about the attainment of progress targets triggers incentives that, in theory, reinforce positive practices (those that lead to student achievement) and sanction negative ones. The law requires students to be taught by qualified teachers using proven practices, but beyond that, the intention is for local educators to have control over curriculum, instruction, school organization, and other features of schooling. To many observers outside the education system, emphasis on accountability seems like a logical approach to ensuring good performance on the part of educators. At the same time, those who work in schools, districts, and state departments of education have expressed concerns that the law imposes unrealistic targets and will prove detrimental to the quality of public education. There is research to support the views of both advocates and critics. On the positive side, schools, teachers, and students seem to respond to the incentives created by accountability systems, and scores on state tests typically rise after these systems are introduced. There is also evidence that scores on some external tests, such as the National Assessment of Educational Progress (NAEP), may rise when states implement accountability systems.1 On the negative side, higher test scores do not necessarily reflect real gains in student mastery of content standards; they may, for example, reflect students' learning of particular test content or formats. Even when NAEP scores rise, the gains tend to be many times smaller than those on the state test of the same subject matter.2 In addition, test-based accountability sometimes leads to changes in curriculum and instruction that are not necessarily desirable.3 Regardless of whether the research to date is interpreted as positive or negative, educators must learn to operate in the environment of accountability defined by NCLB. They must find ways to respond to the law and provide the best services possible to the students they serve. …

  • Research Article
  • Cite Count Icon 4
  • 10.1177/003172170808900816
Science Education: Cassandra's Prophecy
  • Apr 1, 2008
  • Phi Delta Kappan
  • Thomas E Brady

AFTER A Nation at Risk was released, the state of American education was widely discussed, and not just by educators. The 1980s produced a number of reports on the status of science education that complained of declining science and mathematics achievement, falling enrollment in the subjects, and a shortage of qualified teachers. All the attention sparked renewed federal interest in science and mathematics education, school/industry coalitions were established, and a call for help was issued to all interested parties. Business and industry, concerned about the scientific literacy of their current and future employees, offered to assist with money, training, equipment, support for competitions, and other cooperative efforts. While the new reports were troubling in their own right, in light of the Sputnik-associated science education reforms of a generation earlier, they were downright disturbing. We had all been warned that our system of science education wasn't working well, but, in spite of a national effort, it seemed that the message--like Cassandra's prophecy of the fate of Troy--was going unheeded. Concern about science education and science standards has usually been driven by worries that American students are falling behind their international peers in achievement. In the case of Sputnik, the international touchstones were students in the Soviet Union. In the years immediately following A Nation at Risk, students in Japan and Germany were the comparison groups. Test scores, as always, were the comparative measure. And the concerns seemed at least partly justified by scores on the National Assessment of Educational Progress (NAEP). Eighty-two percent of the nation's 12th-graders performed below the proficient level on the 2000 NAEP science test. Moreover, in recent years, business leaders such as Bill Gates, chairman of Microsoft, have called for more emphasis on science education, arguing that the U.S. risks losing its economic edge if it does not change. Public opinion surveys, on the other hand, indicate most U.S. parents are satisfied with science education. Their level of concern has actually declined in recent years. (1) The current emphasis on content standards, which has its roots in the years immediately following the release of A Nation at Risk, was strengthened with the passage of the No Child Left Behind (NCLB) Act. Among its many mandates, NCLB required that, beginning in the 2007-08 school year, schools must administer annual tests in science achievement at least once during three grade spans: 3-5, 6-9, and 10-12. Yet despite its own requirement, the law has succeeded in pushing science to the back burner. Schools now focus on reading and mathematics, with little time left for science in the rush to prepare teachers and students for high-stakes standardized tests. In April 2002, shortly after NCLB was passed but before any of its mandates had taken effect, Olaf Jorgenson and Rick Vanosdall explained in these pages that some districts were already so fixated on basic-skills preparation that students were spending 20% of total class time on test preparation. When NCLB took effect, they speculated, things would only intensify. (2) Yet the predicted failure of science education to meet the demands of the modern world seems to continue to be ignored. In spite of efforts to keep science from becoming a second-class discipline, the science scores on the most recent NAEP indicate that science achievement in the U.S. is at best in a holding pattern. Scores were up slightly in 2005 over both 1996 and 2000 at the fourth-grade level; they held steady for both comparison years at the eighth-grade level; and they showed a slight decline from 1996 at the 12th-grade level, but not from 2000. GATHERING THE WHOLE VILLAGE The continued reports of the unsatisfactory state of U.S. science education and the predicted consequences for our nation's economic and intellectual vitality attracted the attention not only of educators and politicians but also of an increasing number of professional scientists and engineers. …

  • Research Article
  • Cite Count Icon 33
  • 10.1080/00220671.2014.993461
Academic growth trajectories of ELLs in NAEP data: The case of fourth- and eighth-grade ELLs and non-ELLs on mathematics and reading tests
  • May 27, 2016
  • The Journal of Educational Research
  • Nihat Polat + 2 more

ABSTRACTUtilizing the National Assessment of Educational Progress (NAEP) data, this study examined (1) how fourth and eighth-grade ELLs' mathematics and reading scores on national tests compared to their non-ELL peers' scores over the testing period between 2003 and 2011, and (2) if gender and ethnicity contributed to variation in the growth patterns among the student groups across grade levels and content areas. Since the NAEP data, which provides a national sample of 10,000–20,000 students, is collected using a probability sample design, sampling weights are adjusted so inferences can be appropriately made. Sample sizes within NAEP are large enough to generate adequate power for statistical significance. Thus, to display the data in a multivariate mode, Tableau 8.0.0 software was used. Results suggested that the achievement gap between non-ELLs and ELLs is either steady or slightly widening in both mathematics and reading, with multiple paths across the content areas, grade levels, and gender and ethnic groups.

  • Research Article
  • Cite Count Icon 2
  • 10.1177/003172170809000118
Performance at the Top
  • Sep 1, 2008
  • Phi Delta Kappan
  • Gerald W Bracey

Anumber of recent studies have indicated that many districts focus on the bubble or the tippers--kids who with some extra instruction might tip the school into achieving Adequate Yearly Progress (AYP). Jennifer Booer-Jennings found that teachers were told to ignore hopeless cases and sure things (see the December 2005 Research column, Blowing Bubbles in Texas) in favor of improving the scores of kids on the bubble. Does the focus on making AYP and lowering the achievement gap mean gifted kids are losing out? Logically, they can't make the same progress as low-achieving kids or else the gap will never close. Three recent studies throw some light on the issue. Loveless Study of NAEP Scores Tom Loveless of the Brookings Institution examined National Assessment of Educational Progress (NAEP) data in reading and mathematics at grades 4 and 8 for the 10th and 90th percentiles. This column focuses only on fourth-grade data. Among fourth graders, Loveless found that between 2000 and 2007 the 10th percentile scores in math rose 13 points, about a year of achievement growth, while scores for the 90th percentile kids rose only 5 points. Why look at test scores in 2000 when NCLB didn't take effect until 2002? Loveless says that by 2000, school people knew something like NCLB was coming and, practically, there was no later assessment until 2003, over a year after NCLB became law. But recent stories indicate states made plans to achieve 100% proficiency that required little effort in the early NCLB implementation. This is important because for the 10th percentile students, 7 points of the 13-point gain came in that interval between 2000 and 2003. It's impossible to say when the gains occurred, but NCLB seems unlikely to be responsible for them. Fourth-grade reading scores are even more problematic. The 90th percentile score went up only 3 points over a seven-year period. The 10th percentile rose 16 points, but most of that-12 points--occurred between 2000 and 2002-again, before NCLB was actually in place. Loveless says that he uses a conversion: 11 NAEP scale points = one year of academic growth. Some researchers use 10, some 12, but if we take Loveless' 11, the difference in reading between the 10th percentile and the 90th percentiles in 2007 was 90 points, or 7+ years of growth. This is pretty close to the results in 1990 when the difference was 82 points. These figures become more ominous when we look at the data between 1990 and 2000. For math, the 90th percentile rose 12 points and the 10th percentile 13. In reading, the 90th percentile rose by only one point, but the 10th percentile lost 11 points. The 10th percentile bottomed over this time in 1994, and the score is up 1 point from 2000. So, for reading at least, NCLB has not acted as Robin Hood, taking from the rich and giving to the poor. However, the data overall do seem to corroborate an unpublished report from Marshall Smith of the Hewlett Foundation that NAEP scores have grown more slowly after NCLB became law than before it. Loveless uses the average per year gain and thinks it's faster after NCLB. But as I pointed out, most of the gains came before NCLB was really in place. I argue that the trend line paints a better picture than average gain or loss per year. States that were deemed to have accountability programs showed more gains in math for their 10th percentile than for their 90th percentile. Non-accountability states showed little difference in gains. In both accountability and non-accountability states, the 10th percentile rose more than the 90th. The 90th percentile showed little change in either. The 10th percentile included more black students (36.9%) and about equal numbers of white (28.4%) and Hispanic students (29.8%). The 90th percentile students were largely white. Only 16.1% of the teachers teaching 90th percentile kids had 0-4 years of experience, compared to 29. …

  • Book Chapter
  • 10.1093/obo/9780199874002-0279
National Assessment of Educational Progress (NAEP) in Geography Education
  • Oct 26, 2023
  • Michael Solem + 1 more

The National Assessment of Educational Progress (NAEP) is a large-scale school-based study of student achievement in K-12 education in the United States. NAEP assessments are designed to be representative of the demographic, socioeconomic, and geographic diversity of the United States. Geography was included from 1994 to 2018. In 2019, the National Assessment Governing Board announced geography was being eliminated from the next decade-long assessment cycle along with economics, foreign language, and art. There have been five NAEP Geography assessments, three of which assessed the geographic knowledge of students in the fourth, eighth, and twelfth grades in 1994, 2001, 2010 and only at the eighth-grade level in 2014 and 2018. The NAEP program also collects contextual data to further provide information about the students, teachers, and schools participating in each assessment. After the assessment has been completed, NAEP shares the data publicly in reports known as the “NAEP Report Card.” Additionally, NAEP offers a variety of digital tools to support research regarding student achievement, including the NAEP Data Explorer, NAEP Item Maps, and released assessment items. Researchers may apply for a data license from the Institute of Education Sciences to obtain raw, restricted-use data files containing individual responses to NAEP assessments for advanced statistical analyses of student achievement. NAEP data have been leveraged by researchers to analyze relationships between student achievement and a variety of educational factors (e.g., instructional exposure, teacher quality, classroom technology) and noneducational factors (e.g., student and school socioeconomic status, school neighborhood effects). A comprehensive statistical analysis of NAEP Geography assessments at the eighth-grade level produced a predictive model that comprised of student- and school-level variables. Student characteristics including race, gender, and eligibility for free- or reduced-price lunches were consistently predictive of geography scores, conditional on all covariates. In contrast, school attributes including school type (e.g., private or public school); U.S. regional location (e.g., Southeast, West); and urbanicity (e.g., suburb, rural) were not statistically significant predictors on most assessments. Subsequent research involving NAEP Geography data has explored relationships between geography achievement and differences in students’ opportunity to learn geography, including disparities in students’ access to experienced geography teachers and exposure to high-quality instruction and learning activities.

  • Research Article
  • Cite Count Icon 176
  • 10.1177/003172170308500109
Teaching Democracy: What Schools Need to Do
  • Sep 1, 2003
  • Phi Delta Kappan
  • Joseph Kahne + 1 more

Mr. Kahne and Mr. Westheimer studied 10 educational programs whose objective -- to develop democratic citizens -- is largely ignored by school reform policy. Schools can fulfill this mission, they discovered, through specific strategies that promote civic commitments, capacities, and connections. WHICH OF THE following headlines never appeared in a daily newspaper? a. Capital City Students Show No Gain in Reading, Math -- Governor Threatens Takeover b. Middletown Schools to Be Taken Over by State for Failure to Develop Democratic Citizens If you answered b, you not only answered correctly, your response also reflected an important challenge facing our democracy today. While we say that we value a democratic society, the very institutions expected to prepare democratic citizens -- our schools -- have moved far from this central mission. There is now frequent talk of state takeovers of schools that fail to raise test scores in math or reading, but it is unimaginable that any school would face such an action because it failed to prepare its graduates for democratic citizenship. The headlines we read are about test scores, basic skills, and the role schools play in preparing students for jobs in the Information Age. The vast bulk of school resources are going to literacy, mathematics, science, and vocational education. In 2003, for example, federal expenditures by the Department of Education on civic education totaled less than half of one percent of the overall department budget.1 And when it comes to assessment, civic goals get very little attention. The No Child Left Behind (NCLB) Act mandates yearly testing in math, reading, and, beginning in 2005, science. Social studies and civic education, the areas of the curriculum most tied to the democratic mission of schools, share no such requirements. Similarly, the National Assessment of Educational Progress (NAEP), which is often referred to as the Nation's Report Card, measures performance in math and reading annually -- but administers a civics assessment only about once every 10 years. Clearly, math, reading, and science are important, but, from the standpoint of supporting a democratic society, academic subject matter, when disconnected from its social relevance, is insufficient. The same can be said of colleges and universities. Their commitments to democratic priorities are more rhetorical than substantive. We are concerned here with what is not being discussed in the newspapers. We are concerned that the great school debates of our time give short shrift to a fundamental principle that, for more than two centuries, informed efforts to advance the notion of public schooling. This article addresses an important gap in our education agenda: preparing students to be effective democratic citizens. For the past three years, we studied 10 educational programs, funded by the Surdna Foundation, that were unusual in that they put the challenge of educating for democratic citizenship at the center of their efforts.2 We studied 10th-graders evaluating a juvenile detention center, ninth-graders studying the feasibility of curbside recycling, and 11th-graders reporting to the public on the availability of affordable housing in their community. We examined programs that exposed university students to community development projects in Silicon Valley, brought theology majors to a reservation to study the history of Native American experience, and led students interested in social movements on an intensive journey through historical sites of the civil rights movement. We visited an adult education program with a 70-year history of working for social and economic change through education and democratic action. All in all, we interviewed dozens of instructors and students, administered more than 500 surveys, observed pedagogical practices, and examined portfolios of student work. These programs share an emphasis on helping students to identify and act on issues of importance to themselves and to society. …

  • Research Article
  • Cite Count Icon 1
  • 10.1177/003172170008200202
The Notorious G-A-P
  • Oct 1, 2000
  • Phi Delta Kappan
  • Anne C Lewis

THE GAP is back. Actually, the differences between the achievement of white and Asian students and that of black, Hispanic, and other minority students never completely disappeared, but so much progress was made during the 1970s and 1980s that there seemed to be a tacit assumption that the gap would eventually close altogether. The standards and assessment efforts of the past decade were at least partly meant to speed up the process of closing the gap. If districts, schools, and parents are chafing under accountability measures (often hastily and thoughtlessly implemented), it is because state policy makers were under the gun to find ways to close the gap even further. Federal laws, especially Title I, said that all students were to be held to the same high standards, a mandate that rippled through state policies for all districts and schools. The requirement that assessment data be disaggregated by race and ethnicity also ensured that the problem of the gap would remain public. Indeed it did, especially in a rash of reports released as the new school year began, including the annual release of SAT scores from the College Board. One who believes, as I do, that the standards-based movement is essential to closing the gap, must conclude that not enough schools are using standards-based reforms to give low-performing students, especially minorities, access to the same curriculum as everyone else. The latest trend report from the National Assessment of Educational Progress (NAEP) tells the story of the gap for the past 25-plus years. And it was a good story for a long time. Between 1970 and 1988, the gap between the NAEP reading scores of black students and those of white students was cut in half; the math score gap, cut by one-third. Obviously, schools were doing something right, and black students were showing they could indeed learn when given greater opportunities to do so. Then in the 1990s, the progress stopped. Actually, it stopped for everyone (at slightly higher levels than in the 1970s), but black and Hispanic NAEP scores climbed aboard a roller coaster. They fell back, began to improve in some subjects, then regressed again. By 1999, the scores of black and Hispanic students were far below the records set in 1988. Trying to put a good face on a disturbing picture, NAEP officials and Secretary of Education Richard Riley pointed to steady improvements overall in math scores and higher rates of enrollment in more advanced math classes. The same arguments were heard with the College Board's release of the 2000 SAT scores: last year's seniors posted the highest math scores since 1969; verbal scores were unchanged. The percentage of minorities taking the SAT has increased 47% since 1987 and would be expected to pull the average scores down, so this was good news. However, the gap remains very wide, with blacks scoring 104 points lower than whites on the math portion of the SAT and 94 points lower on the verbal portion. These score gaps on the SAT actually widened during the 1990s. So what happened? At a time when state statutes, a general consensus about content standards, improved professional development, greater knowledge from research on learning, and pressure for accountability have been working more or less in tandem, why have the academic gaps grown larger? Researchers are scratching their heads on this one. David Grissmer of RAND points to the unprecedented resources made available to low-income schools by the War on Poverty as one factor in narrowing the gap and to a rising black middle class as another. However, he couldn't explain why the progress ended. His most recent research, described in Improving Student Achievement (RAND, 2000), uses the improvement of fourth- and eighth-grade NAEP math scores as the context for making state comparisons. The rise in scores in some states, he says, is a result primar-ily of both higher per-pupil expenditures and how the money is spent. …

  • Research Article
  • Cite Count Icon 372
  • 10.1002/pam.20586
The impact of no Child Left Behind on student achievement
  • May 31, 2011
  • Journal of Policy Analysis and Management
  • Thomas S Dee + 1 more

The No Child Left Behind (NCLB) Act compelled states to design school accountability systems based on annual student assessments. The effect of this federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state‐level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test‐score changes across states that already had school accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of fourth graders (effect size 5 0.23 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in eighth‐grade math achievement, particularly among traditionally low‐achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased fourth‐grade reading achievement. © 2011 by the Association for Public Policy Analysis and Management.

  • Single Report
  • Cite Count Icon 210
  • 10.3386/w15531
The Impact of No Child Left Behind on Student Achievement
  • Nov 1, 2009
  • Thomas Dee + 1 more

The No Child Left Behind (NCLB) Act compelled states to design school accountability systems based on annual student assessments. The effect of this federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state-level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test-score changes across states that already had school accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of fourth graders (effect size 5 0.23 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in eighth-grade math achievement, particularly among traditionally low-achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased fourth-grade reading achievement. © 2011 by the Association for Public Policy Analysis and Management.

  • PDF Download Icon
  • Research Article
  • 10.29121/granthaalayah.v8.i11.2020.2404
EXAMINING DUAL ACADEMIC CREDIT AND THE LOCATION OF INSTRUCTION AS A PREDICTOR OF 12TH GRADE NAEP READING SCORES
  • Dec 16, 2020
  • International Journal of Research -GRANTHAALAYAH
  • Mingyuan Zhang + 1 more

This study explored 12th grade reading scores on the 2015 National Assessment of Educational Progress (NAEP) and how they relate to enrollment in dual credit college courses and the location in which they were taught. To gain a better understanding of dual credit attainment through various instructional delivery methods and the prediction of 12th grade reading scores, data was mined from the 2015 NAEP and presented in this descriptive research study. The findings of this study include for 12th grades students: (1) Students who earned ELA dual college credit delivered at their high school campus did not result in higher NAEP reading scores. (2) Students who earned ELA dual college credit delivered at a postsecondary campus did not result in higher NAEP reading scores. (3) Students who earned ELA dual college credit delivered through distance learning had higher NAEP reading scores when 11-25% of the 12th grade students enrolled. These findings make evident the high school campus or postsecondary campus dual credit courses for 12th grade students did not have an impact or can be used as a predictor on the NAEP reading test. The dual credit distance learning did have a positive impact on the 12th grade NAEP reading scores, specifically if a smaller percentage of students at the school enrolled, identifying that increasing access to dual credit may not always translate to increased college readiness and rigor.

  • Research Article
  • Cite Count Icon 172
  • 10.2307/3211180
A Closer Look at Black-White Mathematics Gaps: Intersections of Race and SES in NAEP Achievement and Instructional Practices Data
  • Jan 1, 2002
  • The Journal of Negro Education
  • Sarah Theule Lubienski

Drawing from the 1990, 1996, and 2000 National Assessment of Educational Progress, this study examines Black-White disparities in 4th-, 8th-, and 12th-grade mathematics achievement and instruction. Substantial Black-White achievement gaps were identified, such as 12th-grade Black students scoring below 8th-grade White students. Furthermore, an analysis of race and SES together in the 1996 data revealed that student SES failed to account for much of the Black-White achievement gaps. Several instruction-related factors were also found to differ by race even after controlling for students' SES. This study provides evidence that, despite current reforms promoting highquality mathematics education for all, Black students of both low and high SES are being left behind. Although achievement gaps between White and Black' students narrowed in the 1970s and 1980s (Tate, 1997), these gaps now appear to be widening (Campbell, Hombo, & Mazzeo, 2000; Jencks & Phillips, 1998; Lee, 2002). Recent examinations of Black-White achievement gaps have tended to focus on students' overall school achievement and experiences. In contrast, this article focuses specifically on students' achievement and learning experiences in a subject that is used as a key gatekeeper in U.S. society: mathematics. One important tool for monitoring mathematics achievement gaps is the National Assessment of Educational Progress (NAEP). The NAEP is the only nationally representative, ongoing assessment of academic achievement in the United States. The NAEP measures student performance at grades 4, 8, and 12 in mathematics and other subject areas. The NAEP also provides information from student and teacher questionnaires regarding mathematical backgrounds, beliefs, and instructional practices. This study uses NAEP data to examine the nature of Black-White mathematics achievement gaps, including the extent to which such gaps appear linked to student socioeconomic status (SES) differences, as measured by parent education level and literacy resources in the home.3 Given that substantial portions of the Black-White gaps were not accounted for by SES differences, the study also examines the extent to which these gaps might be related to differences in the instruction students received. Specifically, this study addresses the following questions: 1. How large were the Black-White gaps in mathematics achievement at grades 4, 8, and 12, as measured by the 1990, 1996, and 2000 NAEP? 2. To what extent were the Black-White gaps attributable to differences in students' socioeconomic status? 3. For which instruction-related variables were there Black-White differences that were not attributable to student SES differences, and, therefore, could underlie Black-White achievement gaps that persist after controlling for SES? The 1990 assessment was chosen as a starting point for examining achievement trends because it was the first assessment aligned with current mathematics education reforms (National Council of Teachers of Mathematics [NCTM], 1989, 1991, 2000). The 2000 assessment is the most recent for which general mathematics achievement data are available. However, the 1996 NAEP data set is used to answer questions about SES in conjunction with race, because that is the most recent raw data available. REVIEW OF LITERATURE Several examinations of Black-White achievement gaps have been published recently. Perhaps the two most famous examples are Hernstein and Murray's (1994) The Bell Curve, and Jencks and Phillips' (1998) The Black-White Test Score Gap. Whereas Hernstein and Murray make the controversial claim that achievement gaps could be genetically based, Jencks and Phillips argue that achievement gaps can be closed with proper attention to issues related to schooling (e.g., class size, teacher competency requirements) and family supports. In a recent review, Lucas (2000) noted that both of these books describe statistical attempts to reveal whether Black-White achievement gaps are due to genetic or societal factors. …

Save Icon
Up Arrow
Open/Close
  • Ask R Discovery Star icon
  • Chat PDF Star icon

AI summaries and top papers from 250M+ research sources.