National Assessment Of Educational Progress Reading Research Articles

Closing Reporting Gaps: A Comparison of Methods for Estimating Unreported Subgroup Achievement on NAEP

ABSTRACT The National Assessment of Educational Progress (NAEP) program only reports state-level subgroup results if it samples at least 62 students identifying with the subgroup. Since some subgroups constitute small proportions of many states’ general student populations, these minority subgroups are seldom sufficiently sampled to meet this sample size requirement. Consequently, education researchers and policymakers are regularly left without a comprehensive understanding of how states are supporting the learning and achievement of different subgroups of students, including underserved subgroups. Using grade 8 mathematics results in 2015, this study compares the performance of two separate techniques in predicting mean subgroup achievement on NAEP. Results demonstrate that a small area estimation technique, known as the Fay-Herriot (FH) model, represents a promising approach. Follow-up research should employ the FH technique with math achievement data from other testing years, as well as data from NAEP reading to evaluate the technique’s generalizability.

The Intersection of Language and Disability Progress of English Learners With Disabilities on NAEP Reading

Despite evidence of unequal learning opportunities, there is limited understanding of how English learners (ELs) with disabilities are faring academically nationwide in comparison with their EL and student with disability counterparts. In response, we conducted secondary data analyses of the 2009 to 2019 National Assessment of Educational Progress (NAEP) reading assessments. Compared to their peers, ELs with disabilities performed lower in both Grades 4 and 8. There is a visible improvement in this important student population’s performance on the NAEP 2015 reading assessment—a year in which disaggregated reporting for ELs with disabilities performance became mandatory. These findings have implications for improved instructional, assessment, and program supports to serve ELs with disabilities.

Advances and Missed Opportunities in the Development of the 2026 NAEP Reading Framework

The National Assessment of Educational Progress (NAEP) in Reading aims to measure reading comprehension for students in the United States and to monitor progress in our education system. NAEP Reading is developed based on an assessment framework document that is periodically revised to reflect the latest understandings about reading comprehension and its assessment. A key goal of the Visioning Panel (VP) and the Development Panel (DP) charged with updating the NAEP Reading Assessment Framework for 2026 was to lay the foundation for an assessment that made progress toward greater fairness, equity, and validity. In this essay, we discuss how the Framework development process unfolded and its results. We document the unusual ways the National Assessment Governing Board (“the Board”) shaped the development process. We provide evidence that a small group of Board members aimed to preserve the status quo in reading assessment by downplaying reliance on expertise and authoritative sources of research on reading, learning, and assessment and by removing attention to equity in NAEP Reading. We also discuss both successful (i.e., approved by the Board) and unsuccessful (i.e., rejected by the Board) recommendations for changes to the 2026 Framework that initially were proposed by the DP. We end by considering how, despite the efforts of the small group of Board members, we as a literacy field can improve the nature and impact of our large-scale reading assessments, and NAEP in particular.

WHAT IS THE IMPACT OF AFFECTIVE DISPOSITION ON 4TH-GRADE NAEP READING SCORES?

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of affective disposition on 2019 NAEP reading scores of fourth-grade students. In order to gain a better understanding of the impact of affective disposition on the reading achievement of fourth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2019 NAEP data set. 
 The results found in this study showed that students’ affective disposition in the areas of making a great effort after making a mistake, continuing to work hard even when they felt like quitting, paying attention and resisting distractions, and feeling happy at school had positive impact on their 2019 NAEP reading assessments scores when the students strongly agreed with the statements. The findings may indicate that improving affective disposition in students may increase reading scores.

What Does “Below Basic” Mean on NAEP Reading?

The fourth-grade 2019 National Assessment of Educational Progress (NAEP) Reading assessment shows that 34% of the nation’s students perform below the NAEP Basic level. However, because there is no achievement-level description for below Basic, educators and policymakers lack information on the nature of the reading difficulties that these students face. To help fill this gap, we analyze data from the 2018 NAEP Oral Reading Fluency study. We find that, compared with students who perform at the NAEP Basic level and above, students who perform below NAEP Basic level are much more likely to have poor oral reading fluency and word reading skills.

EXAMINING DUAL ACADEMIC CREDIT AND THE LOCATION OF INSTRUCTION AS A PREDICTOR OF 12TH GRADE NAEP READING SCORES

This study explored 12th grade reading scores on the 2015 National Assessment of Educational Progress (NAEP) and how they relate to enrollment in dual credit college courses and the location in which they were taught. To gain a better understanding of dual credit attainment through various instructional delivery methods and the prediction of 12th grade reading scores, data was mined from the 2015 NAEP and presented in this descriptive research study. The findings of this study include for 12th grades students: (1) Students who earned ELA dual college credit delivered at their high school campus did not result in higher NAEP reading scores. (2) Students who earned ELA dual college credit delivered at a postsecondary campus did not result in higher NAEP reading scores. (3) Students who earned ELA dual college credit delivered through distance learning had higher NAEP reading scores when 11-25% of the 12th grade students enrolled. These findings make evident the high school campus or postsecondary campus dual credit courses for 12th grade students did not have an impact or can be used as a predictor on the NAEP reading test. The dual credit distance learning did have a positive impact on the 12th grade NAEP reading scores, specifically if a smaller percentage of students at the school enrolled, identifying that increasing access to dual credit may not always translate to increased college readiness and rigor.

Relating Reading Comprehension to Oral Reading Performance in the NAEP Fourth‐Grade Special Study of Oral Reading

AbstractIn this secondary analysis study, the authors explored the relations between reading comprehension and oral reading performance in fourth‐grade students, using a data set from the U.S. National Assessment of Educational Progress (NAEP) special study of oral reading. The data set consisted of 1,713 students randomly selected from the 140,000 fourth graders who participated in the 2002 main NAEP reading assessment. Oral reading was measured by word‐reading accuracy, rate, and prosody. The authors investigated how these variables related to NAEP comprehension across the ability distribution, with a focus on students with low comprehension scores. The results add to the literature in several ways. First, indicators of oral reading fluency continue to explain variation in reading comprehension, even when students had been given multiple opportunities to familiarize themselves with the passage that they read aloud. Second, word‐reading accuracy and reading rate independently explained comprehension scores. Third, the authors observed qualitatively different profiles of fourth‐grade readers with low comprehension scores. Nearly all students with reading rates lower than one standard deviation below the mean also had problems with word recognition, as evidenced by the relatively higher number of word‐reading errors and low prosody ratings. The authors interpret these findings in light of the lexical quality hypothesis and the verbal efficiency theory, which emphasize the importance of word‐level accuracy and automaticity to reading comprehension.

Impact of National Board for the Professional Teaching Standards (NBPTS) certification on student achievement

A growing number of teachers have undertaken National Board for Professional Teaching Standards (NBPTS) certification training since its inception over twenty-five years ago. Previous empirical research on the impact of NBCTs on student performance has focused on state or district-level exams in individual states and found mixed results. This study examines the relationship between National Board Certified Teachers (NBCTs) and student achievement on the reading and math assessments of the National Assessment of Educational Progress (NAEP). We argue that achievement can be affected both directly by the certified teacher and indirectly as NBCTs provide mentoring to colleagues and assume school leadership positions. This study focuses on a nationally representative assessment to measure student achievement rather than state- or district-level assessment exams. We find that the percentage of National Board certified teachers in a state is positively related to scores on state-level NAEP Reading and Math assessments.

An Experimental Study of the Effects of Monetary Incentives on Performance on the 12th-Grade NAEP Reading Assessment

Background/context The National Assessment of Educational Progress (NAEP) is the only comparative assessment of academic competencies regularly administered to nationally representative samples of students enrolled in Grades 4, 8, and 12. Because NAEP is a low-stakes assessment, there are long-standing questions about the level of engagement and effort of the 12th graders who participate in the assessment and, consequently, about the validity of the reported results. Purpose/Focus This study investigated the effects of monetary incentives on the performance of 12th graders on a reading assessment closely modeled on the NAEP reading test in order to evaluate the likelihood that scores obtained at regular administrations underestimate student capabilities. Population The study assessed more than 2,600 students in a convenience sample of 59 schools in seven states. The schools are heterogeneous with respect to demographics and type of location. Intervention There were three conditions: a control and two incentive interventions. For the fixed incentive, students were offered $20 at the start of the session. For the contingent incentive, students were offered $5 in advance and $15 for correct responses to each of two randomly chosen questions, for a maximum payout of $35. All students were administered one of eight booklets comprising two reading blocks (a passage with associated questions) and a background questionnaire. All reading blocks were operational blocks released by NAEP. Research Design This was a randomized controlled field trial. Students agreed to participate without knowing that monetary incentives would be offered. Random allocation to condition was conducted independently in each school. Data Collection/Analysis Regular NAEP contractors administered the assessments and carried out preliminary data processing. Scaling of results and linking to the NAEP reporting scale were conducted using standard NAEP procedures. Findings Monetary incentives have a statistically significant and substantively important impact on both student engagement/effort and performance overall, and for most subgroups defined by gender, race, and background characteristics. For both males and females, the effect of the contingent incentive was more than 5 NAEP score points, corresponding to one quarter of the difference in the average scores between Grades 8 and 12. In general, the effect of the contingent incentive was larger than that of the fixed incentive, particularly for lower scoring subgroups. Conclusions/Recommendations There is now credible evidence that NAEP may both underestimate the reading abilities of students enrolled in 12th grade and yield biased estimates of certain achievement gaps. Responsible officials should take this into account as they plan changes to the NAEP reading framework and expand the scope of the 12th-grade assessment survey.

On the Estimation of Hierarchical Latent Regression Models for Large-Scale Assessments

To find population proficiency distributions, a two-level hierarchical linear model may be applied to large-scale survey assessments such as the National Assessment of Educational Progress (NAEP). The model and parameter estimation are developed and a simulation was carried out to evaluate parameter recovery. Subsequently, both a hierarchical and a simple model were applied to NAEP reading data. The impact of using a hierarchical model was found to be relatively modest in this case, mostly due to modest clustering. Several other applications and future studies are discussed.

EVIDENCE‐CENTERED ASSESSMENT DESIGN FOR REASONING ABOUT ACCOMMODATIONS FOR INDIVIDUALS WITH DISABILITIES IN NAEP READING AND MATHEMATICS

ABSTRACTAccommodations play a key role in enabling individuals with disabilities to participate in the National Assessment of Educational Progress (NAEP) and other large‐scale assessments. However, it can be difficult to know how accommodations affect the validity of results, thus making it difficult to determine which accommodations should be allowed. This study describes recent extension of evidence‐centered assessment design (ECD) for reasoning about the impact of accommodations and other accessibility features (e.g., universal design features) on the validity of assessment results, using examples from NAEP reading and mathematics. The study found that the ECD‐based techniques were useful in analyzing the effects of accommodations and other accessibility features on validity. Such design capabilities may increase assessment designers' capacity to employ accessibility features without undermining validity.

FITTING THE STRUCTURED GENERAL DIAGNOSTIC MODEL TO NAEP DATA

ABSTRACTXu and von Davier (2006) demonstrated the feasibility of using the general diagnostic model (GDM) to analyze National Assessment of Educational Progress (NAEP) proficiency data. Their work showed that the GDM analysis not only led to conclusions for gender and race groups similar to those published in the NAEP Report Card, but also allowed flexibility in estimating multidimensional skills simultaneously. However, Xu and von Davier noticed that estimating the latent skill distributions will be much more challenging with this model when there is a large number of subgroups to estimate. To make the GDM more applicable to NAEP data analysis, which requires a fairly large subgroups analysis, this study developed a log‐linear model to reduce the number of parameters in the latent skill distribution without sacrificing the accuracy of inferences. This paper describes such a model and applies the model in the analysis of NAEP reading assessments for 2003 and 2005. The comparisons between using this model and the unstructured model were made through the use of various results, such as the differences between item parameter estimates and the differences between estimated latent class distributions. The results in general show that using the log‐linear model is efficient.

Conditional Covariance Theory and Detect for Polytomous Items

This paper extends the theory of conditional covariances to polytomous items. It has been proven that under some mild conditions, commonly assumed in the analysis of response data, the conditional covariance of two items, dichotomously or polytomously scored, given an appropriately chosen composite is positive if, and only if, the two items measure similar constructs besides the composite. The theory provides a theoretical foundation for dimensionality assessment procedures based on conditional covariances or correlations, such as DETECT and DIMTEST, so that the performance of these procedures is theoretically justified when applied to response data with polytomous items. Various estimators of conditional covariances are constructed, and special attention is paid to the case of complex sampling data, such as those from the National Assessment of Educational Progress (NAEP). As such, the new version of DETECT can be applied to response data sets not only with polytomous items but also with missing values, either by design or at random. DETECT is then applied to analyze the dimensional structure of the 2002 NAEP reading samples of grades 4 and 8. The DETECT results show that the substantive test structure based on the purposes for reading is consistent with the statistical dimensional structure for either grade.

An Ammunition Dump?

WHEN EDUCATORS open their Fibber McGee's closets of past reforms, a pile of silver bullets will spill out onto the floor. For at least the past 20 years, the sense of public confidence in public education has been fed by an arsenal of such quick, often creative, but largely unsubstantiated remedies for turning schools and kids around. And the pile became much larger this fall. The highest-caliber bullet of all--the No Child Left Behind (NCLB) Act--landed on top of the pile. But it was not the only presumed salvation of the public schools to lose some of its luster recently. Collecting data about these failed silver bullets can often tell us more than we've known before. The first National Assessment of Educational Progress (NAEP) report on reading and math scores that fairly reflects changes under NCLB showed essentially that, so far, this silver bullet is pretty much of a dud. Reading scores were flat, and math scores were up only a bit. Despite statements by the Bush Administration that sought to put a happy face on the findings, the NAEP data showed that progress had been better before NCLB became law than it has been since it was enacted. The data also indicate something that few pundits seem willing to acknowledge. Since the passage of NCLB, about four years ago, changes in the demographics of the student population have accelerated, and schools all over the country are feeling the impact of more English-language learners than ever before. It is to the schools' credit that NAEP reading scores--in the early grades, at least --did not go down. Moreover, the very slight improvement in math scores could be attributed partially to the fact that math test items in the early grades, and even to some extent on the eighth-grade tests, do not require advanced reading skills. (Another possible explanatory factor is that the efforts of the National Council of Teachers of Mathematics have given us some semblance of national math standards and that these standards have been in various stages of implementation for more than 15 years.) The NAEP data also underscore the accident-of-birth unfairness built into state accountability systems. The erratic performance of students on state measures of achievement, compared to their performance on NAEP, demonstrates that students in some states are being held accountable for much higher levels of performance than students in other states. And no state has made much progress, as revealed in the NAEP scores. Some 20 states claim to have made progress on their own eighth-grade reading assessments between 2003 and 2005, but only three showed any progress at even the basic level on NAEP. And, according to the Thomas B. Fordham Foundation, only eight of 25 states that did announce progress on their own fourth-grade reading tests had more than one-third as much progress at the basic level on NAEP. As might be expected, proponents and opponents of charter schools took the very same data and came up with different analyses. The National Alliance for Public Charter Schools found them to be making more progress than other schools; the American Federation of Teachers found them to be making less, even after accounting for student-level racial/economic variables. Another silver bullet to land atop the pile of spent ones is the grandiose array of promises made by Edison Schools, Inc. In the immortal tradition of Edward R. Murrow's series You Are There, I was there when Tennessee entrepreneur Chris Whittle announced that he would launch a network of at least 100 new schools that would rival public schools in terms of technology and student achievement. …

The Near Impossibility of Testing for Teacher Quality

The federal No Child Left Behind Act of 2001 (NCLB) mandated that a highly qualified teacher be in all our nation's classrooms by academic year 2005-2006. To accomplish this laudable goal, each state must define what it means by a highly qualified teacher. States are permitted to use teacher licensure tests to demonstrate to the federal government that their teachers are highly qualified, that is, capable, competent, skilled, trained, practiced, and so forth. The theory of action behind the policy is that if America's teachers were of sufficiently high quality, then education would improve. But all this appears to be political spectacle (Smith, 2004); pure theater with no other purpose than to look like something positive is happening, whereas it is not. The way to recognize spectacle is through analysis of the slogans and policies promulgated by politicians. For example, although the call is to ensure highly qualified teachers, there is no evidence to suggest that teachers, as a group, are not now highly qualified. Perhaps this legislation is designed in part to scare ordinary citizens into thinking that millions of unqualified teachers are in charge of their children, although the 3 million teachers we now have in our schools appear to be overwhelmingly qualified to be teaching. How do we know that America's teachers as a group are competent, skilled, and qualified to teach? One way we know is from the evidence we collect on how well our children are learning, because the ability to promote learning in children must be part of the criteria applied when we designate a teacher as highly qualified. Evidence of learning is obtained from the National Assessment of Educational Progress (NAEP), from which there is evidence of growth for all racial and ethnic groups among 9-, 13-, and 17-year-olds in all three of the subject areas regularly tested: reading, mathematics, and science. In fact, on the NAEP reading tests, scores for 9-year-old African Americans place them approximately 2 years ahead of where their parents scored when the tests were first introduced (Berliner, 2004) The quality of our teachers can also be inferred through the performance of American 9-year-old students in reading. On Progress in International Reading Literacy Study (PIRLS) tests, U.S. students performed remarkably well. Despite the Bush administration's criticism of reading instruction in the United States, PIRLS shows White American students performing better than those of the highest performing nation in the world; all U.S. students, combined, placed about third in the world statistically; and even America's minority public school students scored above the international average (Berliner, 2004). These results raise questions about the claim that we lack qualified teachers of reading (for more on how the premise of a failing school system cannot be supported, see Berliner, 2004). Certainly America's poor and minority students do not do well in most national and international assessments. In particular, it is these students who need access to more qualified teachers in their schools. However, it is hard to make the case that America's teachers in general are not qualified, as implied by the NCLB legislation. Additional examination of the federal requirement for highly qualified teachers reveals that no means are offered to accomplish the desired goal, suggesting that nothing is expected to happen. Moreover, the language used to rally the politically faithful is kept purposely ambiguous, with the term highly qualified providing no concrete referents for anyone to understand what is so ardently being promoted. Furthermore, demands for highly qualified teachers must take into account the relationship between teacher quality and the pay and status of teachers in our society. Attracting high-quality teachers--whatever that might turn out to be--may be more difficult than imagined by legislators, given the economic and social status of teachers in our society. …

A BAYESIAN HIERARCHICAL MODEL FOR LARGE‐SCALE EDUCATIONAL SURVEYS: AN APPLICATION TO THE NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS

ABSTRACTLarge‐scale educational assessments such as the National Assessment of Educational Progress (NAEP) sample examinees to whom an exam will be administered. In most situations the sampling design is not a simple random sample and must be accounted for in the estimating model. After reviewing the current operational estimation procedure for NAEP, this paper describes a Bayesian hierarchical model for the analysis of complex large‐scale assessments. The model clusters students within schools and schools within primary sampling units. The paper discusses an estimation procedure that utilizes a Markov chain Monte Carlo algorithm to approximate the posterior distribution of the model parameters. Results from two Bayesian models, one treating item parameters as known and one treating them as unknown, are compared to results from the current operational method on a simulated data set and on a subset of data from the 1998 NAEP reading assessment. The point estimates from the Bayesian model and the operational method are quite similar in most cases, but there does seem to be systematic differences in measures of uncertainty (e.g., standard errors, confidence intervals).

There's More Than Mythology to California's Reading Decline

Stephen Krashen got the numbers wrong in his June 2002 Kappan article, according to Mr. Innes, who seeks to set the record straight. THERE is compelling evidence, writes Stephen Krashen, that a reported decline in reading performance in California in the late 1980s and early 1990s really didn't occur, or at the very least was not caused by the introduction of whole-language reading in the state in 1987.1 He presents two sets of testing data and some other statistics to support his assertions. But Krashen's analysis of those data has a number of problems. Krashen Gets the Scoring Data Wrong Krashen claims there is an absence of a clear pattern of changes in the California Achievement Program (CAP) reading scores. That, he says, indicates whole language did not cause a noticeable decline in reading. As evidence he presents a table of CAP data for grades 3, 6, 8, and 12 for the years from 1984 to 1990. But Krashen's line of reasoning concerning the CAP is illogical. When the data are thoughtfully examined for impacts on young readers, the results actually refute Krashen's contentions. Even if teachers throughout California adopted whole-language methods instantaneously in 1987 (a highly unlikely situation), very few of Krashen's data points are relevant to his argument. For example, the first-graders of 1987 took the third-grade CAP in 1989, and 1988's first- graders took the CAP in 1990. Among all the CAP data Krashen provides, only these performances can be considered as purely the product of the whole-language program. In addition, second-graders from 1987 took the CAP in 1988. They had a mixed experience of whole language and the previous instructional methods for reading. But all of the other students listed in Krashen's CAP data table should have learned the basics of reading before California's whole-language reform ever started. In particular the large amount of CAP data that Krashen presents for grades 6 through 12 can provide no support for his argument whatsoever. None of these older students had any involvement with California's whole-language reading reform until well after their reading habits were established. Quite simply, most of Krashen's data are superfluous to his argument and only serve to cloud the issue. But third-grade data in Krashen's table do pertain, and they disprove his contentions. Third-grade reading scores on the CAP rose from 268 to 282 between 1984 and 1987. That period predates California's whole-language reform. These CAP data actually support the idea that California was improving early reading instruction before the whole-language reform began. Things changed when whole language came along. CAP third-grade scores stayed flat in 1988. Students who were in second grade when the whole- language reform began took CAP in this year, and these are the students who had a mixed experience with whole language. Then in 1989, the first group of students who had been taught exclusively from first grade on with California's whole-language program took CAP. Their scores declined. Third-grade reading declined again the following year as the second group of students with exclusively whole-language experience took CAP. Thus the CAP scores, thoughtfully analyzed, directly undermine Krashen's argument. Krashen's problems with analyzing test scores do not end with the CAP. His discussion of California's fourth-grade reading scores on the National Assessment of Educational Progress (NAEP) is incomplete. Krashen correctly says that NAEP didn't start generating state-level fourth-grade reading scores until 1992. He correctly discusses the fact that California scored a 202 on the fourth-grade NAEP reading assessment in 1992 and got a score of 197 in 1994. California's fourth-grade NAEP reading scores then recovered to 202 in 1998. But what Krashen does not say (and NAEP reports are not terribly forthcoming about this situation, either) is that the exclusion of students with learning disabilities steadily eroded NAEP's validity during this period. …

The Notorious G-A-P

THE GAP is back. Actually, the differences between the achievement of white and Asian students and that of black, Hispanic, and other minority students never completely disappeared, but so much progress was made during the 1970s and 1980s that there seemed to be a tacit assumption that the gap would eventually close altogether. The standards and assessment efforts of the past decade were at least partly meant to speed up the process of closing the gap. If districts, schools, and parents are chafing under accountability measures (often hastily and thoughtlessly implemented), it is because state policy makers were under the gun to find ways to close the gap even further. Federal laws, especially Title I, said that all students were to be held to the same high standards, a mandate that rippled through state policies for all districts and schools. The requirement that assessment data be disaggregated by race and ethnicity also ensured that the problem of the gap would remain public. Indeed it did, especially in a rash of reports released as the new school year began, including the annual release of SAT scores from the College Board. One who believes, as I do, that the standards-based movement is essential to closing the gap, must conclude that not enough schools are using standards-based reforms to give low-performing students, especially minorities, access to the same curriculum as everyone else. The latest trend report from the National Assessment of Educational Progress (NAEP) tells the story of the gap for the past 25-plus years. And it was a good story for a long time. Between 1970 and 1988, the gap between the NAEP reading scores of black students and those of white students was cut in half; the math score gap, cut by one-third. Obviously, schools were doing something right, and black students were showing they could indeed learn when given greater opportunities to do so. Then in the 1990s, the progress stopped. Actually, it stopped for everyone (at slightly higher levels than in the 1970s), but black and Hispanic NAEP scores climbed aboard a roller coaster. They fell back, began to improve in some subjects, then regressed again. By 1999, the scores of black and Hispanic students were far below the records set in 1988. Trying to put a good face on a disturbing picture, NAEP officials and Secretary of Education Richard Riley pointed to steady improvements overall in math scores and higher rates of enrollment in more advanced math classes. The same arguments were heard with the College Board's release of the 2000 SAT scores: last year's seniors posted the highest math scores since 1969; verbal scores were unchanged. The percentage of minorities taking the SAT has increased 47% since 1987 and would be expected to pull the average scores down, so this was good news. However, the gap remains very wide, with blacks scoring 104 points lower than whites on the math portion of the SAT and 94 points lower on the verbal portion. These score gaps on the SAT actually widened during the 1990s. So what happened? At a time when state statutes, a general consensus about content standards, improved professional development, greater knowledge from research on learning, and pressure for accountability have been working more or less in tandem, why have the academic gaps grown larger? Researchers are scratching their heads on this one. David Grissmer of RAND points to the unprecedented resources made available to low-income schools by the War on Poverty as one factor in narrowing the gap and to a rising black middle class as another. However, he couldn't explain why the progress ended. His most recent research, described in Improving Student Achievement (RAND, 2000), uses the improvement of fourth- and eighth-grade NAEP math scores as the context for making state comparisons. The rise in scores in some states, he says, is a result primar-ily of both higher per-pupil expenditures and how the money is spent. …

Chapter 7: Statistical and Psychometric Issues in the Measurement of Educational Achievement Trends: Examples From the National Assessment of Educational Progress

Like all studies involved in the assessment of trends in educational performance, the National Assessment of Educational Progress (NAEP) is confronted with an array of unresolved methodological and philosophical issues. One of the basic dilemmas faced by NAEP is how to measure performance change while remaining responsive to advances in curriculum and the technology of assessment. NAEP has become much more cautious about making seemingly insubstantial changes in the assessment because of the so-called NAEP reading anomaly—an apparently steep drop between 1984 and 1986 in estimated reading proficiency that was found to have resulted in part from changes in the order and context in which items appeared. Other issues that NAEP must consider in reporting performance trends are the effect of measurement scale indeterminacies and the ways in which interpretation of trend results can depend on the statistics that are selected for comparing proficiency distributions over time.