National Assessment Of Educational Progress Mathematics Research Articles

Driving Assessment Scores with Effective Calculator Implementation

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of calculator exposures on eighth-grade students' 2019 NAEP mathematics assessment scores. To better understand the impact of calculator use on the mathematics achievement of eighth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2019 NAEP data set. The findings are: (1) the frequency of calculator use does not impact assessment scores during math lessons. (2) Students who practice using calculators on math tests and quizzes have higher math assessment scores. (3) Frequent primary calculator use can impact math assessment scores. (4) Students who use graphing calculators frequently score higher on math assessments. In conclusion, these findings indicate that using calculators in mathematics is impactful; however, teachers must be trained to implement them effectively.

Exploring the Relationship between Interest in Higher Education and 12th Grade Mathematics NAEP Scores

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined if a gap exists between the mathematics scores of 12th-grade public school students who have different levels of interest in higher education. - This study used a quantitative descriptive research design to analyze data from the 2013, 2015, and 2019 NAEP data sets. The findings include (1) the average mathematics scale score of students who complete college entrance exams, ACT/SAT, is higher than those who do not complete these exams. (2) The average mathematics scale score of students who complete the FAFSA is higher than those who do not complete the FAFSA. (3) Students who applied to four-year colleges performed significantly higher on the 12th-grade mathematics NAEP than those who did not. (4) Students who applied to two-year colleges performed significantly lower on the 12th-grade mathematics NAEP than those who did not. (5) Students who perceived a future benefit to mathematics scored higher on the mathematics NAEP. These findings indicate that students who are interested in higher education, particularly four-year education, do have higher 12th NAEP mathematics scores. These findings may provide insight into college preparation and guidance at the high school level.

The 2013-15 Decline in NAEP Mathematics: What it Teaches Us about NAEP and the Common Core

ABSTRACT After 25 years with small to moderate gains in performance in mathematics, scores on the National Assessment of Educational Progress (NAEP) main assessment declined between 2013 and 2015 in Grades 4 and 8. Previous research has suggested the decline may be linked to the implementation of the Common Core state standards. In this article, the decline in the NAEP composite score is shown to be driven primarily by losses in the content strands of Geometry and of Data Analysis, Statistics, and Probability. A gain in fractions achievement is also evident in an item-level examination of the NAEP results, but not in reported NAEP scores. These effects are discussed with respect to the CCSS, the rationale for evaluating national progress, and a potential redesign of the NAEP assessment.

How Do Other Countries Measure Up to the Mathematics Achievement Levels on the National Assessment of Educational Progress?

In this study, we mapped achievement levels from the National Assessment of Educational Progress (NAEP) onto the score scales for selected assessments from the Trends in International Mathematics and Science Study (TIMSS) and the Program for International Student Achievement (PISA). The mapping was conducted on NAEP, TIMSS, and PISA Mathematics assessments in 2003. A focus of the study was on whether the NAEP achievement levels were set too high. The results indicated that students from many other countries had substantially larger percentages of students meeting NAEP mathematics achievement levels. In general, the findings suggest the NAEP standard for Advanced is high, but not too high when considered within an international context. With respect to the NAEP standard of Proficient, none of the top-performing countries approached 100% proficient, which seems to underscore the different conceptualizations of “Proficient” in NAEP and No Child Left Behind.

EVIDENCE‐CENTERED ASSESSMENT DESIGN FOR REASONING ABOUT ACCOMMODATIONS FOR INDIVIDUALS WITH DISABILITIES IN NAEP READING AND MATHEMATICS

ABSTRACTAccommodations play a key role in enabling individuals with disabilities to participate in the National Assessment of Educational Progress (NAEP) and other large‐scale assessments. However, it can be difficult to know how accommodations affect the validity of results, thus making it difficult to determine which accommodations should be allowed. This study describes recent extension of evidence‐centered assessment design (ECD) for reasoning about the impact of accommodations and other accessibility features (e.g., universal design features) on the validity of assessment results, using examples from NAEP reading and mathematics. The study found that the ECD‐based techniques were useful in analyzing the effects of accommodations and other accessibility features on validity. Such design capabilities may increase assessment designers' capacity to employ accessibility features without undermining validity.

A New Look at Public and Private Schools: Student Background and Mathematics Achievement

The claim that schools do a better job of educating their students than public schools -- an argument central to the push for market-style reforms -- is rarely questioned. But do the data back it up? The authors examined fourth- and eighth-grade NAEP mathematics achievement in over 1,000 public and schools to arrive at a surprising answer. A RECENT report of mathematics results from the National Assessment of Educational Progress (NAEP) highlighted this major Public- school students scored lower on average than non-public-school students at both grades 4 and 8.1 Of course, this finding is nothing new. Indeed, it is part of the common wisdom in the United States that school students outscore public school students on standardized tests. Furthermore, studies have suggested that this is true even when researchers account for the fact that the enrollment at public schools differs from the enrollment at schools. This belief is based, in part, on past studies involving the 1980 High School and Beyond dataset that found that schools are more effective than public schools at boosting student achievement, including that of disadvantaged students.2 These studies of test performance, which controlled for some potentially confounding variables such as socioeconomic status (SES), affirmed widespread assumptions about the superiority of schools. These assumptions, in turn, have influenced recent reform efforts promoting various forms of privatization of public schools, including the No Child Left Behind Act, which makes use of a variety of sector sanctions for failing public schools. However, the seminal studies on school effects are becoming dated, with most based on a sample of students who began school over a generation ago. In addition, most of these studies were limited to high school.3 Some used rough measures of SES that may not have been sufficiently sensitive to the differences in populations served by public and schools. Given the current interest in choice and privatization, it is an important time to examine current evidence relating to achievement in public and schools. While undertaking a broader study of mathematics instruction and equity, we became intrigued by an unexpected finding: when controlling for school status and student background variables in our statistical models, we saw that mathematics achievement in public schools actually appeared higher than that in schools.4 We decided to do a special substudy focusing specifically on achievement differences between public and schools. Using a powerful SES variable created for the broader study, we were able to more carefully examine the question of whether the widely assumed private school effect is due more to the population of students served than to institutional effectiveness. The study focused solely on student achievement in mathematics -- a subject generally thought to be less influenced by family background and more influenced by institutional effects than other school subjects such as literacy. The Role of Family Background NAEP has been an important tool for monitoring trends in U.S. student achievement. It is the only nationally representative, ongoing assessment of U.S. academic achievement, measuring student performance at fourth, eighth, and 12th grades in mathematics and other subject areas. Our study used fourth- and eighth-grade data; eliminating 12th- grade data removed the issue of high school dropouts, which could bias our samples. Furthermore, the focus on students in the earlier grades reduces the likelihood that the students in the sample had transferred between public and schools. We analyzed achievement and survey data from the 2000 Main NAEP mathematics assessment,5 which we obtained from a restricted-use CD-ROM. (When the study was conducted in 2004, the 2000 assessment was the most recent for which raw data were available to researchers. …

Tips for Readers of Research: ‘Seeing Through’ the Graphs

IN SEVERAL books, I have encouraged readers to graphs. To see through a graph is to ignore the picture and ascertain the meaning. A picture might be worth a thousand words, but pictures also lie on occasion. More often, they simply mislead. In the November 2002 issue of the NCTM News Bulletin, Johnny Lott, president of the National Council of Teachers of Mathematics, takes Brookings Institution fellow Tom Loveless to task for using misleading graphs concerning improvements on the NAEP mathematics assessments. The graphs appear in the Brookings publication The 2002 Brown Center Report on American Education: How Well Are American Students Learning? Lott begins by invoking the statistician's bible of sarcasm: Darrell Huff's 1954 How to Lie with Statistics, which is still in print. One way of making a graph look more impressive, Huff advised, is to change the proportion between the units used for the ordinate (vertical axis) and abscissa (horizontal axis). Lott accuses Loveless of using this deception, which is shown in Figure 1. The first points designating the NAEP (National Assessment of Educational Progress) testing years are two years apart -- 1990 and 1992 -- while the other two are four years apart -- 1996 and 2000. Yet the space of all three intervals is represented as the same. Loveless should have made the distance between 1990 and 1992 half the size of the distances between 1992 and 1996 and 1996 and 2000. In correcting Loveless, though, it seems to me that Lott has created a graph that is just as arbitrary as the original one -- and perhaps even more misleading. Lott converts Loveless' rectangles to squares, with each point between squares on the abscissa representing two years and each point between squares on the ordinate representing two points of the NAEP scale. Lott's version is shown in Figure 2. Lott claims that this is a more honest graph, and it does fix the problem of unequal intervals on the abscissa. But the choice of how to proportion the units in this case (and for most any graph) is arbitrary. Certainly there is nothing inherently meaningful in plotting every two NAEP points. We could plot every single NAEP point or every fifth NAEP point with equal justification. Similarly, there is no theoretical justification whatsoever for making the space between two points on the NAEP scale equal in size to two years of time, but that's what Lott's squares do. I repeat: the choice of proportions is arbitrary, not in the sense of being capricious but in the sense that judgment is required. If the two graphs represented rising costs of something and if you were a superintendent, you might choose one or the other depending on the message you wished to convey to your school board. Moreover, the gains are the same in both graphs. They show changes in average scores over time. The honesty -- or at least the meaningfulness -- of both graphs would be greatly improved by an indication of the variability of the scores around these averages. Remember from basic statistics: No measure of central tendency without a measure of dispersion. The scores rise from 213 in 1990 to 228 in 2000. Is this a meaningful gain? An important gain? Given what psychometricians have been saying lately about the volatility of test scores, we might also ask whether the gain is due mostly to chance or to factors that have nothing to do with the quality of instruction. We would interpret the gains differently if 228 were a full standard deviation above 213 or only one-fourth of a standard deviation above. We can't tell from the graphs. If we knew what the standard deviation was, we could calculate an effect size, a statistic that leads more to practical interpretations than does a calculation of whether or not the gain is statistically significant. …

Impact of Changing Difficulty on Inferences From the National Assessment of Educational Progress

The U.S. Department of Education measures student achievement through the National Assessment of Educational Progress (NAEP). NAEP estimates of population proficiency quantiles are based on a Bayesian multiple‐imputation procedure. This article shows (a) that the resulting estimates depend directly on the mix of item difficulties on the test, and (b) the difficulty of items on the NAEP mathematics exam has increased over time. Does the increasing difficulty of the exam lead to observable changes in student performance over time? This study compared the simulated performance of 1990 examinees on the easier 1990 exam and the more difficult 1996 exam. No significant differences were found. While our results instill confidence that these changes have not impacted the NAEP trend line, our findings are both data‐specific and limited in scope, and NAEP should carefully evaluate future adjustments to the test in this manner.

Using Trilinear Plots for NAEP State Data

Understanding the distribution of achievement levels of students’ performance in the National Assessment of Educational Progress (NAEP) is aided through the use of the trilinear chart. In this article, this chart is described and its use illustrated with data from the 1992 state NAEP mathematics assessment. It is shown that one can see readily the trends in performance for different demographic groups for all of the 44 participating jurisdictions simultaneously. It is suggested that this graphical form may be useful in other contexts, as well.

Brief Reports: An Examination of Relationships between the 1990 NAEP Mathematics Items for Grade 8 and Selected Themes from the NCTM Standards

For about 20 years, the National Assessment of Educational Progress (NAEP) has reported on the status and progress of U.S. educational achievement in a variety of subject areas, including mathematics (Mullis, 1990). The 1990 NAEP mathematics assessment, which was the fifth in this subject area, was different from the previous four assessments in some important ways. For example, the 1990 NAEP assessment was the first NAEP for which it was possible to report state-level results for those states willing to participate. In fact. the 1990 NAEP consisted of two tests: one given to a national sample at grades 4, 8, and 12 as in prior assessments, and the other given only at grade 8 to a different sample drawn specifically for the stateby-state reporting of results.

Chapter 4: Item Response Theory Scale Linking in NAEP

In educational assessments, it is often necessary to compare the performance of groups of individuals who have been administered different forms of a test. If these groups are to be validly compared, all results need to be expressed on a common scale. When assessment results are to be reported using an item response theory (IRT) proficiency metric, as is done for the National Assessment of Educational Progress (NAEP), establishing a common metric becomes synonymous with expressing IRT item parameter estimates on a common scale. Procedures that accomplish this are referred to here as scale linking procedures. This chapter discusses the need for scale linking in NAEP and illustrates the specific procedures used to carry out the linking in the context of the major analyses conducted for the 1990 NAEP mathematics assessment.

Item Response Theory Scale Linking in NAEP

In educational assessments, it is often necessary to compare the performance of groups of individuals who have been administered different forms of a test. If these groups are to be validly compared, all results need to be expressed on a common scale. When assessment results are to be reported using an item response theory (IRT) proficiency metric, as is done for the National Assessment of Educational Progress (NAEP), establishing a common metric becomes synonymous with expressing IRT item parameter estimates on a common scale. Procedures that accomplish this are referred to here as scale linking procedures. This chapter discusses the need for scale linking in NAEP and illustrates the specific procedures used to carry out the linking in the context of the major analyses conducted for the 1990 NAEP mathematics assessment.

The Fourth NAEP Mathematics Assessment: Performance Trends and Results and Trends for Instructional Indicators

This article is the third one to appear in the Mathematics Teacher reporting the results and analysis compiled by the NCTM Interpretive Team for the Fourth Mathematics Assessment of the National Assessment of Educational Progress (NAEP). The first two articles reported on the performance of seventh- and eleventh-grade students in specific content areas (Brown et al. 1988a. 1988b). This article discusses trends in performance across the last three NAEP mathematics assessments and reports on indicators of instructional activity. A companion article on the performance of third- and seventh-grade students appears in this month's Arithmetic Teacher (Carpenter et al. 1988).

Secondary School Results for the Fourth NAEP Mathematics Assessment: Algebra, Geometry, Mathematical Methods, and Attitudes

This article is the second of two articles reporting on the seventh-grade and eleventh-grade results of the fourth mathematics assessment of the National Assessment of Educational Progress (NAEP) administered in 1986. The first article (Brown et al. 1988) presented the background, methodology, and the results of students' performance on discrete mathematics, data organization and interpretation, number and operations, and measurement. This article reports students' performance on variables and relations, geometry, fundamental methods of mathematics, and attitudes. An analysis of eleventh-grade students' performance by mathematics course background was possible, and these data will be reported here where appropriate.

Secondary School Results for the Fourth NAEP Mathematics Assessment: Discrete Mathematics, Data Organization and Interpretation, Measurement, Number and Operations

This article is the first of two articles reporting on the seventh-grade and eleventh- grade results of the fourth mathematics assessment of the National Assessment of Educational Progress (NAEP) administered in 1986. The elementary school results appear in companion articles in the Arithmetic Teacher (Kouba et al. 1988a, 1988b). Secondary school data from previous national assessments have been reported in the Mathematics Teacher (see, e. g., Carpenter et al. [1980, 1983))

Results of the Third NAEP Mathematics Assessment: Secondary School

The recent publication of the Report of the National Commission on Excellence in Education (1983) has focused national attention on the state of education in the United States and the academic achievement of students. The results of the third mathematics assessment of the National Assessment of Educational Progress (NAEP) provide a basis for examining students’ performance in mathematics and how it has changed over the last decade.

Results of The Second NAEP Mathematics Assessment: Secondary School

The National Assessment of Educational Progress (NAEP) completed its second mathematics assessment during the 197778 school year. The two major goals of the assessment are to make available comprehensive data on specific educational attainments of young Americans and to measure change in their educational attainments.

Results and Implications of the Second NAEP Mathematics Assessments: Elementary School

The National Assessment of Educational Progress (NAEP) has completed its second mathematics assessment. This article focuses on the results from this second assessment of 9-year-olds and 13-year-olds. A companion article in the May 1980 Mathematics Teacher examines the secondary school data.

A Forum for Researchers: Research Implications and Questions from the Year 04 NAEP Mathematics Assessment

The National Assessment of Educational Progress (NAEP) conducted its first mathematics assessment during 1972–73, obtaining census data on 283 mathematics exercises and involving 9-year-olds, 13-year-olds, 17-year-olds, and young adults. Background information on NAEP and the mathematics assessment is abundantly available (see Merwin and Higgins, 1968; Foreman and Mehrens, 1971; Martin and Wilson, 1974; NAEP Newsletters). Selected results of the 1972–73 mathematics assessment have been discussed in a set of articles (Carpenter, Coburn, Reys, and Wilson, 1975a, 1975b).

Results and implications of the NAEP mathematics assessment: elementary school

The National Assessment of Educational Progress (NAEP) has now reported its first mathematics assessment.* This article will examine the NAEP mathematics assessment for the two youngest age groups: 9-year-olds and 13-year-olds. The mathematics assessment was done in 1972-73 and will be repeated each five years, with about half of the exercises repeated from one assessment to the next. Previous articles in the Arithmetic Teacher (Foreman and Mehrens, 1968; Martin and Wilson, 1974) have provided information on the nature of the exercises, the procedures for assessment, the purposes of assessment, and general information on NAEP.