National Assessment Of Educational Progress Research Articles

Relooking at the Common Core Standards Through the Lens of Equity – Closing the Achievement Gap

On the failure of No Child Left Behind (NCLB) to close the “achievement gap” between whites and minorities, the Common Core State Standards were heralded as the best way of raising academic standards for all children around the country and closing the achievement gap. Numerous reports have emerged questioning the efficacy of the Common Core Standards to deliver what was promised. Public disillusion is apparent. This paper is an attempt to revisit the Common Core Standards through the lens of data generated by its implementation. Quantitative data available from the National Assessment of Educational Progress (NAEP) for 4th, 8th and 12th grade students and their achievements scores for mathematics and reading for the years 2009, 2011, 2013 and 2015 were taken into consideration. Results have revealed no increase, much less significant, in the average achievement scores and no indication that the achievement gap was being narrowed. Recommendations have been made for having a relook at the content and the implementation of the standards.

Cognitive Validity: Can Multiple-Choice Items Tap Historical Thinking Processes?

Cognitive validity examines the relationship between what an assessment aims to measure and what it actually elicits from test takers. The present study examined whether multiple-choice items from the National Assessment of Educational Progress (NAEP) grade 12 U.S. history exam elicited the historical thinking processes they were designed to measure. Think-aloud data from 27 accomplished high school students revealed that in no instances did students engage in the intended processes. Rather, the items typically elicited three construct-irrelevant processes: factual recall/recognition, reading comprehension, and test-taking strategies. Further, findings revealed that although the items often prompted students to engage in factual recall, they were often not sound indicators of student knowledge. Implications for history testing and alternative forms of assessment are discussed.

Book review

Little is known regarding the underlying constructs of writing tests used by U.S. state education authorities and national governments to evaluate the writing performance of their students, especially in middle school grades. Through a content analysis of 78 prompts and 35 rubrics from 27 states’ middle school writing assessments from 2001 to 2007, and three representative prompts and rubrics from the United States’ 2007 National Assessment of Educational Progress (NAEP) writing test, this study illuminates the writing constructs underlying large-scale writing assessments through examination of features in prompts and rubrics and investigation of the connections between prompts and rubrics in terms of genre demands. We found the content of state writing assessments and the NAEP align with respect to measurement parameters associated with (a) emphasis on writing process, audience awareness, and topic knowledge, (b) availability of procedural facilitators (e.g., checklists, rubrics, dictionaries) to assist students in their writing, and (c) inclusion of assessment criteria focused on organization, structure, content, details, sentence fluency, semantics, and general conventions. However, the NAEP’s writing assessment differs from many state tests of writing by including explicit directions for students to review their writing, giving students two timed writing tasks rather than one, making informational text production one of the three genres assessed, and including genre-specific evaluative components in rubrics. This study contributes to our understanding of the direction and path that large-scale writing assessments in the US are taking and how writing assessments are continually evolving.

Similarities and differences in constructs represented by U.S. States’ middle school writing tests and the 2007 National Assessment of Educational Progress writing assessment

Abstract Little is known regarding the underlying constructs of writing tests used by U.S. state education authorities and national governments to evaluate the writing performance of their students, especially in middle school grades. Through a content analysis of 78 prompts and 35 rubrics from 27 states’ middle school writing assessments from 2001 to 2007, and three representative prompts and rubrics from the United States’ 2007 National Assessment of Educational Progress (NAEP) writing test, this study illuminates the writing constructs underlying large-scale writing assessments through examination of features in prompts and rubrics and investigation of the connections between prompts and rubrics in terms of genre demands. We found the content of state writing assessments and the NAEP align with respect to measurement parameters associated with (a) emphasis on writing process, audience awareness, and topic knowledge, (b) availability of procedural facilitators (e.g., checklists, rubrics, dictionaries) to assist students in their writing, and (c) inclusion of assessment criteria focused on organization, structure, content, details, sentence fluency, semantics, and general conventions. However, the NAEP’s writing assessment differs from many state tests of writing by including explicit directions for students to review their writing, giving students two timed writing tasks rather than one, making informational text production one of the three genres assessed, and including genre-specific evaluative components in rubrics. This study contributes to our understanding of the direction and path that large-scale writing assessments in the US are taking and how writing assessments are continually evolving.

Student demographics and teacher characteristics as predictors of elementary-age students' history knowledge: Implications for teacher education and practice

Using student- and teacher-level data from the United States National Assessment of Educational Progress (NAEP), we attempted to disentangle the relationship among student demographics, teacher characteristics, and performance on the fourth grade assessment of U.S. history. Our findings suggest that teacher subject matter background, reported time spent on history/social studies and instructional decision-making were positively associated with learning outcomes. The implications of our study include greater emphasis on subject matter knowledge in history teacher education, increased prioritization of history in elementary classrooms, and the incorporation of interdisciplinary resources and in-class discussion as potential high-leverage practices for early grade history teachers.

Bayesian Estimation of Multivariate Latent Regression Models: Gauss Versus Laplace

A latent multivariate regression model is developed that employs a generalized asymmetric Laplace (GAL) prior distribution for regression coefficients. The model is designed for high-dimensional applications where an approximate sparsity condition is satisfied, such that many regression coefficients are near zero after accounting for all the model predictors. The model is applicable to large-scale assessments such as the National Assessment of Educational Progress (NAEP), which includes hundreds of student, teacher, and school predictors of latent achievement. Monte Carlo evidence suggests that employing the GAL prior provides more precise estimation of coefficients that equal zero in comparison to a multivariate normal (MVN) prior, which translates to more accurate model selection. Furthermore, the GAL yielded less biased estimates of regression coefficients in smaller samples. The developed model is applied to mathematics achievement data from the 2011 NAEP for 175,200 eighth graders. The GAL and MVN NAEP estimates were similar, but the GAL was more parsimonious by selecting 12 fewer (i.e., 83 of the 148) variable groups. There were noticeable differences between estimates computed with a GAL prior and plausible value regressions with the AM software (beta version 0.06.00). Implications of the results are discussed for test developers and applied researchers.

Is the Common Core racing America to the top? Tracking changes in state standards, school practices, and student achievement

This study examines the trends of the U.S. states’ reading and math proficiency standards before and after the adoption of the Common Core State Standards (CCSS) and explores their impact on school practices and student achievement. Drawing on the 2003-15 National Assessment of Educational Progress (NAEP) grades 4 and 8 assessment and survey datasets, the study gives new insights into the CCSS policy challenges and outcomes. The states that adopted CCSS raised the rigor of proficiency standards for their student assessments, reversing ‘race to the bottom’ trend during the earlier period of the No Child Left Behind Act (NCLB). However, the CCSS states failed to improve schools’ alignment of ELA and math programs with state/district assessments and also failed to improve students’ reading and math achievement more than the non-CCSS counterparts. The Common Core has helped America race to the top for performance standards, but not for performance outcomes yet.

Black Girls' Achievement in Middle Grades Mathematics: How Can Socializing Agents Help?

ABSTRACTThe middle grades mathematics classroom is full of transitions that students must overcome to become successful long-term learners of mathematics. This transition can be exorbitantly more tumultuous for Black girls who must overcome gender and racial mathematics achievement stereotypes. Mathematics identities and achievement socialization trends can help Black girls navigate the challenges associated with these transitions. In this article, we argue that socializing agents can play a key role in the development of positive mathematics identities and positive achievement socialization patterns in Black girls. We use recent National Assessment of Educational Progress (NAEP) data to characterize Black girls' middle grades content knowledge through a strength-based and growth-oriented lens. Then using the data, we provide explicit examples of how parents, teachers, and curriculum developers acting as socializing agents can harness Black girls' funds of knowledge to address the observed challenges identified in the mathematics NAEP data.

Civic Education in United States: A Multiple Regression of Civic Education Scores from the National Assessment of Educational Progress

Social studies teachers instruct their students through a variety of instructional pedagogies. Social science education researchers have called for K-12 teachers to transition away from a traditional lecture format and move towards a format that encourages critical thinking. Classroom debates of controversial issues are a common method by which teachers engage students in higher-order thinking. This research study utilizes the 2010 National Assessment of Educational Progress (NAEP) 12th grade civics assessment (n = 9,800) to identify instructional techniques that improve student’s score. Utilizing the NAEP Data Explorer online statistical analysis tool, a linear regression was conducted examining the effects of race, socio-economic status, instructional pedagogies, and access to newspapers and computers on student performance. Results show that discussing current events, at any frequency, increases student performance. Students who participated in classroom debates once or twice a month or less also improved. African-American and Hispanic students scored lower, as did students who qualified for free and reduced lunches. This study suggests that the inclusion of current events should be encouraged civics classes.

Predicting students’ writing performance on the NAEP from student- and state-level variables

This study examines the relationship between students’ demographic background and their experiences with writing at school, the alignment between state and National Assessment of Educational Progress (NAEP) direct writing assessments, and students’ NAEP writing performance. The study utilizes primary data collection via content analysis of writing assessment prompts and rubrics and secondary analysis with NAEP data through hierarchical linear modeling. Results indicate students from states with writing tests more similar to the NAEP do not perform significantly better than students from states with writing tests less similar to the NAEP. Rather, student demographic characteristics, including gender, ethnicity, SES, disability status, and English learner status significantly predict NAEP writing performance, as do factors related to frequency of writing across subject areas, frequency of writing for varied purposes, frequency of writing process use, and computer use in writing. The implications of the findings for writing instruction are discussed.

The Academic Achievement of Limited English Proficient (LEP) Youth in New and Established Immigrant States: Lessons from the National Assessment of Educational Progress (NAEP).

The dramatic growth and dispersal of immigrant families has changed the face of public education at a time when states are experiencing increased school accountability pressures under the No Child Left Behind (NCLB) and its recent successor the Every Student Succeeds Act. Of particular concern is how these demographic shifts affect the academic well-being of Limited English Proficient (LEP) youth, the protected sub-group that most directly targets children from immigrant families. Using individual-level data from the National Association of Educational Progress, we examine how 8th grade test scores of LEP youth differ across new and established immigrant destination states. Results show that achievement for LEP youth is higher in new than in established immigrant states but that this advantage is not consistent across ethnic/racial groups. LEP youth in new immigrant states benefit from more favorable demographic characteristics and more family and school resources, but these differences only explain a small portion of the achievement gap.

Cultural discontinuity between home and school and American Indian and Alaska Native children's achievement

ABSTRACTAn assumption of culture-based education with respect to American Indian and Alaska Native (AI/AN) children is that discontinuity between home and school cultures is responsible for educational underachievement. Using data from the 2009 round of the National Indian Education Study, a subset of the larger National Assessment of Education Progress (NAEP), the author constructed a measure of cultural discontinuity and examined its relationship to AI/AN students' Grade 4 and 8 reading and mathematics achievement. Contrary to the cultural discontinuity hypothesis, there is no statistically significant negative relationship when the culture of the home is discontinued at school. On the Grade 4 NAEP reading assessment, in particular, cultural discontinuity was positively associated with students' achievement, net of both student- and school-level controls. Findings suggest that the assumptions of culture-based education may be overstated, though longitudinal data are still needed to be able to make causal claims.

Testing Civics: State-Level Civic Education Requirements and Political Knowledge

Do state-level exams in civics have a positive impact on young people's civic knowledge? We hypothesize that civics exams have the biggest effect in states where they are a requirement for high school graduation—theincentivehypothesis. We further hypothesize that civics requirements have the biggest effect on young people with less exposure to information about the U.S. political system at home, specifically Latinos and, especially, immigrants—thecompensationhypothesis. We test these hypotheses with the 2006 and 2010 National Assessment of Educational Progress (NAEP) civics test administered to high school students, and with a large national survey of 18–24 year-olds. Across the two datasets, we find modest support for the incentive hypothesis and strong support for the compensation hypothesis.

Assessment of fit of item response theory models used in large-scale educational survey assessments

Latent regression models are used for score-reporting purposes in large-scale educational survey assessments such as the National Assessment of Educational Progress (NAEP) and Trends in International Mathematics and Science Study (TIMSS). One component of these models is based on item response theory. While there exists some research on assessment of fit of item response theory models in the context of large-scale assessments, there is a scope of further research on the topic. We suggest two types of residuals to assess the fit of item response theory models in the context of large-scale assessments. The Type I error rates and power of the residuals are computed from simulated data. The residuals are computed using data from four NAEP assessments. Misfit was found for all data sets for both types of residuals, but the practical significance of the misfit was minimal.

Social Studies Concepts: An Analysis of the NAEP and States’ Standards

ABSTRACTThis study examined the 4th-grade National Assessment of Educational Progress (NAEP) assessments (history, geography, and civics) and 3rd- and 4th-grade social studies standards from nine randomly selected states and organized the content around three global concepts (conflict, movement, discovery) and six specific concepts (war, rebellion, immigration, migration, exploration, invention). The extent to which additional related concepts appeared in both documents was recorded, and alignment between the NAEP and standards was determined. Findings revealed that 3rd- and 4th-graders are learning social studies content that represent these global and specific concepts. However, there is variability among states’ standard documents and inconsistencies between what is assessed on the NAEP and what is presented in the standards. In addition, much instruction for these concepts takes place during 4th grade, the same year the NAEP is administered. Implications of these findings, including the urgent need for conceptually based social studies instruction to meet students’ knowledge and achievement gaps, are discussed.

The effects of prior computer use on computer-based writing: The 2011 NAEP writing assessment

Writing achievement levels are chronically low for K-12 students. As assessments follow the transition to computer-based writing, differences in technology access may exacerbate students’ difficulties. Indeed, the writing process is shaped by the tools we use and computer-based writing is different from writing with pen and paper. We examine the relationship between reported prior use of computers and students’ achievement on the first national computer-based writing assessment in the United States, the 2011 National Assessment of Educational Progress (NAEP) assessment. Using data from over 24,100 eighth grade students, we found that prior use of computers for school-related writing had a direct effect on writing achievement scores on the computer-based NAEP assessment. One standard deviation increase in prior use led to a 0.14 and 0.16 standard deviation increase in mean and scaled writing achievement scores respectively, with demographic controls and jackknife weighting in our SEM analysis. We also looked at earlier NAEP assessments and found that prior computer use did not positively affect the earlier pen and paper-based writing assessments.

Academic growth trajectories of ELLs in NAEP data: The case of fourth- and eighth-grade ELLs and non-ELLs on mathematics and reading tests

ABSTRACTUtilizing the National Assessment of Educational Progress (NAEP) data, this study examined (1) how fourth and eighth-grade ELLs' mathematics and reading scores on national tests compared to their non-ELL peers' scores over the testing period between 2003 and 2011, and (2) if gender and ethnicity contributed to variation in the growth patterns among the student groups across grade levels and content areas. Since the NAEP data, which provides a national sample of 10,000–20,000 students, is collected using a probability sample design, sampling weights are adjusted so inferences can be appropriately made. Sample sizes within NAEP are large enough to generate adequate power for statistical significance. Thus, to display the data in a multivariate mode, Tableau 8.0.0 software was used. Results suggested that the achievement gap between non-ELLs and ELLs is either steady or slightly widening in both mathematics and reading, with multiple paths across the content areas, grade levels, and gender and ethnic groups.

The 2008 National Assessment of Educational Progress (NAEP): A visual arts replication study

ABSTRACTThe 2008 National Assessment of Educational Progress (NAEP) Arts Assessment was administered to selected 8th grade students throughout the nation, and in 2009 the results from that administration were publicly reported. In the spring of 2010, building on the format and structure of the 2008 national assessment, the researcher administered a self-portrait test locally to 166 students across three regions in a small southeastern state. In replicating the content and process as closely as possible given the publicly available resources, the purpose of this study was to compare the reported 2008 NAEP self-portrait scores with the results of the 2010 replication study self-portrait scores. Specifically, given that all schools in this small state are required to have dedicated art teachers for K–8 schools, a requirement not typical in all states, the researcher sought to understand the degree to which the local performance mirrored the national scores. After confirming that this study met the requirements to be considered a replication study, the researcher determined that based on comparative results from the 2008 NEAP test and 2010 replication assessment, there was evidence that supported having a stand-alone art teacher in K–8 schools. In fact, the researcher affirmed that early and consistent exposure to art and art making is consistent with a student's positive performance on NAEP Arts Assessments.

Factors Associated with Mathematics Performance in Brazilian Basic Education

Abstract Several studies have indicated that student performance is strongly influenced by school context. Within this frame of reference, the present study sought to identify contextual variables that affect proficiency in mathematics taking the multilevel structure of schools into consideration. The information of 4,338,885 students who participated in the Brazilian national assessment of educational progress of 2009 was used. Multilevel analysis showed an intra-class correlation of .27 and .19 for the 5th grade of elementary school and 9th grade of middle school, the last year of middle school. In both models, more than 50% of school level variance and more than 11% of the variance on student level was explained. The final model is composed of control variables, 13 student level variables, 9 school level variables, and random and interaction effects. The results contribute to a better comprehension of factors that affect school performance, which permits the orientation of public policies.

A Quartile Analysis of Selected Variables From the 2008 NAEP Visual Arts Report Card

This article reports the results of a quartile analysis of the 2008 National Assessment of Educational Progress (NAEP) Visual Arts Report Card. The analysis was made from original datasets supplied by the National Center for Education Statistics. It focuses on 8th-grade students who were currently taking art or recently had taken art. Fourteen Responding and Creating variables are reported in this article. Of the 14 variables, four Responding variables and four Creating variables were found statistically significant. Three Responding and Creating variables were mutually significant. Statistically significant results were found, including Painting and Drawing, Making Art from Clay, Weekly Homework, and Visiting a Museum or Gallery. The analysis suggests that studio production remains a strong component in the art curriculum, and homework assignments and museum visits also can contribute to students’ art education. Several areas where students never or hardly ever have experiences are also identified. Comparisons are drawn between the findings from the 2008 NAEP Visual Report Card (Keiper, Sandene, Perskey, & Kuang, 2009b) and the expectations of the 2014 NCCAS National Core Art Standards for the future.