National Assessment Of Educational Progress Data Research Articles

The relationship between mathematics self-efficacy and mathematics achievement: multilevel analysis with NAEP 2019

BackgroundThis empirical study aims to investigate the association between mathematics self-efficacy and mathematics achievement gaps among students in Grades 4, 8, and 12, utilizing data from the 2019 National Assessment of Educational Progress (NAEP). The study also considers student-level (e.g., mathematics self-efficacy, gender, race/ethnicity) and school-level (e.g., school location, proportion of underrepresented students) demographics to provide a comprehensive analysis of the factors contributing to mathematics achievement gaps.MethodsA two-level cross-sectional multilevel modeling approach was employed to analyze the variance in mathematics achievement, partitioning it into within- and between-school components. This approach allowed for an examination of association between mathematics self-efficacy and achievement gaps while considering various student and school characteristics. The rationale behind this methodology lies in its ability to capture the hierarchical nature of educational data and provide a nuanced understanding of the factors associated with mathematics achievement.ResultsThe analysis of the NAEP data revealed substantial variability in mathematics achievement across schools in the United States at all grade levels. Furthermore, mathematics self-efficacy emerged as a robust predictor of students' mathematics achievement, exhibiting significant effect sizes for Grades 4, 8, and 12. Remarkably, when students' mathematics self-efficacy was held constant, the mathematics achievement gaps among different student subgroups by gender, race/ethnicity, ELL, IEP, NSLP status narrowed, highlighting the importance of self-efficacy in addressing these disparities. The study also identified the presence of significant school contextual effects, further emphasizing the role of the educational environment in shaping mathematics achievement.ConclusionsThis study underscores the critical role of mathematics self-efficacy in influencing mathematics achievement gaps among students. By acknowledging the association between self-efficacy beliefs and mathematics achievement, policymakers and educators can develop targeted interventions to enhance students' confidence and motivation in mathematics, ultimately promoting equitable educational outcomes. The findings also emphasize the significance of school-level factors, calling for comprehensive approaches that consider both individual and contextual factors in narrowing achievement gaps. The implications of adopting a self-efficacy perspective to address mathematics achievement gaps extend to educational policy, curriculum development, and instructional practices, ultimately fostering more inclusive and effective mathematics education for all students.

Driving Assessment Scores with Effective Calculator Implementation

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of calculator exposures on eighth-grade students' 2019 NAEP mathematics assessment scores. To better understand the impact of calculator use on the mathematics achievement of eighth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2019 NAEP data set. The findings are: (1) the frequency of calculator use does not impact assessment scores during math lessons. (2) Students who practice using calculators on math tests and quizzes have higher math assessment scores. (3) Frequent primary calculator use can impact math assessment scores. (4) Students who use graphing calculators frequently score higher on math assessments. In conclusion, these findings indicate that using calculators in mathematics is impactful; however, teachers must be trained to implement them effectively.

EXPLORING THE RELATIONSHIP BETWEEN AFFECTIVE DISPOSITION AND NAEP MUSIC SCORES

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) data set. The paper explored the relationship between affective disposition and average scores on the 2016 NAEP music assessment for eighth-grade students. To gain a better understanding of the relationship between students' affective disposition and average music scores, this study used a quantitative descriptive research design to analyze the secondary data extracted from the 2016 NAEP data set. The findings include (1) students who agreed that they had a talent for music had significantly higher average music scores. (2) Students who had been told by others that they are good musicians had significantly higher average music scores. (3) Students who reported that they liked playing music for others had significantly higher average music scores. These findings indicate that budget constraints and cuts to music education in public schools may limit student intelligence and the development of a positive self-concept.

Exploring the Relationship between Interest in Higher Education and 12th Grade Mathematics NAEP Scores

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined if a gap exists between the mathematics scores of 12th-grade public school students who have different levels of interest in higher education. - This study used a quantitative descriptive research design to analyze data from the 2013, 2015, and 2019 NAEP data sets. The findings include (1) the average mathematics scale score of students who complete college entrance exams, ACT/SAT, is higher than those who do not complete these exams. (2) The average mathematics scale score of students who complete the FAFSA is higher than those who do not complete the FAFSA. (3) Students who applied to four-year colleges performed significantly higher on the 12th-grade mathematics NAEP than those who did not. (4) Students who applied to two-year colleges performed significantly lower on the 12th-grade mathematics NAEP than those who did not. (5) Students who perceived a future benefit to mathematics scored higher on the mathematics NAEP. These findings indicate that students who are interested in higher education, particularly four-year education, do have higher 12th NAEP mathematics scores. These findings may provide insight into college preparation and guidance at the high school level.

WHAT IS THE IMPACT OF AFFECTIVE DISPOSITION ON 4TH-GRADE NAEP READING SCORES?

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of affective disposition on 2019 NAEP reading scores of fourth-grade students. In order to gain a better understanding of the impact of affective disposition on the reading achievement of fourth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2019 NAEP data set. 
 The results found in this study showed that students’ affective disposition in the areas of making a great effort after making a mistake, continuing to work hard even when they felt like quitting, paying attention and resisting distractions, and feeling happy at school had positive impact on their 2019 NAEP reading assessments scores when the students strongly agreed with the statements. The findings may indicate that improving affective disposition in students may increase reading scores.

Examining the Impacts of Ignoring Rater Effects in Mixed‐Format Tests

AbstractThe use of mixed‐format tests made up of multiple‐choice (MC) items and constructed response (CR) items is popular in large‐scale testing programs, including the National Assessment of Educational Progress (NAEP) and many district‐ and state‐level assessments in the United States. Rater effects, or raters’ scoring tendencies that result in performances receiving different scores than are warranted given their quality, are concerns for the interpretation of scores on CR items. However, there are few published studies in which researchers have systematically considered the impact of ignoring rater effects when they are present on estimates of student ability using large‐scale mixed‐format assessments. Using results from an analysis of NAEP data, we systematically explored the impacts of rater effects on student achievement estimates. Our results suggest that in conditions that reflect many large‐scale mixed‐format assessments, directly modeling rater effects yields more accurate student achievement estimates than estimation procedures that do not incorporate raters. We consider the implications of our findings for research and practice.

EXAMINING THE RELATIONSHIP BETWEEN STUDENTS’ CREATION OF SPECIFIC DIGITAL PRODUCTS AND THEIR TECHNOLOGY ASSESSMENT SCORES

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of eighth grade students’ specific digital product creation on their Technology & Engineering Literacy ICT scores. In order to gain a better understanding of the impact of computer use technology achievement of eighth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2018 NAEP data set. The findings include (1) using computers to create, edit, or organize digital media both for school work and activities beyond school increased overall ICT scores. (2) The average score of students who used computers to create presentations in school increased while those who did this activity after school time saw a decrease in scores. (3) Students who reported that they create spreadsheets mostly saw a decrease in their average ICT score no matter the frequency or purpose for the activity. These findings may indicate that there is an ideal frequency for digital product creation in school, but that these types of activities may not be indicative of real-world use which is how students are assessed.

Assessment for Monitoring of Education Systems: The U.S. Example

Standardized tests are regularly used as education system monitoring tools to compare the average performance of students living in different states or belonging to different subgroups (e.g., defined by race/ethnicity, sex, or parental income) and to track their progress over time. This article describes some uses and design features of tests in system monitoring contexts. We provide the example of the National Assessment of Educational Progress (NAEP), the only large-scale system monitoring test in the United States. The availability of NAEP data, in turn, has facilitated the construction of the Stanford Education Data Archive (SEDA), a publicly available database that can be used to describe patterns of achievement for nearly all school districts in the United States. Here, we discuss progress in and challenges to the use of standardized tests as system monitoring tools.

The Impact of Conversations on Fourth Grade Reading Performance - What NAEP Data Explorer Tells?

<p style="text-align:justify">This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper explored the differences between fourth grade reading scores by examining (1) how often the fourth graders discuss their school work at home with family and (2) how often the fourth graders discuss what they were reading with friends. The results by NAEP Data Explorer indicate that the average scale score (M=214, SD=36) of students who report “never or hardly ever” talking about studies at home was significantly (p <0.001) lower than all other groups. The results of how often they talked to their friends about what they were reading were mixed: Students reporting talk with friends once or twice a month (M=228, SD=35) had significantly (p<0.001) higher average scale scores than those in the never or hardly ever and almost every day groups and slightly lower than those talking once or twice a week.</p>

Black Girls' Achievement in Middle Grades Mathematics: How Can Socializing Agents Help?

ABSTRACTThe middle grades mathematics classroom is full of transitions that students must overcome to become successful long-term learners of mathematics. This transition can be exorbitantly more tumultuous for Black girls who must overcome gender and racial mathematics achievement stereotypes. Mathematics identities and achievement socialization trends can help Black girls navigate the challenges associated with these transitions. In this article, we argue that socializing agents can play a key role in the development of positive mathematics identities and positive achievement socialization patterns in Black girls. We use recent National Assessment of Educational Progress (NAEP) data to characterize Black girls' middle grades content knowledge through a strength-based and growth-oriented lens. Then using the data, we provide explicit examples of how parents, teachers, and curriculum developers acting as socializing agents can harness Black girls' funds of knowledge to address the observed challenges identified in the mathematics NAEP data.

Civic Education in United States: A Multiple Regression of Civic Education Scores from the National Assessment of Educational Progress

Social studies teachers instruct their students through a variety of instructional pedagogies. Social science education researchers have called for K-12 teachers to transition away from a traditional lecture format and move towards a format that encourages critical thinking. Classroom debates of controversial issues are a common method by which teachers engage students in higher-order thinking. This research study utilizes the 2010 National Assessment of Educational Progress (NAEP) 12th grade civics assessment (n = 9,800) to identify instructional techniques that improve student’s score. Utilizing the NAEP Data Explorer online statistical analysis tool, a linear regression was conducted examining the effects of race, socio-economic status, instructional pedagogies, and access to newspapers and computers on student performance. Results show that discussing current events, at any frequency, increases student performance. Students who participated in classroom debates once or twice a month or less also improved. African-American and Hispanic students scored lower, as did students who qualified for free and reduced lunches. This study suggests that the inclusion of current events should be encouraged civics classes.

Predicting students’ writing performance on the NAEP from student- and state-level variables

This study examines the relationship between students’ demographic background and their experiences with writing at school, the alignment between state and National Assessment of Educational Progress (NAEP) direct writing assessments, and students’ NAEP writing performance. The study utilizes primary data collection via content analysis of writing assessment prompts and rubrics and secondary analysis with NAEP data through hierarchical linear modeling. Results indicate students from states with writing tests more similar to the NAEP do not perform significantly better than students from states with writing tests less similar to the NAEP. Rather, student demographic characteristics, including gender, ethnicity, SES, disability status, and English learner status significantly predict NAEP writing performance, as do factors related to frequency of writing across subject areas, frequency of writing for varied purposes, frequency of writing process use, and computer use in writing. The implications of the findings for writing instruction are discussed.

Academic growth trajectories of ELLs in NAEP data: The case of fourth- and eighth-grade ELLs and non-ELLs on mathematics and reading tests

ABSTRACTUtilizing the National Assessment of Educational Progress (NAEP) data, this study examined (1) how fourth and eighth-grade ELLs' mathematics and reading scores on national tests compared to their non-ELL peers' scores over the testing period between 2003 and 2011, and (2) if gender and ethnicity contributed to variation in the growth patterns among the student groups across grade levels and content areas. Since the NAEP data, which provides a national sample of 10,000–20,000 students, is collected using a probability sample design, sampling weights are adjusted so inferences can be appropriately made. Sample sizes within NAEP are large enough to generate adequate power for statistical significance. Thus, to display the data in a multivariate mode, Tableau 8.0.0 software was used. Results suggested that the achievement gap between non-ELLs and ELLs is either steady or slightly widening in both mathematics and reading, with multiple paths across the content areas, grade levels, and gender and ethnic groups.

<em>The Value of a Triangle: Mathematics Education in Industrial and Classical Schools in the Segregated South</em>

Mainstream education research and policy is dominated by social science orientations that primarily rely on positivist theories, models, and interpretations of the mathematical competence of African Americans or Blacks (terms are used interchangeably). Contemporary scholarship and national discourse surrounding the education of Blacks often center on the notion of pathology, a view largely supported by deficit perspectives (Martin, 2009a; Stinson 2006). As Martin has pointed out, research and policy is a site where degradation of African American children occurs (Martin, 2009a, p. 1) in part because of the on-going discussions of racial achievement gaps and comparisons to White and Asian American students.An official and public conversation about achievement gaps has been going on at least since 1969 when The National Assessment of Educational Progress (NAEP) began reporting racially disaggregated achievement data (see National Center for Education Statistics website at http://nces.ed.gov/nationsreportcard/). Most policymakers and researchers use NAEP data to make consequential claims about U.S. students' outcomes - Black students' outcomes in particular. The claims can be consequential because school districts, teachers, and even families and students themselves view the interpretations of the standardized assessments as normative, correlated with intelligence, and generally disconnected from institutional issues and contexts.There are explanations in the literature for why achievement gaps exist. Some researchers point to factors such as inequitable distribution of quality teachers (Adamson & Darling-Hammond, 2012). Others position Black parents negatively, blaming them for placing low value on early education (Jencks & Phillips, 1998). Still other scholars contend that the U.S. does not have an achievement gap at all, but an educational debt (Ladson-Billings, 2006). Mathematics education research and policy continue to use achievement gap language in decontextualized ways despite a growing body of literature that suggests race-comparative approaches position Black students at the bottom of the racial hierarchy of ability (Martin, 2009a, b, c). These approaches do not address issues of identity and power, which are important for decentering the assimilationist perspective in many education policy documents (Gutierrez, 2008). The perspective can be viewed as assimilationist because there is little recognition of the linguistic and cultural resources that marginalized students bring to the classroom or to the discipline of mathematics (Gutierrez, 2008, p. 361). Moreover, there is limited effort by mainstream education research and policy to understand what it means to learn while also being Black (Martin, 2012). Critical Race Theory (Crenshaw, Gotanda, Peller, & Thomas, 2000; Delgado & Stefancic, 2012) points out that race and racism are endemic in U.S. society; therefore, it is critical to analyze the intersection at the teaching and learning of and the story of race to interrogate ways race influences Black people's educational experiences and production of mathematical knowledge.The rise in national focus on the achievement of Blacks using state and national summative assessments can be marked by important legal decisions beginning with Brown v. Board of Education (1954), but also the Civil Rights Act of 1964 and the Elementary and Secondary Education Act (ESEA) of 1965. What did the education of Blacks look like before these legal decisions? While very little is known about the education of Blacks pre-Brown, there is some documentation of important issues we might find at that intersection of teaching and learning and race during segregation (Russell, 2014). The field also knows that new narratives describing the segregated schooling of Blacks, mainly in the South have challenged the master narrative that reduced all Black schools and students to unilateral inferiority (Anderson, 1988; Siddle-Walker, 2009). …

Application of Markov Chains to Analyze and Predict the Mathematical Achievement Gap between African American and White American Students

A stochastic, mathematical model known as a discrete Markov Chain was used to show how to estimate the probability that the mathematical achievement gap between African Americans and White Americans would close during a particular calendar year. The implications of race in the achievement of mathematics in the United States are profound and well-documented in mathematics education research literature. The authors used historical data drawn from the National Assessment of Educational Progress (NAEP) to examine trends of mathematical achievement between African Americans and White Americans during the assessment years of 1973–2012. The authors provide a discussion of NAEP data in the context of the discrete Markov Chain model and describe how specific properties of the Markov process were used to estimate the probability that the mathematical achievement gap will close within the next 50 years.

Can Writing Attitudes and Learning Behavior Overcome Gender Difference in Writing? Evidence From NAEP

Based on eighth-grade writing assessment data from the 1998 ( N = 20,586) and 2007 ( N = 139,900) National Assessment of Educational Progress (NAEP), this study examines the relationships among students’ writing attitudes, learning-related behaviors, and gender in relation to writing performance. Overall, the effects of attitudes were slightly larger than the effects of learning behaviors on writing performance, and gender differences were more prominent in attitudes than learning behaviors related to writing. Perhaps the most surprising finding from the 2007 NAEP data was that females with the most negative attitudes toward writing outperformed males with the most positive attitudes (i.e., writing scores based on two measures of attitudes: females, 157 and 161; males, 151 and 149). Overall, a similar pattern was observed with learning behaviors and gender differences in writing scores. Furthermore, medium effect sizes of gender difference in writing scores (females scoring substantially higher than males) were present even though the students reported to be at the same level in terms of writing attitudes and learning behaviors. The present study demonstrates that gender disparity in students’ writing performance is persistent and strong; it cannot be explained by gender differences in attitudes or behavior alone or in attitudes and behavior combined.

RTI Progress Monitoring Tools

RTI Progress Monitoring Tools

ASSESSING FIT OF LATENT REGRESSION MODELS

ABSTRACTThe reporting methods used in large‐scale educational assessments such as the National Assessment of Educational Progress (NAEP) rely on a latent regression model. There is a lack of research on the assessment of fit of latent regression models. This paper suggests a simulation‐based model‐fit technique to assess the fit of such models. The technique consists of investigating whether basic statistical summaries are predicted adequately by the latent regression model. Application of the suggested technique to an operational NAEP data set reveals important information regarding the fit of the latent regression model to the data.

EVALUATION OF METHODS TO COMPUTE COMPLEX SAMPLE STANDARD ERRORS IN LATENT REGRESSION MODELS

ABSTRACTSeveral complex sample standard error estimators based on linearization and resampling for the latent regression model of the National Assessment of Educational Progress (NAEP) are studied with respect to design choices such as number of items, number of regressors, and the efficiency of the sample. This paper provides an evaluation of the extent these estimators are appropriate for the models and test lengths often encountered in NAEP and what the effect is on the NAEP imputation model. It is shown that in general the resampling method used in this study provides the most accurate standard errors. However, the differences with the linearization method chosen in this study are relatively small if only small models are used with respect to the independent variables of the latent regression. Illustration is provided through several small simulation studies and NAEP data analysis.

Getting the Message Out: An Evaluation of NAEP Score Reporting Practices With Implications for Disseminating Test Results1

How a testing agency approaches score reporting can have a significant impact on the perception of that assessment and the usefulness of the information among intended users and stakeholders. Too often, important decisions about reporting test data are left to the end of the test development cycle, but by considering the audience(s) and the kinds of information that they find useful earlier in the process, the agency can capitalize on that knowledge and increase the likelihood that test scores are used as intended and are regarded positively. To that end, the National Assessment of Educational Progress (NAEP) is “The Nation's Report Card,” charged with making available “reliable information about the academic performance of American students in various learning areas,2” and the purpose of this evaluation of NAEP was to review NAEP score reporting efforts with respect to this charge. The research questions guiding the study focused on the utility of Web-based reporting resources, the understandability and accessibility of current NAEP data displays, and reporting interests of NAEP stakeholders. Findings provided considerable information about the depth and breadth of users' experiences with NAEP reporting, and led to our recommendations for enhancing NAEP reports, and strategies for evaluating score reporting practices.