Northwest Evaluation Association Research Articles

Comparing Northwest Evaluation Association’s Benchmark Assessment Scores to College Board’s ACCUPLACER Scores: Is There A Correlation?

The research examined the relationship between the Northwest Evaluation Association's (NWEA) benchmark assessment (Measure of Academic Progress [MAP]) scores and the score a student receives on the College Board's ACCUPLACER Reading and Quantitative Reasoning, Algebra, and Statistics test and differences between test scores. The correlation between the two scores could help predict which of their students would score as college-ready on the ACCUPLACER test by their NWEA scores. The school started administering the ACCUPLACER tests to classify students as college ready in 2022-2023. Students previously at Maine Virtual Academy had NWEA scores, and the question addressed is how the NWEA scores correlated to ACCUPLACER scores. This is important when considering what interventions are necessary to prepare students for their postsecondary goals, whether their plans include college or a career. The data analysis of 99 high school students in grades 11 and 12 revealed a significant statistical correlation between the NWEA scores and the ACCUPLACER scores in both Reading and Math. The NWEA scores of a student can be used to predict their college readiness. Keywords: NWEA, ACCUPLACER, college-readiness DOI: 10.7176/JEP/14-6-14 Publication date: February 28 th 2023

Markers of At-Risk Literacy Development in Gifted Students

This study explored the role that select components of executive functioning play in predicting reading outcomes in gifted students both directly and indirectly through a broader set of preliteracy skills. A comparison of mediation models indicated the relationships between working memory and inhibition to reading outcomes were fully, rather than partially, mediated by the preliteracy skills. The relationship between inhibition and reading outcomes was mediated through symbol imagery. Symbol imagery served as a mediator to all reading outcomes in this study (Letter Word, Oral Reading Fluency [ORF], and Northwest Evaluation Association Reading Fluency). A measurement of Working Memory in kindergarten was related to ORF in the second grade through speeded naming as a mediator. Neither early phonological processing nor early processing speed were related to the reading outcomes. Implications regarding the use of screening measures for identifying reading difficulties in primary age gifted students are discussed.

Differential return on investment: Academic growth in mathematics and reading based on initial performance.

Students vary in their initial achievement when they enter school and their rate of academic growth as they move through school. These differences have implications for classroom instruction and educational policy. Although previous research has examined initial achievement and growth differences, a gap remains in understanding how initial level of achievement interacts with subsequent growth as children move through school. Using Vygotsky's zone of proximal development (ZPD) and return on investment as theoretical grounding, this registered report examined how students' initial academic performance relative to their school predicts their subsequent academic achievement. The stage 1 accepted registered report is available at https://osf.io/9zmak/. Specifically, we tracked the achievement of a cohort of students who started at or above their school's mean at the beginning of third grade and tested a range of hypotheses regarding their achievement and growth as well as which students showed the greatest gains from their time in school. Using a large database of student academic achievement in the United States, this registered report included de-identified data from all students from fall 2014 to spring 2017 in grades three through five from the ten US states with the highest participation for the Northwest Evaluation Association's Measures of Academic Progress (MAP®) - a computer adaptive test of academic achievement in mathematics and reading. Because the MAP is taken at least twice per school year, up to six scores were included on mathematics and reading achievement for effective samples of approximately 220,000 students. We built separate reading and mathematics three-level piecewise longitudinal hierarchical linear models (student repeated measures, nested within students, nested within schools) to model student growth from the beginning of third grade to the end of fifth grade (i.e., three academic years and two summers). For both mathematics and reading, average student achievement growth slowed as they progressed from third through fifth grade. From there, the findings diverged. In mathematics, student growth was mostly similar across achievement levels and grades from third through fifth. However, in reading, above-average students demonstrated slower growth than average students during the school year but faster growth during the summer. Also of note, at the beginning of third grade, the highest achieving students outscored average students in their school by more than 2 years in mathematics and 3 years in reading. Our results may be able to be explained via a ZPD model, which posits development only occurs when students are placed in appropriately challenging environments. In mathematics, the observed pattern of relatively consistent growth across achievement levels suggests average students were just as likely to be in their ZPD as higher achieving students. In reading, as initial achievement increased, student reading growth slowed, which suggests the higher the initial achievement, the less likely students were to be in their ZPD. If a goal of education is for students to learn new things, our results suggest existing school offerings in reading are not meeting that goal equitably for students across the performance spectrum. Differential growth patterns should be considered when designing learning experiences for students who enter with a wide range of prior mastery.

Schools as Refractors: Comparing Summertime and School-Year Skill Inequality Trajectories

How does schooling affect inequality in students’ academic skills? Studies comparing children’s trajectories during summers and school years provide a provocative way of addressing this question, but the most persuasive seasonal studies (1) focus primarily on skill gaps between social categories (e.g., social class, race/ethnicity), which constitute only a small fraction of overall skill inequality, and (2) are restricted to early grades, making it difficult to know whether the patterns extend into later grades. In this study, we use seasonal comparisons to examine the possibilities that schooling exacerbates, reduces, or reproduces overall skill inequality in math, reading, language use, and science with recent national data on U.S. public school students spanning numerous grade levels from the Northwest Evaluation Association. Our results suggest that schooling has a compensatory effect on inequality in reading, language, and science skills but not math skills. We conclude by discussing the theoretical implications of our findings, possible reasons why the math findings differ from those of other subjects, and discrepant seasonal patterns across national data sets.

Modeling within‐year growth on the Mathematics Measure of Academic Progress

AbstractPrior research on the Measures of Academic Progress (MAP), a computer‐adaptive test distributed by the Northwest Evaluation Association, has primarily focused on the Reading MAP for screening/benchmarking in elementary grades. The purpose of this study was to explore the functional form of growth and the extent to which student variability in growth was captured by the Mathematics MAP across a single school year using linear mixed‐effects regression. Participants were 1608 middle school students from a school district in the Upper Midwest. Results supported a statistically significant nonlinear growth term for sixth grade and statistically significant, but small, monthly linear growth terms for sixth, seventh, and eighth grades. The Math MAP moderately captured individual variability across grades. Implications for research and practice are discussed.

Impacting 9th Grade Educational Outcomes: Results from a Multisite Randomized Controlled Trial of the BARR Model

In this paper, we report results from a multisite, student-level randomized controlled trial that examined the impact of the Building Assets, Reducing Risks (BARR) model on ninth-grade students. The BARR model is a comprehensive, strength-based approach that uses eight interlocking strategies to build intentional staff-to-staff, staff-to-student, and student-to-student relationships. This student-level RCT included approximately 4,000 ninth-grade students randomly assigned to treatment or control conditions in eleven schools. We examined six constructs of student experience including expectations and rigor, engagement, supportive relationships, social and emotional learning, sense of belonging, and grit; and five measures of academic success including course failure, core credits earned, grade point average, and Northwest Evaluation Association’s (NWEA) Measures of Academic Progress (MAP) English Language Arts and Mathematics test scores. Findings suggest that BARR significantly increased core credits earned and mathematics achievement as measured by the NWEA MAP. Relative to the control group, BARR improved several aspects of student experiences in school, including an increased sense of supportive relationships and teacher expectations and rigor.

Counting Differently: Assessing Mathematics Achievement of Signing Deaf and Hard of Hearing Children Through a Unique Lens.

Limited studies exist that connect using signed language with mathematics performance of deaf and hard of hearing children. In the present study, the authors examined 257 participants and compared their results on the Northwest Evaluation Association: Measures of Academic Progress (NWEA MAP) to their results on an assessment of American Sign Language (ASL) skills. It was found that better ASL skills tended to result in better MAP performance. These results are moderated by factors such as age, gender, parental hearing status, and learning disability identification.

Assessment of teaching to predict gains in student achievement in urban schools.

We examined the degree to which assessment of teachers' instructional and behavior management practices, as measured by the Classroom Strategies Assessment System (CSAS; Reddy & Dudek, 2014), relates to gains in student achievement as measured by the Measures of Academic Progress (Northwest Evaluation Association [NWEA], 2011). Two-level hierarchical linear modeling was applied to achievement scores from 2,771 students in 130 kindergarten through 8th-grade classrooms in 13 urban schools serving students in communities with high concentrations of poverty. Results suggest that teachers' use of evidence-based instructional and behavior management strategies, as measured by the CSAS, were associated with reading and mathematics gains. In general, students in classrooms with higher quality use of evidence-based teaching strategies exhibited greater gains, whereas students in classrooms with lower quality use of effective strategies exhibited lesser gains. Implications of these findings for research and educational practice are presented. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Using The Lexia Reading Program To Increase NWEA MAP Reading Scores In Grades 1 to 3

This quantitative study evaluated the efficiency of the Lexia Reading Program in 132 early elementary school students in an inner-city school in Hartford, Connecticut. Students who met the criteria were assigned to the experimental Lexia group, and other students were randomly assigned to a comparison group. The Northwest Evaluation Association Measures of Academic Progress (NWEA MAP) was used to provide pre and post measures of literacy skills. Results indicated that students who used Lexia outperformed students in the control group (p <.01): their MAP scores increased from 157 to 174, a 17-point increase, while the control group’s scores increased from 166 to 171, a 5-point increase. The Lexia reading program appears to be an effective tool to improve the literacy skills of struggling first- to third-grade readers. Keywords: Lexia reading, Lexia group, NWEA MAP, Elementary students, https://orcid.org/0000-0003-2703-5767

The Relationship Between Test Item Format and Gender Achievement Gaps on Math and ELA Tests in Fourth and Eighth Grades

Prior research suggests that males outperform females, on average, on multiple-choice items compared to their relative performance on constructed-response items. This paper characterizes the extent to which gender achievement gaps on state accountability tests across the United States are associated with those tests’ item formats. Using roughly 8 million fourth- and eighth-grade students’ scores on state assessments, we estimate state- and district-level math and reading male-female achievement gaps. We find that the estimated gaps are strongly associated with the proportions of the test scores based on multiple-choice and constructed-response questions on state accountability tests, even when controlling for gender achievement gaps as measured by the National Assessment of Educational Progress (NAEP) or Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) assessments, which have the same item format across states. We find that test item format explains approximately 25% of the variation in gender achievement gaps among states.

Understanding the role of schools in the Asian-white gap: a seasonal comparison approach

Asian Americans are the most highly educated racial group in the United States and are commonly heralded as the model minority for their high academic success. Nevertheless, previous research suggests that Asian Americans may face certain disadvantages in school settings. For example, Asian Americans’ academic advantage over non-Hispanic white students diminishes between kindergarten entry and the next several years of schooling. This study provides a closer examination of the educational progress of Asian American students compared to white students through a seasonal comparison approach. Using the Northwest Evaluation Association, we analyze reading and math scores for over 130,000 Asian American and white students in grades K-7 in approximately 675 public schools across the US. We find that Asian Americans have higher academic achievement than white students in general, but that these advantages are maintained primarily through faster rates of learning during the summer months. When school is in session, the Asian advantage either remains unchanged or shrinks, consistent with the view that some school processes undermine the educational progress of Asian American students relative to white students.

Preparing Teachers’ to Raise Students’ Mathematics Learning

The purpose of this study was to investigate the effectiveness of the ORIGO Stepping Stones program in a large suburban school district in the Midwestern U.S.. The sample included 11 elementary schools that implemented the program during the 2013–2016 school years. Findings are presented from teacher surveys, classroom observations, and analysis of standardized student mathematics achievement scores on the Northwest Evaluation Association Measurement of Academic Progress (NWEA MAP). Using multilevel models, the program did not demonstrate a detectably larger effect on mathematics achievement than the comparison. Classroom observations and views of teachers that support program implementation were explored. In terms of fidelity of implementation, findings suggested that the program was delivered differently from how it was intended. With respect to researchers and policymakers, the approach used to evaluating mathematics program effectiveness is informative and could be used as part of larger accountability systems.

School size and student achievement: a longitudinal analysis

ABSTRACTNumerous initiatives by private philanthropies and the US government have supported school size reduction policies as an educational reform intended to improve student outcomes. Empirical evidence to support these claims, however, is underdeveloped. In this article, we draw on information from a longitudinal dataset provided by the Northwest Evaluation Association covering more than 1 million students in 4 US states. Employing a student fixed effects strategy, we estimate how a student’s achievement changes as (s)he moves between schools of different sizes. We find evidence that students’ academic achievement in math and reading declines as school size increases. The negative effects of large schools appear to matter most in higher grades, which is also when schools tend to be the largest.

Academic Achievement of Deaf and Hard-of-Hearing Students in an ASL/English Bilingual Program.

There has been a scarcity of studies exploring the influence of students' American Sign Language (ASL) proficiency on their academic achievement in ASL/English bilingual programs. The aim of this study was to determine the effects of ASL proficiency on reading comprehension skills and academic achievement of 85 deaf or hard-of-hearing signing students. Two subgroups, differing in ASL proficiency, were compared on the Northwest Evaluation Association Measures of Academic Progress and the reading comprehension subtest of the Stanford Achievement Test, 10th edition. Findings suggested that students highly proficient in ASL outperformed their less proficient peers in nationally standardized measures of reading comprehension, English language use, and mathematics. Moreover, a regression model consisting of 5 predictors including variables regarding education, hearing devices, and secondary disabilities as well as ASL proficiency and home language showed that ASL proficiency was the single variable significantly predicting results on all outcome measures. This study calls for a paradigm shift in thinking about deaf education by focusing on characteristics shared among successful deaf signing readers, specifically ASL fluency.

Validating “value added” in the primary grades: one district’s attempts to increase fairness and inclusivity in its teacher evaluation system

One urban district in the state of Arizona sought to use an alternative achievement test (i.e., the Northwest Evaluation Association’s (NWEA) Measures of Academic Progress for Primary Grades (MAP)) to include more value-added ineligible teachers in the districts’ growth and merit pay system. The goal was to allow for its K-2 teachers to be more fairly and inclusively eligible for individual, teacher-level value-added scores and the differential merit pay bonuses that were to come along with growth. At the request of district administrators, researchers examined whether the different tests to be used, along with their growth estimates, yielded similar output (i.e., concurrent-related evidence of validity). Researchers found results to be (disappointingly for the district) chaotic, without underlying trend or order. Using the K-2 test for increased fairness and inclusivity was therefore deemed inappropriate. Research findings might be used to inform other districts’ examinations, particularly in terms of this early childhood test.

Measuring the impact of substance abuse on student academic achievement and academic growth

This research presents data linking the impact of substance disorder to academic achievement, using data gathered at a recovery high school. Recovery schools provide recovery supports and a high-quality education to students with substance use disorders. The Global Appraisal of Individual Needs – Short Screener and the Northwest Evaluation Association Measures of Academic Progress were administered, and paired observations (Testing 1 (T1) vs. Testing 2 (T2)) were categorized based on information from the Global Appraisal of Individual Needs – Short Screen, as increased, decreased, or no change in substance disorder. Results confirm the impact of substance disorder on academic growth, with T1–T2 pairings in which substance disorder increased resulting in a decrease in academic growth, and T1–T2 pairings in which substance disorder decreased resulting in an increase in academic growth. The impact of no change in substance disorder from T1 to T2 varied by the time frame of the substance use, either in the pa...

American Sign Language/English Bilingual Model: A Longitudinal Study of Academic Growth

This study examines reading and mathematics academic growth of deaf and hard-of-hearing students instructed through an American Sign Language (ASL)/English bilingual model. The study participants were exposed to the model for a minimum of 4 years. The study participants' academic growth rates were measured using the Northwest Evaluation Association's Measure of Academic Progress assessment and compared with a national-normed group of grade-level peers that consisted primarily of hearing students. The study also compared academic growth for participants by various characteristics such as gender, parents' hearing status, and secondary disability status and examined the academic outcomes for students after a minimum of 4 years of instruction in an ASL/English bilingual model. The findings support the efficacy of the ASL/English bilingual model.

The Proficiency Illusion

ONE OF the most controversial provisions in No Child Left Behind is that it lets each state define Given the states' primary role in education and our tradition of local control, some find this approach appropriate. For others, it means that there will be a babel of definitions and a total lack of comparability across state lines. Now comes The Proficiency Illusion, an interesting variation the theme from John Cronin, Michael Dahlin, Deborah Adkins, and Gage Kingsbury of the Northwest Evaluation Association (NWEA). This report, sponsored by the Thomas B. Fordham Foundation, with a foreword by Chester Finn and Michael Petrilli, doesn't cure the worst problem of defining because it uses cut scores that result only in reports of percentage proficient. But it does provide an interesting new perspective how the definitions of vary around the nation. Some have wanted to use NAEP as the unifying metric, the gold standard against which to measure state standards. But NAEP isn't satisfactory for this purpose, because it is a national test not specifically oriented to any state's curriculum or standards. As long as states have different curricula, there will be varying degrees of mismatch between NAEP and state tests. Over the years, NWEA has developed Measures of Academic Progress (MAP) tests. These tests are used by a number of states, and the item pool consists of 5,200 items. But using item response theory, NWEA places all MAP items a common scale. Thus we can ask how well Colorado does its own state test and how well it does MAP tests, and we can ask the same about Wisconsin. Then we can compare the performance of the two states, even though they have different tests and the MAP tests for them use some different items. Those different MAP items are still a common scale. Sufficient numbers of students take MAP tests to permit such comparisons for 26 states. The comparison of most interest is in the location of the cut score. For each state, NWEA looked at the proportion of students attaining the state test. It then asked, What score the MAP would produce the same proportion of students? If, say, 75% of the students in a state were the state test, the question would be What score the MAP test would declare 75% of the students proficient? The MAP percentile score that yielded the appropriate of would be reported for NCLB purposes. The results show that state test cut scores vary enormously. Figure 1 indicates that in Texas a student scoring at the 12th percentile the MAP tests would be labeled proficient. (I ignore Colorado's lower score because that state has a tougher standard for internal use than for NCLB purposes.) By contrast, for a student to be called in California, he or she would have to score at the 61st percentile the MAP. This is where it gets tricky and Finn and Petrilli present some misleading exposition. Here they are discussing the of a fourth-grader in Michigan, which has one of the lowest cut scores in reading, versus the in South Carolina, Massachusetts, and California, which have the highest: This fourth-grader [Susie Smith] lives in suburban Detroit, and her parents get word that she has passed Michigan's state test. She's proficient in reading and math. Mr. and Mrs. Smith understandably take this as good news; their daughter must be on grade level and track to do well in later grades of school, maybe even go to college ... [What they don't know is] that proficiency Michigan's state tests has little meaning outside the Wolverine State's borders; if Susie lived in California or Massachusetts or South Carolina, she would have missed the proficiency cut-off by a mile. Maybe, maybe not. Here are the study's authors writing the same issue: A student who can navigate the California or Massachusetts reading requirements has clearly achieved a much different of competence than has one who just meets the Colorado or Wisconsin standard. …

METHODOLOGICAL CONSIDERATIONS AND RECOMMENDATIONS FOR FUTURE RESEARCH

METHODOLOGICAL CONSIDERATIONS AND RECOMMENDATIONS FOR FUTURE RESEARCH