High-stakes Standardized Tests Research Articles

Momentary measures of emotions during technology-enhanced learning prospectively predict standardized test scores in two large samples

BackgroundStudent emotions play a vital role in learning and have been linked to achievement outcomes, but research on the relationship between emotions (other than anxiety) and high-stakes standardized tests is sparse. AimsWe investigated whether emotions experienced during learning predict later performance on a high-stakes, mandatory, standardized test. SampleWe used two cohorts comprising a total of 102,007 grade 7–9 U.S. students. MethodsWe used experience sampling methods to measure 14 emotions (e.g., boredom, confusion, hope) as students engaged in technologically-enhanced learning as part of their math classroom instruction. ResultsStudents’ self-reported positive and negative emotions, positively and negatively predicted test scores, respectively, after covarying prior achievement and sociodemographics. However, there was limited evidence for indirect paths between prior achievement, emotions, and subsequent achievement. ConclusionsOur findings corroborate central predictions of major theories on emotions and learning that highlight the importance of emotions to learning processes and outcomes.

Who benefits from elite colleges’ decreased reliance on high-stakes standardized tests? Evidence from a quasi-field experiment

Despite the increasingly popular perception that elite colleges’ reliance on high-stakes standardized tests to select students exacerbates social inequality, researchers have not demonstrated that a reduced emphasis on such tests actually alters individuals’ admission outcomes. Using all applications submitted in 2022 to National Taiwan University, the most prestigious university in Taiwan, we show who among the applicants would have been admitted under the country’s prior college admission system, which was entirely based on high-stakes standardized test scores, and contrast them with those actually admitted in the current system, which emphasizes holistic evaluations and considers many other criteria. The results indicate that for the majority of students the admission outcomes would be unchanged. Even among those whose outcomes would differ, class backgrounds are not clearly related to the direction of change in the outcome. Rather than students of upper- or lower-class backgrounds, women are the unequivocal beneficiary from the decreased reliance on high-stakes standardized tests. The analysis suggests that the better high school grades and more comprehensive application materials submitted by women applicants contribute to a considerable part of the female advantage in a more holistic admission system. Evidence from this quasi-field experiment has implications for how changes in the structural elements of the educational system affect elite college access.

A study of Test-Taking strategies of Iranian IELTS repeaters: Any change in the strategy use?

This study used Latent Growth Curve Modeling (LGCM) to examine the overtime patterns of the score and test-taking strategy changes in an international high-stakes standardized proficiency test. To this end, the test records of 178 Iranian IELTS repeaters were analyzed, using close- and open-ended questionnaires to measure test scores as a function of construct-relevant and construct-irrelevant test-taking strategy changes. Additionally, this study explored the accountable factors for the changes in the repeaters’ strategies. Results indicated a small and gradual increase in the test scores following an overall augmented use of test-management (TM) and a decreased employment of test-wiseness (TW) strategies. Along with contributing to IELTS validity evidence based on the repeaters’ scores, this study found multiple sources to account for the changes in repeaters’ test-taking strategies. Consideration of changes in repeaters’ test-taking strategies by IELTS instructors and test users may add to the validity of interpretation of test scores to the intended purposes of the tests.

Disrupting majoritarian narratives in high stakes testing: A critical race theory analysis of assessment

This article looks at high stakes standardized tests through the lens of critical race theory (CRT) tenet three (Capper, 2019), counter-storytelling and majoritarian narratives, and explores approaches to assessment that are culturally relevant and affirming. We discuss existing literature and research focused on the impact of high stakes testing, especially on marginalized communities, while also highlighting practices that honor the lived experiences and stories of these communities. We utilize a phenomenological methodology as we interview educators in a variety of positions within the same public school district in Texas in order to identify themes and trends in their perceptions of, and relationships to, high stakes standardized testing. The following three themes were identified: (a)majoritarian narratives, (b) perception of the assessment, and (c) change over time. We also identified examples and recommendations for culturally responsive and culturally affirming assessment practices and include specific examples and future implications and recommendations.

Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence From a National Test.

Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both correct response and distractors. Since multiple-choice tests' outcomes can be strikingly consequential and option position effects constitute a potential source of measurement error, these effects should be clarified. In this study, two experiments in which the position of correct response and distractors was carefully manipulated were performed within a Chilean national high-stakes standardized test, responded by 195,715 examinees. Results show small but clear and systematic effects of options position on examinees' responses in both experiments. They consistently indicate that a five-option item is slightly easier when the correct response is in A rather than E and when the most attractive distractor is after and far away from the correct response. They clarify and extend previous findings, showing that the appeal of all options is influenced by position. The existence and nature of a potential interference phenomenon between the options' processing are discussed, and implications for test development are considered.

Gender differences in leaving questions blank on high-stakes standardized tests

Around the world, multiple-choice tests are widely used as part of high-stakes examinations. To counteract lucky guessing, many of them have instituted a penalty for wrong answers. In this paper, we use administrative data from Turkish college admissions test to study the heterogeneity in gender differences in tendency to leave questions blank across subjects, difficulty levels, and stakes. Exploiting the tracking system and using the resulting variation in the effective guessing penalty across different test sections, we find that female test-takers skip significantly more questions than male test-takers in the quantitative track while we do not find a significant difference in other tracks. Among quantitative track students, the gender gap is larger in Math and when questions are more difficult while it reverses in Literature. We also find that self-assessment is related to skipping behavior and explains part of the gender gap. Male test-takers are more likely than female test-takers to report that they are good at Math, Science, and Social Science after conditioning on their number of correct answers in the corresponding test sections. This gender gap, consistently with the one in skipping behavior, reverses when it comes to Literature. Differently from previous literature, our findings suggest that the magnitude and the sign of the gender gap in answering questions under uncertainty is context dependent.

Supporting English Language Learners in High-Stakes Literacy Testing

In Ontario, a mandatory high-stakes standardized literacy test called the OSSLT is administered in the tenth grade. With notable failure rates and acknowledged test anxiety, students are in need of better test preparatory methods. In this article, we examine some of the challenges and analyze ways that online multimedia learning tools can be designed in order to support diverse student needs. The online tool developed in this study features learning principles and practices based on research in literacy testing and online education. Our discussion contributes new ideas and future directions for task designers and online test preparatory tools.

"Into the Unknown": Supervising Teacher Candidates During the 2020 COVID-19 Pandemic

In mid-March, our public schools ended classroom instruction due to COVID-19. The timing of the suspension of face-to-face instruction was in the middle of the student teaching clinical experience for our secondary education teacher candidates. Without preparation, teacher candidates were to guide their middle and high school students through online learning. University faculty were experiencing a similar challenge — how to support and direct their teacher candidates in mid-experience.
 This was a change in the logistics of teaching and in the focus of education. The first priority of the school was not high-stakes standardized testing, nor daily pacing guides, but rather the emotional and social health of students.
 This change in the schools’ priorities fit well with the preparation of our teacher candidates at our Midwest regional teaching university. Our focus was to prepare teachers of students, not teachers of mathematics or English. While our secondary teacher candidates did not have all the tools and technology skills needed to switch to online teaching immediately, they did know it was the relationship with the learner that was most important.
 The second change of the schools’ priorities was a time-allocation switch from mostly teaching to mostly planning, communicating, and supporting one another as teachers. Our teacher candidates already knew that effective communication makes everyone a better instructor and benefits student learning. In order to help our teacher candidates make this transition, we developed the Clinical Practice Interview Protocol and met with our candidates regularly online.
 A third aspect was teacher candidates’ prior experience as students of online classes themselves. Many had taken several online or hybrid university courses and knew what worked well and what they needed to avoid as an online teacher.
 In this reflective essay, we discuss how the university supervisor supported and facilitated our teacher candidates in preparing and implementing quality instruction.

Reframing School Culture through Project-Based Assessment Tasks: Cultivating Transformative Agency and Humanizing Practices in NYC Public Schools

Background/Context In the last two decades, high-stakes testing policies have proliferated exponentially, radically altering the broader educational landscape in the United States. Although these policies continue to dominate educational reform agendas, researchers argue that they have not improved educational outcomes for youth and have exacerbated inequities in schooling across racial, economic, geographic, and linguistic lines. Alternative project-based assessments, like ones used by the New York Performance Standards Consortium (Consortium) are one type of practice to have shown promise in aiding in the creation of humanizing and transformative educational spaces. Purpose/Objective/Research Question/Focus of Study This article examines how teachers and students make meaning of their experiences transitioning away from high-stakes standardized tests to project-based assessment tasks (PBATs) and specifically considers the role that PBATs might play in shaping school culture. Drawing from three years of data collection at 10 New York City public high schools new to the Consortium, we discern how students and teachers negotiate this shift, paying attention to the ways in which PBATs fostered transformative and humanizing pedagogies and practices. We raise the following questions: How can schools that use project-based assessment reinvigorate school culture to address enduring inequities that persist in schools? How might PBATs reframe schools to be more humanizing and transformative spaces? Research Design We used multiple methods to understand how project-based assessment shapes school culture and curriculum in these transitioning schools, and drew from qualitative and quantitative traditions. The research involved: (1) a historical inquiry into the role of the Consortium in school reform; (2) a broad investigation of the 10 schools transitioning into the Consortium (including three rounds of annual surveys with teachers and administrators); (3) three in-depth focal case studies of transitioning schools (including observations, interviews with teachers, and surveys with students); and (4) surveys with experienced teachers new to established Consortium schools. Conclusions PBATs are a useful tool to engage students and teachers more actively as participatory actors in the school environment, particularly when overall school structures collectively support its integration. Although there were inevitable challenges in the process of transition, our data suggest that the school actors mediated some of these tensions and ultimately felt that PBATs helped create more dignified spaces for youth. By anchoring the assessment process in the concept of transformative agency, we consider how the transition to PBATs might reinvigorate school culture, redress harmful systemic injustices, and serve as a necessary part of school reform and education policy.

The Curriculum in an era of global reform: Bobbitt’s ideas on efficiency and teacher knowledge

ABSTRACT Over a hundred years later, The Curriculum by John Franklin Bobbitt continues to be relevant for understanding contemporary issues in education. One issue has been the association with elements of reform such as scripted curriculum programmes and high-stakes standardized testing. This article argues that while Bobbitt’s message was one of scientific management and efficiency, this was to be pursued in a particular kind of way, quite distinct from that used in industry, and involving an extended role for the teacher. Bobbitt sought to create the conditions for a more humanizing education experience, in the pursuit of the greater human welfare and a more democratic society—quite different from the outcomes inherent in contemporary global educational reform. The article offers a close reading of The Curriculum, drawing in particular on Knoll’s recent work on the concept of efficiency, together with a typology of the kind of knowledge which teachers need to teach well. Using this twin framework puts Bobbitt’s legacy in a different light, and shows the need to take account of the full range of his ideas. Doing so can offer better insight into challenges in education over a hundred years later.

Perceptions of Directly Targeting SEL Learning Skills in Public Education

Social and emotional learning (SEL) has garnered increasing attention across education over the last decade. The continual decline in the mental health of students in PK-12 education, in light of the widespread impact of the COVID-19 pandemic, has forced school districts nationwide to re-examine their reliance on student performance data as measured by high-stakes standardized tests. This is particularly true for students in underserved populations who were among the greatest impacted. Many schools in districts across the nation have begun to examine SEL and its impact on student performance. Despite numerous studies highlighting the benefits of SEL programming, school administrators and teachers still struggle with implementing the practices in their classrooms and campuses. This research sought to identify current perceptions of SEL held by public school practitioners. Practitioner understanding of SEL was based on a ranking of key SEL skills needed for student success. Perceptions regarding potential barriers and with whom the responsibility for directly targeting SEL skills lies were also examined. A non-experimental survey design was utilized to compare data between two groups of public education practitioners: teachers and administrative staff. Overall, it was determined that educators, for the most part, agree on what SEL skills make a student successful; however, reported barriers to implementation and designation of the individual responsible for directly teaching SEL skills differed between the two groups.

Assessments and Accommodations for English Language Learners: A Literature Review

While research in how English language learners (ELLs) use assessment accommodations is lacking, there are some general conclusions that one can draw. First, teachers must know their students’ abilities. This includes knowledge of their English proficiency, knowledge of their first language skills, especially as it pertains to literacy skills, and knowledge of their content area understanding. If teachers are aware of areas of weakness in students’ assessments, they should work to compensate for them by either changing their instruction or providing assessment accommodations. Second, it is important for teachers to recognize the various types of assessment accommodations that are available in their teaching situation. More vitally, teachers must be able to assign assessment accommodations appropriately to their students and know differences between ELL accommodations and Special Education accommodations. If accommodations are applied without regard to individual student differences, it can cause more harm to the assessment procedures and evaluations than benefits. Finally, there is a difference between high-stakes standardized tests and classroom assessments. While content-area teachers may have more freedom in modifying their classroom assessments, standardized tests are usually more constrained in their testing format and accommodations they are allowed to offer. Identifying gaps in research related to assessments for ELLs will benefit the field as content-area teachers continue to increase their work with students of diverse backgrounds.

Future doctors in training: translanguaging in an evolution course

ABSTRACT As the de facto language of science, English proficiency often serves as a gatekeeper for aspiring medical students in Puerto Rico because they encounter English-language instruction, textbooks, and high-stakes standardized tests throughout their studies. Employing ethnographic research methods, this case study documents the language practices in an Evolution course at a bilingual Puerto Rican university, which was cotaught by two non-Puerto Rican professors, neither of whom use English as their native language. Using interviews, a focus group, and video-recorded data, this study analyzes the language practices and pedagogies of both professors as they teach the same course and attempt to make the content comprehensible for their students. Our findings document the nonpurist linguistic behavior of the professors and how they built on their students’ linguistic repertoires. Findings also point to how they adopted a nonprescriptive pedagogy that challenges pervasive monolingual ideologies in STEM.

A Comparative Study of Higher-Order Thinking and Teaching in the United States and United Kingdom

The United States and the United Kingdom have used standardized high-stakes testing as a measurement of students’ cognitive level to determine success in the 21st century. Standardized tests have given teachers guidance to help them determine what to teach students and how to teach to the test. With such increased emphasis on high-stakes standardized tests, students are being taught based on tested content. This study evaluates the frequency of higher-and lower-order items in the respective country’s standardized test, and analyzes the teaching of higher-order thinking within classroom instruction.

A Critical Review of 'Getting Tough? The Impact of High School Graduation Exams'

The highly-praised and influential study, “Getting Tough?,” was published in 2001. Briefly, while controlling for a host of student, school, state, and educator background variables, the study regressed 1988 to 1992 student-level achievement score gains onto a dummy variable for the presence (or not) of a high school graduation test at the student’s school. The 1992-1988 difference in scores on the embedded cognitive test in a US Educational Department longitudinal survey comprised the gain scores. The study was praised for its methodology, controlling for multiple baseline variables which previous researchers allegedly had not, and by some opposed to high-stakes standardized testing for its finding of no achievement gains. Indeed, some characterized the work as so far superior in method it justified dismissing all previous work on the topic. Moreover, the article was timely, its appearance in print coincident with congressional consideration of the No Child Left Behind Act (2001) and its new federal mandate requiring annual testing in seven grades and three subjects in all U.S. public schools. The article also served as the foundation for a string of ensuing studies nominally showing that graduation exams bore few benefits and outsized costs (e.g., in dropout rates). Graduation exam opponents would employ these critical studies as evidence to political effect. From a high number of more than thirty states around the turn of the millennium, graduation tests are now administered in only seven or eight states. The multivariate analysis in “Getting Tough?” should have had the advantage of authenticity — an analysis of a phenomenon studied in its actual context. But that should mean that the context is understood and specified in the analysis, not ignored as if it couldn’t matter. And, it could have been understood and specified. Most of the relevant information left out of “Getting Tough?” — specific values for other factors that tend to affect test performance or student achievement — was available from the three contemporary surveys, and the rest could have been obtained from a more detailed evidence-gathering effort. The study could have been more insightful had it been done differently, perhaps with less emphasis on “more sophisticated” and “more rigorous” mathematical analysis, and more emphasis on understanding and specifying the context — how testing programs are organized, how tests are administered, the effective differences among the wide variety and forms of tests and how students respond differently to each, the legal context of testing in the late 1980s and early 1990s, and so on.

Understanding writing quality change: A longitudinal study of repeaters of a high-stakes standardized English proficiency test:

This study examined the writing score and writing feature changes of 562 repeat test takers who took the Canadian English Language Proficiency Index Program–General (CELPIP–General) test at least t...

Understanding writing quality change: A longitudinal study of repeaters of a high-stakes standardized English proficiency test

This study examined the writing score and writing feature changes of 562 repeat test takers who took the Canadian English Language Proficiency Index Program–General (CELPIP–General) test at least three times, with a short (30–40 day) interval between the first and second attempts and a longer (90–180 day) interval between the first and third attempts. Analysis was conducted to uncover whether changes occurred at different testing durations (short vs. long) and whether the observed changes varied across repeater’s initial proficiency groups (low, mid, high). The writing scores measured by CELPIP bands showed great stability over the 6-month period, but the trends of development differed by proficiency group. Low proficiency test takers were more likely to have faster observable score gains, compared to the medium proficiency group, whereas high proficiency repeaters may not maintain their score levels at later attempts. Writing quality was analyzed using natural language processing (NLP) tools. Results suggested that for all proficiency groups, lexical features were more likely to improve over the 6-month period, with some measures showing improvement at 1 month; features in cohesion and syntactic sophistication, however, did not change significantly.

Privatization of early childhood education (ECE): Implications for social justice in Nepal and Kenya

This ethnographic study examined the (un)intended1consequences of increased privatization of Early Childhood Education (ECE) in Nepal and Kenya. Qualitative data showed overreliance on high-stakes standardized tests increased competition for ‘good grades or examination scores’, thus (un)intentionally creating ideal conditions for proliferation of for-profit private schools that predominantly taught culturally decontextualized education at all levels of schooling. Private schools in both countries served high-income families and children, while low-income families and children did not have access to ECE or attended government and not-for-profit programmes. Rather than bridging the gap between low and high-income families, these educational spaces influenced existing social divisions and inequalities. Therefore, this study concluded that private schools in Nepal and Kenya function like businesses, which (un)intentionally promoted educational injustice2against children from low-income families. Consequently, authors recommend enactment of new educational policies and practices that promote culturally contextualized curricula in ECE programmes.

‘Solidarity, Sisters! We’re All Crazy’: The Moral Madness of Opting Out of High-Stakes Testing

This article explores how moral madness manifests in the Opt Out Florida Network, a group committed to ending high-stakes standardized testing. Using data culled from a critical online ethnography of the movement, we show how parents and teachers in the Opt Out Florida Network experience moral madness as a result of standardization and high-stakes accountability in U.S. schools. Our findings, positioned within the neoliberal policy environment, reflect a larger state of crisis in the education profession, wherein both teachers and parents find their autonomy and expertise devalued. Repositioning our participants’ madness as moral injury, we recommend democratic pathways to healing.

“Our Schools Turned Into Literal Police States.”: Disciplinary Power and Novice Teachers Enduring a Cheating Scandal

The voices of teachers experiencing and reacting to highly-publicized testing scandals are rarely heard, despite high-levels of criticism and blame from many stakeholders. Drawing on Foucault’s conception of disciplinary power (hierarchical observation, normalizing judgment, and examination) (1975/1995), we analyzed 5,897 written reflections from 38 novice teachers working in 26 different elementary schools in an urban school district as teachers wrote about their experiences of teaching during a high-profile, high-stakes standardized test cheating scandal. Reflections depicted chronic manifestations of student and teacher stress and pressure, rigid testing procedures, mandated fear-based training, miscommunications, disrupted routines, developmentally inappropriate practices, and surveillance. These findings complicate dominant narratives about the cheating scandal and call all stakeholders to disrupt current discourses of accountability in order to recreate schools as liberatory and ethical spaces. In an era where accountability policies claim to work toward the goals of equal education for all and social justice, the effects of said policies must be critically examined by policymakers at all levels.