High-stakes Testing Research Articles

Focusing the Juggernaut

Education technology (EdTech) is at the forefront of many universities' desires to improve, evolve and better the student experience. From learning management systems to hybrid capabilities, many processes are updated to ensure relevance in the face of challenges and modern trends (Josué et al., 2023). These updates can be small and incremental but can also come in the form of large-scale, institution-wide EdTech rollouts. Due to the scale and the diversity of our institutions, this is fraught with issues including aligning technology with pedagogy, ensuring digital literacy and training for educators (Bower, 2017), and infrastructure adequacy (Al-Busaidi and Al-Shihi, 2012). Universities invest heavily in resources to address challenges and promote sustained EdTech adoption (Deacon et al., 2022). Strategic and flexible educators, sometimes with differing perspectives from their institutions, can leverage these resources to achieve their individual educational objectives. This case study, situated at the University of New South Wales (UNSW), involved one such large scale EdTech adoption in the form of an institutionally wide digital assessment platform, InsperaTM, with a view to streamline and enhance the online assessment experience. This had a significant impact, reaching approximately 67 courses and 8.8k students in its first 3 terms of piloting, and to aid this implementation UNSW deployed multiple resources in the form of dedicated staff, in-depth training and central support for high stakes exams. With many educators already using other forms of digital assessment, the adoption was not uniformly accepted, creating friction and causing some to reject the platform. It also created opportunities for others. Wanting to uplift a newly inherited course, Dr Megan Kalucy saw InsperaTM, and the resources associated, as an ideal opportunity to improve student experience and reduce educator workload. With the support of the implementation team, Dr Kalucy adopted InsperaTM for specific in-course assessments, reaching approximately 300 students. Paper-based, short-answer, invigilated assessments, which were administratively inefficient and suffered from delays were replaced by weekly multiple-choice-question (MCQ) quizzes in InsperaTM. Automated marking, immediate feedback, and broader content coverage resulted in an overall improved experience for both staff and students, which was reflected in improved student feedback and reduced workload. This work investigates sector wide issues focusing on the research question of how educators can leverage collective EdTech resources to achieve individual goals. It addresses two key challenges; achieving widespread institutional adoption of EdTech and helping educators with increasing workloads find the time and resources to update their courses. A mixed-methods approach was used, employing a case study methodology comparing student satisfaction data, before and after, to validate the improvements made to the course through the adoption of Inspera. Qualitative data from staff and students was used to inform, implement and revise these improvements. The study found that technology adoption in the educational landscape is not exclusively dependent on knowledge of advancements, but more significantly about leveraging the resources available (Oyetade et al., 2020). Here, the universities adoption of Inspera employed strategies such as training, hands-on support, as well as platform and student support, lowering the energy barrier for improvement to Education Focused staff. These resources were deployed practically and with a focus on educators, increasing the number of early adopters and the uptake of Inspera across the university. This case study also shows that the diverse needs of various disciplines need to be considered for uniform EdTech adoption, as a ‘one size fits all’ approach is often ineffective.

A generative AI-driven interactive listening assessment task

IntroductionAssessments of interactional competence have traditionally been limited in large-scale language assessments. The listening portion suffers from construct underrepresentation, whereas the speaking portion suffers from limited task formats such as in-person interviews or role plays. Human-delivered tasks are challenging to administer at large scales, while automated assessments are typically very narrow in their assessment of the construct because they have carried over the limitations of traditional paper-based tasks to digital formats. However, computer-based assessments do allow for more interactive, automatically administered tasks, but come with increased complexity in task creation. Large language models present new opportunities for enhanced automated item generation (AIG) processes that can create complex content types and tasks at scale that support richer assessments.MethodsThis paper describes the use of such methods to generate content at scale for an interactive listening measure of interactional competence for the Duolingo English Test (DET), a large-scale, high-stakes test of English proficiency. The Interactive Listening task assesses test takers’ ability to participate in a full conversation, resulting in a more authentic assessment of interactive listening ability than prior automated assessments by positing comprehension and interaction as purposes of listening.Results and discussionThe results of a pilot of 713 tasks with hundreds of responses per task, along with the results of human review, demonstrate the feasibility of a human-in-the-loop, generative AI-driven approach for automatic creation of complex educational assessments at scale.

Test Anxiety Among Czech Pupils in Lower-Secondary Education

Test anxiety remains a compelling focus of contemporary research. This attention may be, in part, due to the continued reliance on high-stakes testing, which plays a pivotal role in students' academic trajectories. Our research had several objectives. Given the lack of research on test anxiety in the Czech Republic, our primary aim was to adapt and validate the Children's Test Anxiety Scale (CTAS) on a sample of 740 lower-secondary students (ages 13-15). The results demonstrated adequate construct validity, supporting the three-factor model of the CTAS. Moreover, we investigated the relationship between test anxiety, academic self-concept, and educational outcomes alongside other demographic variables. Our findings revealed a strong negative correlation linking students' academic self-concept and test anxiety. Although no direct correlation emerged between test anxiety and educational outcomes, a linear regression indicated a significant shift: educational outcomes (grades) became significant predictors when considered in conjunction with academic selfconcept. Additionally, our study confirmed a significant impact of gender on test anxiety levels (girls exhibited higher test anxiety than boys) and a modest but significant correlation with parental educational background.

Retelling of Stories with Common Phrasal Expressions by High-Proficiency Learners: Implications for Learning and High-Stakes Testing

The goal of this research was to investigate how well L2 English speakers of different proficiency levels were able to perform on a test of auditory memory like that used in the Wechsler Memory Scale (WMS). In this test, participants must retell heard stories. While the validity of elicited imitation tests is well established in second-language acquisition research, the effectiveness of using retelling as a neuropsychological measurement when a language other than the test taker’s L1 is used is unclear. Further, due to their importance in memory function, this study also analyzed the role of common phrasal expressions in how well participants retold stories in three groups of different proficiency. The results indicated that the increase in scores in the retelling of stories aligned with the proficiency level of the non-native participants. Higher-proficiency NNSs were also able to use more of the commonly used spoken phrases in their retelling. Further, there was no difference in this measure between the higher-proficiency Second-Language English and First-Language English participants. While the effectiveness of this test method as a neuropsychological tool when a language other than the test taker’s L1 is used is unclear, these results indicate significant progress by the higher-proficiency participants. Nevertheless, given that this test is often used in this context with immigrants and minority language patients, doubts remain about its suitability for less proficient individuals.

‘Until now, nobody’s ever actually said straight out how do you feel about them’: children’s experiences of the impact of high-stakes testing through participatory methods

ABSTRACT Research into the impact of high-stakes testing is often adult-centric. Using participatory methods with 10–11-year olds, this paper demonstrates how research with children provides a more nuanced understanding of the impacts of high-stakes testing on wellbeing, curriculum and pedagogy. This paper also demonstrates how children have an implicit understanding of the mechanisms of neoliberalism and how these mechanisms lead to competition with peers, which can exclude lower attaining children. We recommend further participatory research with children to inform wider policy and that schools develop children’s emotional literacy to help them to manage peer relations around high-stakes testing.

The equivalence of a high-stakes objective structured clinical exam adapted to suit a virtual delivery format.

The COVID-19 pandemic necessitated rapid adaptation of clinical competence assessments, including the transition of Objective Structured Clinical Examinations (OSCE) from in-person to virtual formats. This study investigates the construct equivalence of a high-stakes OSCE, originally designed for in-person delivery, when adapted for a virtual format. A retrospective analysis was conducted using OSCE scores from the Internationally Educated Nurse Competency Assessment Program (IENCAP®). Data were collected from 15 exam administrations between January 2018 and June 2022, encompassing 2021 examinees (1936 in-person, 85 virtual). The Many-Facet Rasch Measurement (MFRM) model was employed to analyze the invariance of examinee ability, case difficulty, and criteria difficulty across in-person and virtual formats. Results revealed overall examinee ability estimates remained invariant regardless of the OSCE format, while invariant violations were identified in only three of the 15 cases (N = 20%) adapted to suit the virtual format. The most significant adaptation, namely the use of a verbal physical examination to suit the virtual context achieved equivalence to its hands-on in-person counterpart given evidence of invariance across criteria estimates. Interestingly, criteria scores in invariant violated cases displayed a higher level of stability or consistency across the virtual OSCE formats versus their in-person counterpart highlighting a potential benefit of the virtual versus in-person format and potentially linked to the verbal physical examination. The study found that while examinee ability and case difficulty estimates exhibited some invariance between in-person and virtual OSCE formats, criteria involving physical assessments faced challenges in maintaining construct equivalence. These findings highlight the need for careful consideration in adapting high-stakes clinical assessments to virtual formats to ensure fairness and reliability.

Shove Less, Nudge More: Stakeholders’ Perspective from Writing Classrooms

Academic writing courses are critical in higher education. However, they often rely on directive measures, or "shoves," that impose rigid guidelines, high-stakes assessments, and punitive consequences. These approaches, such as inflexible deadlines and harsh grading penalties, can increase student anxiety, disengagement, and surface learning. As a result, some students resort to unethical strategies, such as using essay mills or AI-generated content. This qualitative study, conducted through interviews with 20 writing professors and 30 students, identified several common shoves in academic writing courses and explored their negative impacts on student engagement and academic integrity. The findings highlight critical areas of concern, including strict rubrics, high-stakes deadlines, standardized feedback, and plagiarism threats. In response, the study proposes a shift from punitive shoves to supportive nudges, categorizing the latter into intuitive and didactic interventions. These nudges, such as automated deadline reminders, scaffolded assignments, and ethical AI usage prompts, aim to foster more positive student behavior and engagement. The next phase of this research will investigate how these behavioral nudges influence learning outcomes and student well-being.

Bioscience Students in Physics Courses with Higher Test Anxiety Have Lower Grades on High-Stakes Assessments and Women Report More Test Anxiety than Men

Test anxiety is beginning to be recognized as a significant factor affecting student performance in science, technology, engineering, and mathematics (STEM) courses, potentially contributing to gender inequity within these fields. Additionally, the management of test anxiety can improve self-efficacy, which is a construct that has been well studied in the physics context. In this study, we investigated the relationship between self-efficacy, test anxiety, and gender differences in performance in a two-semester-long introductory physics course sequence for bioscience students in which women outnumber men. Using validated survey data and grade information from students in a two-semester introductory physics course sequence, we compared the predictive power of self-efficacy and test anxiety on female and male students’ performance on both low- and high-stakes assessments. We found that there were gender differences disadvantaging women in self-efficacy and test anxiety in both Physics 1 and Physics 2, as well as gender differences in high-stakes outcomes in Physics 1. There were no gender differences in low-stakes assessment scores. We also found that self-efficacy and test anxiety predicted high-stakes (but not low-stakes) assessment outcomes in both Physics 1 and Physics 2. Comparison of these findings with prior studies involving physical science and engineering students shows that although women outnumber men in physics courses for bioscience students and the career goals of bioscience students are very different from the earlier researched group, most of the negative trends hold even for this new population. Thus, these findings, in a new context involving bioscience students in physics courses, are very important because they reinforce the systemic nature of women being affected more adversely by anxiety in high-stakes assessments, which is a threat to creating equitable and inclusive learning environments. An important implication is that course instructors should carefully consider how high-stakes and low-stakes assessments are used to determine grades and how to create an overall equitable, inclusive, and low-anxiety learning environment.

Assessing the Speed–Accuracy Tradeoff in Psychological Testing Using Experimental Manipulations

The speed–accuracy tradeoff (SAT), where increased response speed often leads to decreased accuracy, is well established in experimental psychology. However, its implications for psychological assessments, especially in high-stakes settings, remain less understood. This study presents an experimental approach to investigate the SAT within a high-stakes spatial ability assessment. By manipulating instructions in a within-subjects design to induce speed variations in a large sample ( N = 1,305) of applicants for an air traffic controller training program, we demonstrate the feasibility of manipulating working speed. Our findings confirm the presence of the SAT for most participants, suggesting that traditional ability scores may not fully reflect performance in high-stakes assessments. Importantly, we observed individual differences in the SAT, challenging the assumption of uniform SAT functions across test takers. These results highlight the complexity of interpreting high-stakes assessment outcomes and the influence of test conditions on performance dynamics. This study offers a valuable addition to the methodological toolkit for assessing the intraindividual relationship between speed and accuracy in psychological testing (including SAT research), providing a controlled approach while acknowledging the need to address potential confounders. Future research may apply this method across various cognitive domains, populations, and testing contexts to deepen our understanding of the SAT’s broader implications for psychological measurement.

Cutting the Gordian knot of the education ‘crisis’ with teacher testing. A blunt instrument that crushes marginalised pre-service teachers

ABSTRACT This paper offers a brief yet evocative glimpse into marginalised pre-service teachers’ (PST) experiences of teacher testing in Australia’s High-stakes Literacy and Numeracy Test for Initial Teacher Education Students (LANTITE). Utilising Critical Disability Theory (CDT)in particular, Goodley’s (2016) concept of neoliberal-ableism, we problematise teacher testing as a gatekeeping tool for students undertaking teacher education. The article highlights how neoliberal education policies in Australia have disempowered and turned away talented and empathetic future teachers. By illuminating their embodied experiences of stress and anxiety, we interrogate neoliberal discourses of power and how teacher testing is used as a blunt instrument to solve complex problems and funnel public funding to private corporations. As part of a larger longitudinal research project, we offer four narrative portraits giving voice to vulnerable PSTs who have become unwitting victims of the high-stakes test juggernaut. This article focuses on five emergent themes from the research: (a) the embodied impact of stress and anxiety on test-takers, (b) withholding of information regarding testing processes and support, (c) the lack of differentiation available to PSTs (d) impacts of edu-businesses and the business of education on vulnerable participants and (e) a passion for differentiation.

High-stakes Technical Performance Assessments Across the Continuum of Surgical Training.

The aim of this article to outline proposals for the use of high stakes technical performance assessments, defined as assessments of technical skill that take place outside of the operating theatre, across the continuum of surgical training. Technological advancements have allowed for the assessment of operative technical skill outside of the workplace environment, developed and evaluated in line with modern concepts of assessment validity. With the introduction of competency-based education principles across surgical training curricula, increasing scrutiny is being placed on the validity and reliability of assessments used to inform high-stakes training decisions regarding selection, progression through training, autonomy granting and end-of-training certification. The role of performance assessments, conducted outside of the workplace environment, is yet to be fully established. This article discusses the use of high-stakes performance assessments across the continuum of surgical training, with reference to relevant theories of surgical skill acquisition and assessment, along with contemporary evidence from the performance science literature. Assessments of increasing professional authenticity are required across the continuum of surgical training. In an attempt to capture inherent ability rather than skill at the time of trainee selection, assessments should be of deliberately low levels of professional authenticity. Assessment in early surgical training should focus on ensuring competence in core surgical skills, and then assessing for canonical competence across a series of commonly encountered procedures. Baseline competence in these procedures should ideally be assessed prior to the transition to (supervised) lead operator, with a view to ensuring patient safety and improving the efficiency of in-theatre training. Assessment later in training relies more heavily on the workplace, with simulation offering a complimentary role in ensuring a minimum competency standard in procedures less likely to be encountered in the workplace. High-stakes performance assessments can play an important role as part of a programmatic approach to surgical training and certification. They can and should be designed with an appropriate level of professional authenticity to ensure adequate and appropriate representation of the assessment construct.

Teacher Factors as a Mediator between Teaching and Testing at the Higher Education Level

The washback effects of high-stakes tests have become conspicuous and prevalent in Thai educational contexts. Yet, researchers in language assessment still have much to strive towards to understand the nuanced role of teacher factors as a mediator between the washback effects of high-stakes tests and teaching. This study aimed to investigate how teacher factors mediated between the Srinakharinwirot University Standardized English Test (SWU-SET), an exit examination for undergraduate students at a Thai public university in Thailand, and teaching. A mixed-methods design was implemented in this study. The researchers asked 25 full-time university teachers to complete a questionnaire, and five of them were purposively selected to be the informants for an interview and reflective journals. Descriptive statistics and content analysis were employed for data analysis. The main findings revealed that teacher factors (i.e., knowledge, beliefs, and experience) mediated between the SWU-SET and teaching. A smorgasbord of teacher knowledge of such tests and assessment literacy, beliefs in test writer integrity and professionalism, and experience in test development and taking tests could help induce the intended washback. Finally, this study also proposed a model of mediating factors and the washback effects of the SWU-SET on teaching to help stakeholders make informed decisions on teaching.

Effects of loneliness and depression on pre-service teachers’ self-esteem: a mixed method study

ABSTRACT The purpose of the current study was to explore the relationships between depression, self-esteem, and loneliness among Turkish pre-service teachers. The study emphasized that it is valuable to address psychological well-being and social support of students preparing for the Public Personnel Selection Examination (PPSE). The study, which was a sequential explanatory design, included 322 college of education students. The quantitative results indicated that there was a strong correlation between depression, self-esteem, and loneliness scores. Depression significantly predicted self-esteem. When depression scores increased, self-esteem scores decreased. Furthermore, the findings indicated that individuals who had higher self-esteem scores, had lower loneliness scores. Qualitative results showed that time management, test anxiety, and performance-related issues were among the difficulties encountered during PPSE preparation. The results of the current study demonstrated the significance of emphasizing social support, useful coping mechanisms, and attending to students’ psychological and emotional health during high-stakes exams such as the PPSE.

How might Apple, Freire, and hooks redesign the modern school as a site for social transformation?

In Western society, schools have largely been designed to reproduce capitalist, patriarchal, white supremacist, colonial systems and values. Numerous structural elements—from redlining and modern segregation to scripted curricula and high-stakes testing—work to ensure that schools solidify existing social inequalities, produce good worker bees, and keep hegemonic powers in place. Shaped within these forces of reproduction, our schools are fraught with grave problems: racism and discrimination in every form, physical and emotional bullying, hunger and food insecurity, technology addiction, sexual harassment, teen suicide, conflicts and gang violence, drug use, mindless consumption, and ecological destruction. The modern school is a perfect microcosm and reflection of an unhealthy society. How might we redesign schools such that they become sites of social transformation, rather than reproduction? How might we cultivate kind, ethical, empowered global citizens within our classrooms? In this paper, I will explore how Michael Apple, Paulo Freire, and bell hooks might address these questions. I will explore each scholar’s theories in an attempt to imagine what a school based in their pedagogical philosophy might look like—one that nurtures kind, ethical, and empowered global citizens.

Developing Critical Thinking in Grades 3-4

This study aims to analyze the impact of high-stakes testing practices in Asian countries, particularly Singapore, on students' learning outcomes and overall intellectual development. While the education system that focuses on standardized testing has garnered international recognition, it also raises concerns about the dominance of rote memorization over the development of critical thinking skills. This research proposes integrating Bronfenbrenner's Ecological Systems Theory and Critical Theory in Education as a framework for curriculum design to create a more balanced educational approach. The research method employed was observational and participatory, conducted over one academic year in a 3rd and 4th-grade classroom in Singapore. Data were collected through classroom observations, student writing samples, and parental involvement in the writing program. Parental involvement included weekly writing facilitation sessions and providing insights into the influence of parents on student development. The results of the study indicate that high-stakes testing practices contribute to improved academic performance but tend to limit students' critical thinking abilities. The use of a holistic approach in the curriculum, emphasizing student engagement and environmental support, proved effective in fostering analytical thinking. The implications of this study highlight the need to balance test preparation with the development of critical thinking skills in the education system, ensuring that students not only excel academically but are also equipped to think critically and adapt to global contexts.

The effect of test anxiety on high-stakes exams: evidence from an RCT using biomarkers

ABSTRACT We conduct a randomised control trial at an Italian university to investigate the impact of test anxiety on high-stakes exams. Students are subjected to two different interventions – silence and music – designed to influence their pre-test anxiety levels. We monitored and compared individual biomarkers before and after each treatment to measure anxiety changes and their influence on exam scores. Our findings revealed that decreased mean arterial and systolic pressures enhanced performance in females, particularly under the silence treatment. However, no significant effect was observed in male students. This study suggests that test anxiety could contribute to gender-based performance disparities.

Optional Exam Retakes Reduce Anxiety but may Exacerbate Score Disparities Between Students with Different Social Identities.

Use of high-stakes exams in a course has been associated with gender, racial, and socioeconomic inequities. We investigated whether offering students the opportunity to retake an exam makes high-stakes exams more equitable. Following the control value theory of achievement emotions, we hypothesized that exam retakes would increase students' perceived control over their performance and decrease the value of a single exam attempt, thereby maximizing exam performance. We collected data on exam scores and experiences with retakes from three large introductory biology courses and assessed the effect of optional exam retakes on gender, racial/ethnic, and socioeconomic disparities in exam scores. We found that Black/African American students and those who worked more than 20 h a week were less likely to retake exams. While exam retakes significantly improved student scores, they slightly increased racial/ethnic and socioeconomic disparities in scores partly because of these differences in participation rates. Most students reported that retake opportunities reduced their anxiety on the initial exam attempt. Together our results suggest that optional exam retakes could be a useful tool to improve student performance and reduce anxiety associated with high-stakes exams. However, barriers to participation must be examined and reduced for retakes to reduce disparities in scores.

The potential for using entrustable professional activities in assessing optometric clinical competence.

Qualifications required to practise optometry seek to provide assurance that those completing them have the necessary competencies to be safe and effective clinicians. Often, this assurance is gained via high stakes assessments such as objective structured clinical examinations. This paper introduces entrustable professional activities (EPAs) as a potential alternative approach to assessing optometric clinical competence. Entrustable professional activities are defined in the context of optometric practice and the potential advantages and challenges of using them are outlined.

Analyzing the Impact of the “Double Reduction” Policy on Student Well-Being and Academic Performance in Urban China

The “Double Reduction” policy, introduced in China in 2021, seeks to alleviate academic pressure on students by reducing homework loads and regulating private tutoring services. This study analyzes the policy’s impact on student well-being and academic performance in urban settings, drawing on quantitative and qualitative data from students, parents, teachers, and school administrators. The findings indicate a dual impact: a decrease in stress and anxiety levels and improvements in social behaviors and time management, but also challenges in maintaining academic discipline and performance, particularly among students preparing for high-stakes exams. While the policy promotes a more balanced approach to education, it requires further refinement and adaptive strategies to address diverse student needs effectively. The study concludes that ongoing support and monitoring are essential to achieve the policy’s long-term objectives of reducing stress while ensuring academic excellence.

Interpretation Evidence for the Multidimensional Test Anxiety Scale: A Brief Report

Standardized testing is an integral part of the English and American education systems. However, the use of high-stakes testing has unintended consequences, one of which is test anxiety. Over the last 50 years, increased attention has been directed to developing tools to identify students experiencing test anxiety. However, many test anxiety instruments have been created for research purposes rather than use within school decision-making and lack evidence for interpretation. The purpose of the current study was to support the use of the Multidimensional Test Anxiety Scale (MTAS) in applied settings by using a latent profile analysis to identify respondent groups to support score interpretation. Participants included 918 secondary students in England.