Low Stakes Research Articles

Differential Performance in High Versus Low Stakes Tests: Evidence from the Gre Test*

Abstract We study how different demographic groups respond to incentives by comparing their performance in ‘high’ and ‘low’ stakes situations. The high stakes situation is the Graduate Record Examination (GRE), and the low stakes situation is a voluntary experimental section of the GRE. We find that males exhibit a larger drop in performance between the high and low stakes examinations than females, and that whites exhibit a larger drop in performance than minorities. Differences between high and low stakes tests are partly explained by the fact that males and whites exert lower effort in low stakes tests compared with females and minorities.

Stakes in Testing: Not a Simple Dichotomy but a Profile of Consequences That Guides Needed Evidence of Measurement Quality

Testing programs are often classified as high or low stakes to indicate how stringently they need to be evaluated. However, in practice, this classification falls short. A high‐stakes label is taken to imply that all indicators of measurement quality must meet high standards; whereas a low‐stakes label is taken to imply the opposite. This approach can result in inappropriate allocation of resources and inadequate attention to needed evidence. We argue that “stakes” are better thought of as a profile of consequences. We suggest generalizable criteria for evaluating and responding to stakes in testing, with applications to licensure, employment, and K–12 accountability testing.

Getting to Beta

Libraries and archives are increasingly producing subject-based digital collections alongside, but separate from, their main digital collections. These smaller projects are often treated as digital one-offs; they are created, launched, promoted, and then largely forgotten. The authors of this study argue that small-scale digital collections instead be treated as test cases for their institutions’ main digitization programs. Because they are lightweight and have relatively low stakes, these collections get pushed through the system quickly and can illuminate its workings and shortcomings in a snapshot form. The authors treat their own experience in developing the Animal Welfare Act History Digital Collection at the National Agricultural Library as a case study in using a digital collection to test and revise an institution’s digitization program. In so doing, this study suggests how agile projects like the AWAHDC can be core components in digital curation policies and their implementation.

The Effects of Effort Monitoring With Proctor Notification on Test-Taking Engagement, Test Performance, and Validity

ABSTRACT When we administer educational achievement tests, we want to be confident that the resulting scores validly indicate what the test takers know and can do. However, if the test is perceived as low stakes by the test taker, disengaged test taking sometimes occurs, which poses a serious threat to score validity. When computer-based tests are used, disengagement can be detected through occurrences of rapid-guessing behavior. This empirical study investigated the impact of a new effort monitoring feature that can detect rapid guessing, as it occurs, and notify proctors that a test taker has become disengaged. The results showed that, after a proctor notification was triggered, test-taking engagement tended to increase, test performance improved, and test scores exhibited higher convergent validation evidence. The findings of this study provide validation evidence that this innovative testing feature can decrease disengaged test taking.

Team SAFE: A large-scale interprofessional simulation-based TeamSTEPPS® curriculum

Abstract Medical errors have been shown to be the third leading cause of death in the United States9 and poor communication is the root cause for more than two thirds of sentinel events.10 Several early Institute of Medicine (IOM) reports recommend interprofessional team training as a way to mitigate medical errors, with a 2003 report specifically recommending simulation as a valuable format for achieving this goal.1–3 In order to reduce medical errors and address the Quadruple Aim4 of healthcare, health professions students must graduate equipped with tools to help them support interprofessional team members in delivering safe, patient-centered care once in practice. Team SAFE (Simulation and Fearlessness Education), a longitudinal, simulation-based curriculum adapted from TeamSTEPPS® (a national, evidence-based patient safety and teamwork training framework) aids students in learning and practicing critical components of teamwork and patient safety in a low stakes environment. Our easily replicable curriculum is delivered in three phases using “flipped classroom” principles and experiential learning to help interprofessional students retain the material. Learners begin with knowledge and acquisition of team-based skills through interactive online pre-work, then apply these skills in the care and observation of care of simulated patients during the course of three cases, and finally, participate in a faculty-facilitated debriefing of each case. The curriculum includes an advanced component for senior students, with more clinically complex cases and nuanced teamwork scenarios to help reinforce the TeamSTEPPS® concepts as students finish their clinical training before beginning to practice in their respective fields. In 2016–2017, 554 students from medicine, nursing, occupational therapy, pharmacy, physician assistant and radiologic sciences completed this program. This Team SAFE curricular model is easily modifiable for different professions and levels of learners, requiring minimal faculty time and resources. Although reportedly forced outside of their comfort zones by participating in the interactive simulations, students comment that they learn effective teamwork skills that have the potential to enhance their leadership skills and improve patient safety.

Differential Performance in High vs. Low Stakes Tests: Evidence from the GRE Test

We study how different demographic groups respond to incentives by comparing their performance in and stakes situations. The high stakes situation is the GRE examination and the low stakes situation is a voluntary experimental section of the GRE. We find that Males exhibit a larger drop in performance between the high and low stakes examinations than females, and Whites exhibit a larger drop in performance compared to minorities. Differences between high and low stakes tests are partly explained by the fact that males and whites exert lower effort in low stakes tests compared to females and minorities.

Aberrant striatal tracking of reward magnitude in youth with current or past-year depression.

Reward dysfunction is often present in youth with major depressive disorder (MDD), but the specific neurobiological bases underlying reward valuation deficits remain unclear. The current study examined whether adolescents and young adults with MDD track brain and behavioral responses according to relative reward magnitude-a neurocognitive valuation process known as magnitude tracking. Female adolescents and young adults ages 15-20 years (n = 56 with current or past-year MDD; n = 26 healthy controls [HCs]) completed a task during functional neuroimaging in which they could win or lose money at high stakes (+$1/-50¢) and low stakes (+20¢/-10¢). Behaviorally, HC accelerated button press responses on high stakes compared to low-stakes trials, whereas MDD did not alter response speed across stakes. Neurally, HC increased recruitment of the ventral and dorsal striatum, canonical reward-processing regions, for high-magnitude versus low-magnitude rewards. However, the MDD group did not exhibit striatal magnitude tracking for low versus high rewards-an effect independent of MDD recency, MDD symptom severity, comorbid anxiety and substance use disorders, and psychiatric medication use. In contrast, striatal recruitment for overall reward reactivity, measured by comparing striatal activity for reward and loss feedback, was similar in the MDD and HC groups. However, reward reactivity was negatively correlated with current depression symptom severity in the MDD group. Taken together, these findings suggest that whereas reward reactivity may vary with current depression severity, reward magnitude tracking may represent an important aberrant valuation process in youth with depression-independent of symptom severity and recency. This valuation deficit may have implications for maladaptive motivation and learning observed in youth with MDD. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Arbitration in classical Athens

The Classical Athenians developed two formal arbitration procedures. They assigned low stakes disputes to a panel of arbitrators, while high stakes cases were handled by a single arbitrator. Given the information aggregation benefit of collective decision making, one would have expected more individuals to be assigned to more important cases. I develop a theoretical model to provide an explanation for their design. Recognizing that arbitrator competence is endogenous, effort put into making a good decision takes time and effort. In larger groups free riding is a concern. Consequently, there exists environments where the free-riding loss is magnified in higher stakes disputes to the point where the socially optimal panel size is inversely related to the stakes involved.

Envy and the Islamic revival: Experimental evidence from Tunisia

I design and conduct a survey with an embedded lab-in-the-field experiment to test whether envy triggers popular support for the Islamic revival using a nationally representative sample of 600 Tunisians. Individuals exposed to high inequality may feel envious if they perceive that the success of others is granted rather than earned. Thus, envious individuals may be motivated to engage in religious and political activities to cope with their feelings. I trigger the sentiment of envy with a 2 × 2 design by interacting a priming video and low stakes. I find that individuals in the envy treatment donate a larger proportion of their endowment to a politically involved religious charity, my measure of support for the Islamic revival. Overall, my experimental findings, consistent with survey data, confirm the idea that envy is an important determinant of popular support for the Islamic revival, even after controlling for religiosity.

Evaluation of an elective course for preparing students to pursue postgraduate residency training

Background and purposeTo describe the effectiveness of an elective for preparing students to pursue postgraduate residency training (PGRT). Educational activity and settingA prospective, observational study of students enrolled in the “Preparation for PGRT” elective conducted between January 2015 and December 2016 was conducted. Survey tools and assessments were developed to evaluate various aspects of the course. Curriculum vitae and letters of intent were evaluated using rubrics. A thematic analysis on students’ reflections of mock interviews was performed using the copy-and-paste method in MAXQDA 12. Student demographics and survey and assessment results were characterized descriptively. Pre- and post-course responses on course content and PGRT intentions were compared using Wilcoxon signed rank and student t-tests. FindingsStudents’ self-assessed understanding of all course objectives increased from pre- to post-course (p < 0.05). Students’ self-assessments for the mock interviews and continuing education presentation revealed growth in abilities and recognition of challenges to address in the future. A significant number of students who were undecided about completing a fellowship (p = 0.0009) or graduate school (p = 0.014) after graduation prior to the course indicated they would not pursue these options. Pre- and post-course knowledge assessments were administered, which demonstrated an increase in knowledge in six of ten course focus areas and decrease in one (p < 0.05). SummaryThe elective improved students’ confidence in achieving course objectives, clarified their intentions to pursue PGRT, and allowed for initial attempts at important skills (i.e., interviewing and providing a formal presentation) in a low stakes environment.

Correlates of Student Performance during Low Stakes Simulation.

Simulation plays a vital role in nursing education, however, modifiable factors influencing students' simulation performance have not been sufficiently examined. The purpose of this study was examine relationships among anxiety, self-efficacy, nursing knowledge, and performance during simulation. The study used a mixed-methods design. Anxiety, self-efficacy, academic achievement, and performance during simulation were measured quantitatively; correlations between key variables were calculated. Qualitative data were collected during post-simulation debriefing and triangulated to inform quantitative findings. Significant relationships were identified between knowledge of nursing care and simulation performance. Student qualitative reports of heightened anxiety and lack of confidence and uncertainty contrasted from quantitative measures. Potential reasons for this finding were explored. The relationship between knowledge of nursing care and simulation performance supports the effectiveness of simulation as a means to evaluate the application of knowledge in a clinical laboratory setting. Further research is needed to explain the complex relationships between anxiety, self-efficacy, and performance during simulation.

From High to Low Stakes Politics: Should Majoritarians Embrace Countermajoritarian Constitutional Provisions?

Robert Dahl asked, How Democratic is the United States Constitution? (2001). To address this question, Dahl embraces majoritarian democracy as his normative standard. Because the framers designed the U.S. Constitution with a wide array of constraints on majorities, Dahl’s answer is, not very. Many other scholars come to similar conclusions. This paper contributes to normative democratic theory by observing that the proponents of majoritarian constitutions ignore the problem of constitutional stability. To see the problem, notice that democracy embodies numerous prescriptions: all parties must reframe from violence to secure political ends; incumbents that lose elections must hand over power. These prescriptions are not self-implementing. Most new democracies fail to follow them, and in short order. I stipulate a constitutional desideratum: regardless of a constitution's normative contents, the constitution does no one any good if it quickly fails. My positive thesis is that all successful constitutions satisfy the limit condition: to survive constitutions must limit the stakes of power. When powerful groups feel threatened by the party holding power, they are likely to support extra-constitutional action to protect themselves. Majoritarians therefore face a fundamental tradeoff. Greater responsiveness to majorities raises the stakes of politics, imply that the more majoritarian, the less stable a constitution. Therefore, majoritarians should embrace countermajoritarian features that limit the stakes, protecting those with the power to disrupt the constitution.

Immersive Technologies to Augment Anatomy Education

Correctly defining the locations and positional relationships between anatomical structures requires learners to complete complex spatial mental transformations. Learners must ultimately be able to visualize structures and their relationships in order to apply anatomical concepts in practice. However, no group of students is homogeneous. Because students have varying levels of innate spatial ability, it is necessary to develop learning tools that provide scaffolding for individuals across this spectrum. To that end our multidisciplinary team is utilizing virtual reality technologies to address this need. Anatomy Builder VR is designed to encourage learning anatomy through exploration of 3Dimensional virtual skeletons. Learners start in a hub room and then choose to build either a human or canine skeleton. Novice learners start with introductory, guided lessons that demonstrate the following: 1) the use of directional nomenclature, 2) correct anatomical orientations, 3) basic movements of the joints of the thoracic limb, and 4) the proper names of each bone. Each lesson is designed with targeted learning outcomes and allows for exploration, practice of concepts, and assessment with immediate feedback. The application also has a sandbox component where learners can assemble skeletons with minimal guidance, and a game room that allows users to test their knowledge of osteology in a low stakes and fun environment. Our goal is to harness the unique features of immersive technology to help learners create their own accurate mental representations of the features and relationships of structures within the body. Overall feedback during interaction has been positive and the next phase of the study is to recruit undergraduate students and begin assessing the impact of using Anatomy Builder VR on visual‐spatial understanding.This abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

Increasing Active Learning in the Biochemistry Classroom: Using Group Quizzes to Stimulate Discussion

A variety of pedagogical methods (such as POGIL and teaching through case studies) that encourage discussion are particularly adaptable for biochemistry courses. However, these methods often fall short when students in the classroom are not motivated to engage in discussion of the material. In the Monmouth College Biochemistry classroom, group multiple‐choice quizzes have been used to stimulate discussion and collaborative work. Students are assigned to a group of 3 or 4 students; each student takes an individual multiple‐choice quiz prior to working on the group quiz. The individual quiz allows each student to organize their thoughts before taking the quiz as a group thereby avoiding the issue of group dynamics and students who move through the quiz at different rates. The group quiz is self‐scored using an IF‐AT scorecard. With these low stakes quizzes, almost all students actively engage in the discussion of the material and take an interest in what is being discussed. Student feedback suggests that this process enhances their ability to work collaboratively and increases their comprehension of the material. Finally, due to the multiple‐choice format of these quizzes and the increased comprehension of the material, students have shown an improvement on the ACS standardized biochemistry exam since this method has been implemented.This abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

Stakes in the eye of the beholder: an international study of learners' perceptions within programmatic assessment.

ObjectivesWithin programmatic assessment, the ambition is to simultaneously optimise the feedback and the decision‐making function of assessment. In this approach, individual assessments are intended to be low stakes. In practice, however, learners often perceive assessments designed to be low stakes as high stakes. In this study, we explored how learners perceive assessment stakes within programmatic assessment and which factors influence these perceptions.MethodsTwenty‐six learners were interviewed from three different countries and five different programmes, ranging from undergraduate to postgraduate medical education. The interviews explored learners’ experience with and perception of assessment stakes. An open and qualitative approach to data gathering and analyses inspired by the constructivist grounded theory approach was used to analyse the data and reveal underlying mechanisms influencing learners’ perceptions.ResultsLearners’ sense of control emerged from the analysis as key for understanding learners’ perception of assessment stakes. Several design factors of the assessment programme provided or hindered learners’ opportunities to exercise control over the assessment experience, mainly the opportunities to influence assessment outcomes, to collect evidence and to improve. Teacher–learner relationships that were characterised by learners’ autonomy and in which learners feel safe were important for learners’ believed ability to exercise control and to use assessment to support their learning.ConclusionsKnowledge of the factors that influence the perception of assessment stakes can help design effective assessment programmes in which assessment supports learning. Learners’ opportunities for agency, a supportive programme structure and the role of the teacher are particularly powerful mechanisms to stimulate the learning value of programmatic assessment.

Practice What You Teach: A Video-Based Practicum Model of Professional Development for Elementary Science Teachers

This study examines an innovative professional development program that provides teachers with an opportunity to practice pedagogical strategies in a low stakes class- room context. Elementary teachers participated in a one-week summer Institute and two-week Practicum focused on learning strategies for facilitating scientific discourse and argumentation in their classrooms. During the Practicum, teachers taught lessons in a summer program for elemen- tary school students and engaged in daily video-based discussions to reflect on their instruction. This study identified the instructional practices that were most emphasized during the Institute and examined the extent to which teachers took up those practices during the subsequent practicum experience. A classroom vignette illustrates how one teacher engaged her students in the discourse practices, and a coaching vignette portrays her video reflection group's discussion of the episode. Findings suggest that the focal instructional practices were taken up to different degrees during the Practicum, and that opportunities for practice and reflection are potentially valuable features of professional development programs. The project illustrates the value of video as a tool for both professional development and research.

Comparing Job Applicants to Non–Applicants Using An Item–Level Bifactor Model on the Hexaco Personality Inventory

The present study evaluated the ability of item–level bifactor models (a) to provide an alternative explanation to current theories of higher order factors of personality and (b) to explain socially desirable responding in both job applicant and non–applicant contexts. Participants (46% male; mean age = 42 years, SD = 11) completed the 200–item HEXACO Personality Inventory–Revised either as part of a job application ( n = 1613) or as part of low–stakes research ( n = 1613). A comprehensive set of invariance tests were performed. Applicants scored higher than non–applicants on honesty–humility ( d = 0.86), extraversion ( d = 0.73), agreeableness ( d = 1.06), and conscientiousness ( d = 0.77). The bifactor model provided improved model fit relative to a standard correlated factor model, and loadings on the evaluative factor of the bifactor model were highly correlated with other indicators of item social desirability. The bifactor model explained approximately two–thirds of the differences between applicants and non–applicants. Results suggest that rather than being a higher order construct, the general factor of personality may be caused by an item–level evaluative process. Results highlight the importance of modelling data at the item–level. Implications for conceptualizing social desirability, higher order structures in personality, test development, and job applicant faking are discussed. Copyright © 2017 European Association of Personality Psychology

Twelve tips for developing an OSCE that measures what you want

The Objective Structured Clinical Examination (OSCE) is used globally for both high and low stakes assessment. Despite its extensive use, very few published articles provide a set of best practices for developing an OSCE, and of those that do, none apply a modern understanding of validity. This article provides 12 tips for developing an OSCE guided by Kane’s validity framework to ensure the OSCE is assessing what it purports to measure. The 12 tips are presented in the order they would be operationalized during OSCE development.

Use of Short Animal-Themed Videos to Enhance Veterinary Students' Mood, Attention, and Understanding of Pharmacology Lectures.

Professional DVM training is inherently stressful and challenging for students. This study evaluated a simple intervention-short breaks during a veterinary pharmacology lecture course in the form of funny/cute animal videos (Mood Induction Procedures, or MIP)-to assess for potential impact on students' mood, interest in material, and perceived understanding of material. Ten YouTube video clips showing cats or dogs were selected to influence students' affective states. The videos were shown in a required pharmacology class offered during the fall semester of the second year of the DVM program at a large, land-grant institution in the western US. The student cohort consisted of 133 students (20 males, 113 females). Twenty days of the course were randomly chosen for the study and ranged from weeks 2 to 13 of the semester. Sessions in which the videos were played were alternated with sessions in which no video was played, for a total of 10 video days and 10 control days. There were significant differences in all three post-class assessment measures between the experimental (video) days and the control days. Results suggest that showing short cute animal videos in the middle of class positively affected students' mood, interest in material, and self-reported understanding of material. While the results of this study are limited to one student cohort at one institution, the ease of implementation of the technique and relatively low stakes support incorporation of the MIP technique across a variety of basic and clinical science courses.

Personality‐based selection of entrepreneurial borrowers to reduce credit risk: Two studies on prediction models in low‐ and high‐stakes settings in developing countries

SummarySmall business growth is critical for economic development and poverty reduction in emerging markets, yet there remains an over $2 trillion gap in financing these entrepreneurs. This study explores the potential of personality assessments to help lenders solve this problem and lend to more entrepreneurs and contributes to psychological selection research by examining the effect of high versus low stakes on response distortions and predictive validity in a new area—entrepreneurship with a new dependent variable—paying back credit. Results of Study 1 show that personality assessments are indeed related to credit risk, but response patterns depend significantly on whether or not the assessment is taken as a mandatory part of the credit application (high stakes) or as an optional research survey after the credit has already been provided (low stakes), and predictive relationships do not generalize between these situations. In Study 2, the distributions of personality dimensions relevant for entrepreneurs applying for a credit—conscientiousness, extraversion, and integrity—are shown to be different for applicants when in high‐ versus low‐stakes settings. These findings convey several implications for the research on and practice of lending to entrepreneurs in emerging markets and offer new directions for future research.