States Medical Licensing Examination Step Research Articles

Pediatric residency milestone performance is not predicted by the United States Medical Licensing Examination Step 2 Clinical Knowledge

Objectives: This study aims to show whether correlation exists between pediatric residency applicants’ quantitative scores on the United States Medical Licensing Exam Step 2 Clinical Knowledge examination and their subsequent performance in residency training based on the Accreditation Council for Graduate Medical Education Milestones, which are competency-based assessments that aim to determine residents’ ability to work unsupervised after postgraduate training. No previous literature has correlated Step 2 Clinical Knowledge scores with pediatric residency performance assessed by Milestones. Methods: In this retrospective cohort study, the United States Medical Licensing Exam Step 2 Clinical Knowledge Scores and Milestones data were collected from all 188 residents enrolled in a single categorical pediatric residency program from 2012 - 2017. Pearson correlation coefficients were calculated amongst available test and milestone data points to determine correlation between test scores and clinical performance. Results: Using Pearson correlation coefficients, no significant correlation was found between quantitative scores on the Step 2 Clinical Knowledge exam and average Milestones ratings (r = -0.1 for post-graduate year 1 residents and r = 0.25 for post-graduate year 3 residents). Conclusions: These results demonstrate that Step 2 scores have no correlation to success in residency training as measured by progression along competency-based Milestones. This information should limit the importance residency programs place on quantitative Step 2 scores in their ranking of residency applicants. Future studies should include multiple residency programs across multiple specialties to help make these findings more generalizable.

Impact of Preference Signals on Interview Selection Across Multiple Residency Specialties and Programs.

Background Program signaling is an innovation that allows applicants to express interest in specific programs while providing programs the opportunity to review genuinely interested applicants during the interview selection process. Objective To examine the influence of program signaling on "selected to interview" status across specialties in the 2022 Electronic Residency Application Service (ERAS) application cycle. Methods Dermatology, general surgery-categorical (GS), and internal medicine-categorical (IM-C) programs that participated in the signaling section of the 2022 supplemental ERAS application (SuppApp) were included. Applicant signal data was collected from SuppApp, applicant self-reported characteristics collected from the MyERAS Application for Residency Applicants, and 2020 program characteristics collected from the 2020 GME Track Survey. Applicant probability of being selected for interview was analyzed using logistic regression, determined by the selected to interview status in the ERAS Program Director's WorkStation. Results Dermatology had a 62% participation rate (73 of 117 programs), GS a 75% participation rate (174 of 232 programs), and IM-C an 86% participation rate (309 of 361 programs). In all 3 specialties examined, on average, signaling increased the likelihood of being selected to interview compared to applicants who did not signal. This finding held across gender and underrepresented in medicine (UIM) groups in all 3 specialties, across applicant types (MDs, DOs, international medical graduates) for GS and IM-C, and after controlling for United States Medical Licensing Examination Step 1 scores. Conclusions Although there was variability by program, signaling increased likelihood of being selected for interview without negatively affecting any specific gender or UIM group.

Predictors of Ophthalmology Resident Performance From Medical Student Application Materials

To determine whether elements in ophthalmology residency applications are predictors of future resident performance. This multi-institutional, cross-sectional, observational study retrospectively reviewed the residency application materials of ophthalmology residents who graduated from residency from 2006 through 2018. Resident performance was scored by 2 faculty reviewers in 4 domains (clinical, surgical, academic, and global performance). Correlation between specific elements of the residency application and resident performance was assessed by Spearman correlation coefficients (univariate) and linear regression (multivariate) for continuous variables and logistic regression (multivariate) for categorical variables. Seven ophthalmology residency programs in the US. Ophthalmology residents who graduated from their residency program. High-performing residents were a diverse group, in terms of sex, ethnicity, visa status, and educational background. Residents with United States Medical Licensing Examination Step 1 scores higher than the national average for that year had significantly higher scores in all 4 performance domains than those who scored at or below the mean (all domains P < 0.05). Residents who had honors in at least 4 core clerkships and who were members of Alpha Omega Alpha Medical Honor Society also had higher scores in all 4 performance domains (all domains P ≤ 0.04). Step 1 score (ρ=0.26, P < 0.001) and the difference between Step 1 score and the national average for that year (ρ=0.19, P = 0.009) positively correlated with total resident performance scores. Residents who passed the American Board of Ophthalmology Written Qualifying Examination or Oral Examination on their first attempt had significantly higher Step 1/2 scores (P ≤ 0.005), Ophthalmology Knowledge Assessment Program scores (P = 0.001), and resident performance scores (P ≤ 0.004). In this new landscape of increasing numbers of applicants to residency programs and changing of the Step 1 score to pass/fail, our findings may help guide selection committees as they holistically review applicants to select exceptional future residents in ophthalmology.

An analysis of osteopathic medical students applying to surgical residencies following transition to a single graduate medical education accreditation system.

Upon requests from osteopathic medical schools, the National Resident Matching Program (NRMP) Charting Outcomes were redesigned to include osteopathic medical school seniors beginning in 2018 and one joint graduate medical education (GME) accreditation system, the Accreditation Council for Graduate Medical Education (ACGME), formed in 2020. The goal of this study is to analyze the match outcomes and characteristics of osteopathic applicants applying to surgical specialties following the ACGME transition. A retrospective analysis of osteopathic senior match outcomes in surgical specialties from the NRMP Main Residency Match data from 2020 to 2022 and the NRMP Charting Outcomes data from 2020 to 2022 was performed. For surgical specialties, results show matching increased as United States Medical Licensing Examination (USMLE) Step 2 CK (clinical knowledge) and Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2 CE (cognitive evaluation) scores increased along with the number of contiguous rankings (p<0.001). The greatest indication for matching looking at scores alone were those who scored greater than 230 on Step 2 CK compared to below (p<0.001) and above 650 on Level 2 CE (p<0.001). However, those who scored 240 (p=0.025) on Step 2 CK were just as likely to match as those who scored 250 (p=0.022) when compared to those who scored below those scores. Increasing research involvement had little to no significance with the likelihood of matching across most surgical subspecialties. Our study demonstrates that there are unique thresholds for Step 2 CK scores, Level 2 CE scores, and the number of contiguous ranks for each surgical specialty that, when reached, are significantly associated with match success. Although certain board score delineations are linked with higher match success rates, the rates level off after this point for most surgical specialties and do not significantly increase further with higher scores. In addition, thresholds within contiguous ranks for increasing match likelihood exist and vary across surgical specialties. Overall, this study highlights that the quantitative metrics utilized to assess applicants lack the correlation reported historically, and the data presently available need to be more substantiated.

Trends in successfully matched neurosurgery residency applicants.

The United States Medical Licensing Examination (USMLE) Step 1 recently transitioned to a pass/fail outcome, renewing interest in how programs select neurosurgical residents. This study investigates the association between match status and key academic metrics over time. Data are from the National Resident Matching Program from 2009 to 2022 for matched and unmatched US allopathic (MD) seniors. Investigated metrics included the mean number of contiguous ranks; mean number of distinct specialties ranked; mean USMLE Step 1 and Step 2 Clinical Knowledge (CK) scores; mean number of abstracts, presentations, and publications; mean number of research, work, and volunteer experiences; Alpha Omega Alpha status; attendance at a top 40 NIH-funded institution; PhD degree; and other degree. Multiple linear regression without an interaction term was used to evaluate how these have varied between the two groups during the study period and whether there is a difference between unmatched and matched MD seniors applying for a neurosurgical residency. Multiple linear regression with an interaction term was then used to test whether the difference in variables between the two groups changed over time. Regardless of match status, MD senior neurosurgical residency applicants exhibited an increase in USMLE Step 1 and 2 scores; average research experiences; abstracts, presentations, and publications; and work and volunteer experiences (p < 0.001). The percentage of applicants from a top 40 NIH-funded school decreased (p = 0.018), and the percentage who held an additional degree increased (p = 0.007). Between groups, there were significant differences in all categories except work experiences and other degree obtained. Over time, the difference between USMLE Step 2 scores between matched and unmatched seniors diminished (p = 0.027); in contrast, the difference in abstracts, presentations, and publications between the two groups increased over time (p < 0.001). From 2009 to 2022, neurosurgical residency applicants grew in their achievements across many metrics. In the advent of Step 1 becoming pass/fail, this study suggests that Step 2 is not viewed by programs as an adequate replacement. However, the Step 1 grading transition may serve as an opportunity for other factors to be considered that may better predict success in neurosurgical residency.

Reducing implicit bias in the neurosurgery application and interview process: a single-institution experience

OBJECTIVE The neurosurgical match is a challenging process for applicants and programs alike. Programs must narrow a wide field of applicants to interview and then determine how to rank them after limited interaction. To streamline this, programs commonly screen applicants using United States Medical Licensing Examination (USMLE) Step scores. However, this approach removes nuance from a consequential decision and exacerbates existing biases. The primary objective of this study was to demonstrate the feasibility of effecting minor modifications to the residency application process, as the authors have done at their institution, specifically by reducing the prominence of USMLE board scores and Alpha Omega Alpha (AΩA) status, both of which have been identified as bearing racial biases. METHODS At the authors’ institution, residents and attendings holistically reviewed applications with intentional redundancy so that every file was reviewed by two individuals. Reviewers were blinded to applicants’ photographs and test scores. On interview day, the applicant was evaluated for their strength in three domains: knowledge, commitment to neurosurgery, and integrity. For rank discussions, applicants were reviewed in the order of their domain scores, and USMLE scores were unblinded. A regression analysis of the authors’ rank list was made by regressing the rank list by AΩA status, Step 1 score, Step 2 score, subinternship, and total interview score. RESULTS No variables had a significant effect on the rank list except total interview score, for which a single-point increase corresponded to a 15-position increase in rank list when holding all other variables constant (p &lt; 0.05). CONCLUSIONS The goal of this holistic review and domain-based interview process is to mitigate bias by shifting the focus to selected core qualities in lieu of traditional metrics. Since implementation, the authors’ final rank lists have closely reflected the total interview score but were not significantly affected by board scores or AΩA status. This system allows for the removal of known sources of bias early in the process, with the aim of reducing potential downstream effects and ultimately promoting a final list that is more reflective of stated values.

Applicants Performance in Interview for a General Surgery Residency Pre- and During Coronavirus Disease-19 Pandemic

IntroductionThe coronavirus disease-19 (COVID-19) pandemic has created a shift from traditional in-person interviews for a residency position to a virtual platform over the last 2 y (2020-2021). With this significant shift, there is controversy about how this may affect applicant performance. We hypothesized that applicants scored higher during virtual interviews than in-person interviews. MethodsApplicants interviewed for postgraduate year 1 positions at a single university-based residency program over six interview cycles were divided into group A (4 cycles, pre-COVID-19) and group B (2 cycles, during COVID-19). All candidates were interviewed by faculty and senior surgical residents and scored on letters of recommendation (LOR), personal statement (PS), research (RS), the way the candidate represented themselves (RP), interest in the geographic area (IN), and degree of connection between candidate and interviewer (CN). Differences in mean scores between candidates in these two groups were assessed using an independent-sample t-test. ResultsOf 332 total applicants, 187 (56.3%) were in group A (cycles 2016-2019) and 145 (43.7%) were in group B (cycles 2020-2022). Group-B had higher United States Medical Licensing Examination Step 2 and Step 1 mean (standard deviation) scores (251.1 ± 10.6 versus 247.6 ± 11.8, P = 0.006 and 241.1 ± 10.3 versus 238.4 ± 10.4, P = 0.02). Group A scored higher median [interquartile range] in CN (4.4 [4.1, 4.6] versus 4.2 [3.9, 4.6], P = 0.005) and IN (4.3 [4.0, 4.5] versus 4.1 [3.8, 4.4], P = 0.002), while group B scored higher in LOR (4.3 [4.1, 4.6] versus 4.2 [3.9, 4.5], P = 0.011) and PS (4.3 [4.1, 4.4] versus 4.1 [3.8, 4.4], P = 0.001). No differences were noted between groups A and B in RS (3.9 [3.4, 4.5] versus 4.1 [3.5, 4.5], P = 0.108) or RP (4.4 [4.1, 4.7] versus 4.4 [4.2, 4.8], P = 0.317). No differences in the median [interquartile range] total scores of personal characteristic categories (25.1 [23.7, 26.5] versus 25.1 [23.8, 26.7], P = 0.703) were observed. ConclusionsFrom a program perspective, there were no differences between groups; however, candidates in the pre-COVID era scored higher in CN and IN, while those who interviewed virtually had higher scores in LOR and PS. Applicants perceptions may differ regarding in-person versus virtual interviews.

Association of Interview and Holistic Review Metrics With Resident Performance-Related Difficulties in an Internal Medicine Residency Program.

Background The utility of traditional academic factors to predict residency candidates' performance is unclear. Many programs utilize holistic review processes assessing applicants on an expanded range of application and interview characteristics. Determining which characteristics might predict performance-related difficulty in residency is needed. Objective We aim to elucidate factors associated with residency performance-related difficulty in a large academic internal medicine residency program. Methods In 2022, we conducted a retrospective cohort study of Electronic Residency Application Service and interview data for residents matriculating between 2018 and 2020. The primary outcome was a composite of performance-related difficulty during residency (referral to the Clinical Competency Committee; any rotation evaluation score of 2 out of 5 or lower; and/or a confidential "comment of concern" to the program director). Logistic regression models were fit to assess associations between resident characteristics and the composite outcome. Results Thirty-eight of 117 residents met the composite outcome. Gold Humanism Honor Society (odds ratio [OR] 0.24, 95% confidence interval [CI] 0.16-0.87) or Alpha Omega Alpha (OR 0.36, 95% CI 0.14-0.99) members were less likely to have performance-related difficulty, as were residents with higher United States Medical Licensing Examination Step 2 Clinical Knowledge scores (OR 0.97, 95% CI 0.47-1.00). One-point increases in general faculty overall interview score, leadership competency score, and leadership overall score were associated with 41% to 63% lower odds of meeting the composite outcome. Interview or file review "flags" had an OR of 2.82 (95% CI 1.37-5.80) for the composite outcome. Conclusions Seven metrics were associated with the composite outcome of resident performance-related difficulty.

Perspectives from two recent medical school graduates on exposure to pathology during undergraduate medical education: A narrative inquiry

The field of pathology is facing an inflection point where the demand for pathology services is not being met by a corresponding rise in recruitment into the field. Many of the myths about the field of pathology have been dispelled elsewhere, but there have not been many formal accounts of the experience medical students’ face when finding their path to pathology. Because of challenges in the visibility of pathology as a specialty and not simply a subject required for United States Medical Licensing Examination Step 1, students tend to fall into one of two categories: early differentiators or late discoverers. Here, we provide anecdotal accounts of these two paths at institutions with different curricular designs and provide a first-hand account of the challenges we faced and opportunities discovered in our journeys to pathology. Based on these experiences, we offer suggestions for ways to address some of the issues medical students must navigate when trying to explore pathology in curricula not built for such exploration.

A Survey of Otolaryngology Residency Programs on Adapting to the United States Medical Licensing Examination (USMLE) Step 1 Transitioning to Pass/Fail.

Objectives In February 2020, the National Board of Medical Examiners (NBME) announced that the United States Medical Licensing Examination (USMLE) Step 1 licensing examination would changefrom a numerical score to Pass/Fail (P/F). After implementation, many believe that USMLE-Step 2-Clinical Knowledge (CK) will become an important metric for students applying to otolaryngology (ENT). The purpose of this study is to determine factors important to resident selection after these changes. Methods A survey containing 15 questions related to resident selection practices and how changing USMLE Step 1 to P/F would impact future resident selection was designed. It was distributed to all ENT residency programs accredited by the Accreditation Council for Graduate Medical Education (ACGME). Results Forty percent of programs responded; 66% (95% confidence interval (CI): 51.1%-78.4%) felt that changing Step 1 scoring would not lead to students being more prepared for clinical rotations; 55% believe class rank will increase in significance (95% CI: 35.7%-64.3%). There was also an increase in the importance of Step 2 CK, which had a mean ranking of 10.67 prior to changes in Step 1 scoring and increased to 7.80 after P/F. Conclusions The changes in Step 1 scoring will likely lead to increasing importance of other objective measures like class rank or Step 2 CK. This may defeat the intended purpose put forth by the NBME. Therefore, further guidance on measures correlated with student performance as a resident will be integral to the selection process.

Impact of the USMLE Step 1 and COMLEX Level 1 transition to Pass/Fail on osteopathic medical student stress levels and board preparation.

The United States Medical Licensing Examination (USMLE) Step 1 and Comprehensive Osteopathic Medical Licensing Exam (COMLEX) Level 1 transitioned from a numeric scoring system to a Pass/Fail designation in 2022. This transition intended to decrease stress, improve medical student well-being, and encourage residency program directors to emphasize other aspects of residency applications. Pass/Fail score transitions in the undergraduate medical education curriculum have improved medical student psychological well-being and satisfaction; whether these same benefits translate to the board examination period is unknown. The objectives of this study are to assess the impact of USMLE Step 1 and COMLEX Level 1 grade scaletransition on medical student stress, wellness, board preparation decisions, and future residency selection processes. Investigators hypothesized that students under the Pass/Fail designation would experience less stress during the intensive study period leading up to USMLE Step 1 andCOMLEX Level 1 and devote more time to other aspects oftheir residency applications. To examine the impact on osteopathic medical student (OMS) stress and approach to board preparation, two surveys were administered to Rocky Vista University College of Osteopathic Medicine (RVU-COM) students before (Class of 2023) and after (Class of 2024) the transition to a Pass/Fail designation. All students within the RVU-COM Classes of 2023 and 2024 were invited to participate. The Cohen Perceived Stress Scale (PSS-10) was administered at the beginning of the focused board study period in May 2021 and 2022 to the Class of 2023 and 2024, respectively. The investigator-designed Licensing Exam Questionnaire (LEQ), meant to capture board preparation patterns, residency application perspectives, and wellness during examination preparation, was administered immediately after the board examination deadline in July 2021 and 2022 to the Class of2023 and 2024, respectively. Statistical analysis included theuse of independent t tests (numeric variables) and chi-square tests (categorical data). This project was considered exempt from full Institutional Review Board review. Approximately one-third of the Class of 2023 (PSS-10: n=86; LEQ: n=93) and 2024 (PSS-10=89; LEQ: n=92) responded. No difference was detected in mean PSS-10 score, 20.14 (SD=7.3) compared to 19.92 (SD=6.56) for the Class of2023 and 2024 (p=0.84), respectively. The Class of 2023 reported more weeks studying (mean 6.27weeks, SD=0.79) vs.the Class of 2024 (mean 5.44weeks, SD=0.007), p<0.001, more practice examinations taken X 2 (1, n=182)=13.75, p<0.001, and a greater proportion scheduled examinations after June 20 X 2 (1, n=182)=29.01, p<0.001. No difference existed in hours studying per day, sequence of Step 1/Level 1, time between examinations, money spent, or type of study resources utilized. The transition of USMLE Step 1 and COMLEX Level 1 to a Pass/Fail designation did not reduce stress for OMSs at a single, multicampus COM. Respondents, however, altered board preparation practices in meaningful ways. Asstudent behaviors and board-study patterns emerge, these insights must be connected to outcomes in the future.

Unintended Consequence: Diversity as a Casualty of Eliminating United States Medical Licensing Examination Step 1 Scores

PurposeThe purpose of this study was to use a discrete-choice experiment to model the trade-offs evaluators make between academic attributes and demographics when the United States Medical Licensing Examination (USMLE) Step 1 switches to pass/fail. MethodsA discrete-choice experiment was administered to faculty members from a geographically diverse mix of 14 academic and community radiology departments in the United States from August through November 2020. Reviewers reviewed 10 applicant pairs with numeric Step 1 scores (part 1) and 10 applicant pairs with a pass Step 1 result (part 2). Applicant attributes included medical school rank, gender, race/ethnicity, USMLE Step 1 score, USMLE Step 2 score, class rank, clerkship honors, and publications. Conditional logistic regression modeled the influence of attribute levels. ResultsTwo hundred twelve evaluators completed the study (response rate 59%). The most influential attribute was Step 1 score in part 1 and medical school rank in part 2. The relative importance of race/ethnicity and gender decreased by 25% and 29%, respectively, when Step 1 switches to pass/fail. Evaluators weigh race/ethnicity the strongest when applicants have the same Step 1 score (preference weights of 0.85 for African American, 1.42 for Hispanic, and 0 for White and Asian applicants). Race/ethnicity is relatively more important when Step 1 scores are higher (preference weights of 1.58 for African American, 0.90 for Hispanic, and 0 for White and Asian applicants). ConclusionsThe loss of numeric Step 1 scores reduced the residency evaluator preference for diversity. Reviewers prioritize underrepresented-in-medicine applicants when Step 1 scores are higher and comparable with White and Asian applicants.

Key factors for residency interview selection from the National Resident Matching Program: analysis of residency Program Director surveys, 2016-2020.

As the number of medical school graduates continues to outpace the available residency training positions, applying for residency in the United States has become a highly competitive process, often associated with a low rate of selection and invitation for interview. The National Resident Matching Program (NRMP) Program Director survey provides data assessing factors considered by Program Directors (PD) in selecting and inviting candidates for interview. Assessing the evolution of these factors over time is efficacious to inform and guide prospective applicants toward improving preparation for residency application. We aim to synthesize NRMP data showing factors that PDs reported and rated as important in their decision to select and invite applicants for interview. Data from residency PD surveys from 2008 to 2021were accessed, but after applying inclusion/exclusion criteria, only the data from 2016 to 2020 were reviewed and analyzed. The NRMP survey reports provided two metrics that characterized PDs' evaluation of the residency factors for interview, namely, "percent citing factor" and "average rating" on a 0 to 5 Likert-type scale. These two metrics were combined into an aggregate measure of importance (AI), and another measure of relative importance (RI) was constructed from normalizing the AI of each individual factor to the sum of the AI within each survey year. The top ranked factors were United States Medical Licensing Examination (USMLE) Step 1/Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 1, Letter of Recommendation (LOR) in the specialty, Medical Student Performance Evaluation (MSPE/Dean's Letter), and USMLE Step 2 Clinical Knowledge (CK)/COMLEX Level 2 Cognitive Exam (CE) score, any failed attempt in USMLE/COMLEX, and perceived commitment to specialty. Factors rising in importance were Audition Elective/Rotation Within Your Department, Personal Statement (PS), Perceived Commitment to Specialty, Perceived Interest in Program, LOR in the Specialty, Other Life Experience, and Personal Prior Knowledge of the Applicant. Factors with declining importance were Interest in Academic Career, Awards or Special Honors in Basic Sciences, Graduate of Highly Regarded US Medical School, Awards or Special Honors in Clinical Clerkships, Lack of Gaps in Medical Education, Awards or Special Honors in Clerkship in Desired Specialty, and Consistency of Grades. Compared to the 2021PD survey, our findings show continued predictive consistency, particularly related to specialty and program commitment. The factors identified for the selection of medical school graduates for interview into a residency program reveal that PDs move toward a more integrated approach. Specifically, PDs are placing increasing emphasis on factors that border on subjective qualities more so than the more traditional, quantitative, and objective metrics. Medical students and educators need to continually apprise themselves of the NRMP data to inform students' preparation endeavors throughout medical school to strengthen their application portfolios and enhance their competitiveness for the matching process.

Authors' Reply to: Additional Considerations for US Residency Selection After Pass/Fail USMLE Step 1. Comment on "The US Residency Selection Process After the United States Medical Licensing Examination Step 1 Pass/Fail Change: Overview for Applicants and Educators".

Authors' Reply to: Additional Considerations for US Residency Selection After Pass/Fail USMLE Step 1. Comment on "The US Residency Selection Process After the United States Medical Licensing Examination Step 1 Pass/Fail Change: Overview for Applicants and Educators".

Additional Considerations for US Residency Selection After Pass/Fail USMLE Step 1. Comment on "The US Residency Selection Process After the United States Medical Licensing Examination Step 1 Pass/Fail Change: Overview for Applicants and Educators".

Additional Considerations for US Residency Selection After Pass/Fail USMLE Step 1. Comment on "The US Residency Selection Process After the United States Medical Licensing Examination Step 1 Pass/Fail Change: Overview for Applicants and Educators".

The Importance of Research Experience With a Scoreless Step 1: A Student Survey at a Community-Based Medical School.

As of January 26, 2022, the United States Medical Licensing Examination (USLME) step 1 exam went from a scored test to pass-fail step 1 (PFS1). The authors were interested in surveying medical students at a community-based medical school to observe their perceptions of the importance of student research given this recent change. A Qualtrics survey was disseminated to medical students (years 1-4) via school emails. Data were analyzed using the Mann-Whitney test to assess Likert scale scores, and narrative comments were grouped as qualitative feedback. Survey dissemination and analysis of data were both conducted at a large community-based medical school. The survey sampled 104 students categorized into pre-clerkship (PC) and clerkship (CL) years, with a response rate of 33%. A contradiction was found, as indicated by the higher number (p = 0.047) of clerkship students interested in Primary Care/Family medicine residency compared to pre-clerkship students at 41% and 59%, respectively. Whereas participants who indicated they are interested in pursuing a competitive specialty for residency were 51% of pre-clerkship students over 41% of clerkship students (p = 0.047).Additionally, given the assessment change to pass/fail, students did in fact believe that residencies would now view research as a higher assessed component than before (79% pre-clerkship and 72% clerkship). However, a minority of students said that they increased their research efforts (41% and 47%). Most students supported the research opportunity improvements proposed in our survey. Efforts to make the step 1 exam pass/fail may have alleviated some stress related to performance but may have increased the perception of the importance of other components in a student's residency application. Our survey highlights how medical students at a community-based medical school perceive this change and how it has affected their research efforts.

Evaluating the Impact of a Medical School Student-Run Research Organization on Scholarly Activity.

Introduction The United States Medical Licensing Examination Step 1 change to Pass/Fail scoring has motivated medical students to pursue more research opportunities. To support students, a student-led organization was created at an allopathic medical school, offering initiatives such as workshops, mentorship, and research projects. Here, we evaluate its impact on medical student research. Methods An observational survey study was conducted to assess students' research involvement and productivity and their sense of support, confidence, and comfort in pursuing research at an institution during the first two years of medical school. These variables were compared between three contiguous classes of students and between club members and non-members. Analyses included t-tests, Chi-square tests, and ANOVA, among others. Results Findings revealed that organization membership was associated with an increased number of research projects. Club members (M= 4.49) reported a significantly greater number of projects compared to non-members (M= 4.49) (p= 0.002). Students who had access to the organization during their preclinical years (M= 4.38) reported significantly more projects compared to students whose preclinical years were before the organization's conception (M= 2.21) (p= 0.041). However, research productivity and feelings of support and confidence in research did not differ by class or club membership. Conclusions Club members engaged in a greater number of research projects as compared to non-members and students who had access to the organization during their preclinical years. The implementation of similar organizations at every medical school can allow more students to engage in scholarly work.

Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study

BackgroundWith the elimination in 2021 of the United States Medical Licensing Examination Step 2 Clinical Skills test, it is incumbent upon U.S. medical schools to develop local validated assessments of clinical reasoning. While much attention has been paid to summative exams for graduating students, formative exams for pre-clerkship students have not been well studied.MethodsWe applied the University of Illinois at Chicago College of Medicine (UIC-COM) Patient Note (PN) Scoring Rubric to templated PNs written by 103 pre-clerkship students for two cases in an objective structured clinical examination (OSCE) at the Yale School of Medicine. The rubric consists of four section scores (Documentation, Differential Diagnosis, Justification, and Workup, each scored 1 to 4) and a composite score (scaled 23 to 100). We calculated item discrimination for each section score and Cronbach’s alpha for each case. We surveyed students about their experience writing the templated PN.ResultsMean Documentation, Differential Diagnosis, Justification, Workup, and composite scores for case A were 2.16, 1.80, 1.65, 2.29, and 47.67, respectively. For case B, the scores were 2.13, 1.21, 1.60, 1.67, and 40.54, respectively. Item discrimination ranged from 0.41 to 0.80. Cronbach’s alpha for cases A and B was 0.48 and 0.25, respectively. A majority of the students felt that the exercise was useful and appropriate to their level of training.ConclusionsDespite performing poorly, pre-clerkship students found the note-writing task beneficial. Reliability of the scoring rubric was suboptimal, and modifications are needed to make this exercise a suitable measure of clinical reasoning.

The Pass/Fail Effect: A Longitudinal Study of United States Medical Licensing Examination (USMLE) Step 1 Performance Over a Decade.

Objectives This study aimed to analyze the impact of theUnited States Medical Licensing Examination (USMLE) Step 1 transition to a pass/fail scoring system in 2022 on the performance of first-time test takers in three distinct groups: Doctor of Osteopathy (DO) and Doctor of Medicine (MD) examinees from US/Canadian schoolsand examinees from non-US/Canadian schools. The analysis spans a decade-long period from 2012 to 2022, offering insights into the implications of this pivotal change in medical education. Methods We analyzed the performance of first-time USMLE Step 1 examinees from US/Canadian MD and DO programsand non-US/Canadian schools from 2012 to 2022, including the transition year to a pass/fail scoring system. Data were obtained from USMLE performance data reports and organized into annual contingency tables. Descriptive statistics and comparative analysis were used to identify trends and differences in performance across the groups. Data visualization techniques were employed to illustrate these findings, and the results were contextualized within the broader changes in medical education. Results In 2021, first-time takers from US/Canadian MD and DO Degree programs had passrates of 96% and 94%, respectively, while non-US/Canadian schools had a pass rate of 82%. However, in 2022, these rates dropped to 93%, 89%, and 74%, respectively. The most significant relative decline was observed among non-US/Canadian Schools' first-time takers, with an 8% decrease. Repeaters consistently had lower pass rates across all groups. Conclusion The study reveals a notable decline in pass rates following the transition to pass/fail scoring, although this is based on just one year of data. This underscores the importance of students not rushing into the exam and dedicating sufficient time for preparation. The potential impact of this research could be transformative for medical education, but more years of data post-transition will be needed to confirm these initial findings. These findings serve as a reminder that the change in scoring does not diminish the rigor of the exam, prompting students to approach their studies with diligence and patienceand potentially paving the way for systemic improvements in medical education and healthcare delivery worldwide.

USMLE Step 2 CK Scores and Radiology Residency Applications: Does it Affect Diversity in Selection of Trainees?

As residency programs in diagnostic radiology aspire to broaden trainee diversity, reliance on certain criteria may affect the selection of candidates from underrepresented groups. With the conversion of reporting of the United States Medical Licensing Examination (USMLE) Step 1 scores to pass/fail, programs may rely more on numerical USMLE Step 2 Clinical Knowledge (CK) scores. The purpose of our investigation is to assess the effects of Step 2 CK scores on the selection of underrepresented minority (URM) and female candidates. Applications from United States senior allopathic medical students to a radiology residency program from the 2021-2023 National Residency Matching Program cycles were analyzed. Subjects were classified as male or female and URM or non-URM by self-identification. Step 2 CK scores were compared and the use of cutoff scores was examined for disparate effects. 1017 subjects fulfilled the entry criteria. There were 721 males and 296 females, with 164 URM and 853 non-URM candidates. Comparing males to females, there was no significant difference in the mean score (p=0.21) and no disparate effects of cutoff scores. There was a significant difference between the mean score of URM versus non-URM candidates of eight points (p<0.00011). The use of cutoffs showed a disparate effect on URM candidates with a cutoff score of 250 (average score for 2022 matched applicants) excluding 71% of URM candidates while excluding only 46% of non-URM candidates. Reliance on USMLE Step 2 CK scores to screen applications for radiology residency can disadvantage URM candidates. Females are not adversely affected.