United States Medical Licensing Examination Step Research Articles

DO seniors and IMGs have lower match probabilities than MD seniors after adjusting for specialty choice and USMLE Step 1 score.

It is unknown if US residency applicants of different educational backgrounds (US allopathic [MD], Doctor of Osteopathic Medicine [DO], and international medical graduates [IMG]) but comparable academic performance have similar match success. Our objective was to compare match probabilities between applicant types after adjusting for specialty choice and United States Medical Licensing Examination (USMLE) Step 1 scores. We performed a secondary analysis of published data in National Resident Matching Program (NRMP) reports from 2016, 2018, 2020, and 2022 for US MD seniors, DO seniors, and IMGs (US citizens and non-US citizens). We examined the 10 specialties with the most available spots in 2022. Average marginal effects from a multiple variable logistic regression model were utilized to estimate each non-MD senior applicant type's probability of matching into their preferred specialty compared to MD seniors adjusting for specialty choice, Step 1 score, and match year. Each non-MD applicant type had a lower adjusted percent difference in matching to their preferred specialty than MD seniors,-7.1 % (95 % confidence interval [CI],-11.3 to-2.9) for DO seniors,-45.6 % (-50.6 to-40.5) for US IMGs, and-56.6 % (-61.5 to-51.6) for non-US IMGs. Similarly, each non-MD applicant type had a lower adjusted percent difference in matching than MD seniors across almost all Step 1 score ranges, except for DO seniors with Step 1 scores <200 (-2.0 % [-9.5 to 5.5]). After adjusting for specialty choice, Step 1 score, and match year, non-US MD applicants had lower probabilities of matching into their preferred specialties than their US MD colleagues.

The Impact of United States Medical Licensing Examination Step 1 Transitioning To Pass/Fail on Medical Student Perception of Research Needed to Match Into One's Preferred Specialty.

Purpose To evaluate how the transition of United States Medical Licensing Examination (USMLE) Step 1 to a pass/fail scoring influenced medical student perceptions of the importance of research required to match into their preferred residency specialty. Methods A 14-item survey was distributed by e-mail to medical students at one medical school in the southeastern United States in November of 2021. Responses were compared between medical students taking USMLE Step 1 pass/fail in the future and medical students taking USMLE Step 1 for a three-digit score. Results A total of 168 medical students responded to the survey with 98 respondents who planned on taking USMLE Step 1 pass/fail (45 first-year medical students (MS1) and 53 MS2) and 70 respondents who took USMLE Step 1 for a numerical score (37 MS3and 33 MS4). There were no differences in how each cohort scored the level of importance of research in matching into their preferred residency specialty (p=0.10); however, those taking USMLE Step 1 pass/fail believe an average of 4.6 research experiences are necessary to match into their preferred residency, compared to only 3.4 research experiences for those who took it for a numerical score (p=0.04). Conclusion No statistically significant difference in the perceived importance of research in matching into one's preferred residency specialty was found between cohorts. However, the pass/fail cohort believes they will need more research experiences to match their chosen specialty than the numerical score cohort. Results could indicate that students participate in more research and extracurricular activities to be more competitive for residency applications.

Racial and Ethnic Disparities in Situational Judgment Testing Among Applicants to an Anesthesiology Residency Program

Background The Computer-Based Assessment for Sampling Personal Characteristics (CASPer) is a situational judgment test (SJT) that assesses noncognitive skills like professionalism, communication, and empathy. There are no reports of the effects of race/ethnicity and sex on CASPer scores among residency applicants. Objective We examined the effects of race/ethnicity, sex, and United States vs international medical school attendance on CASPer performance. Methods Our anesthesiology residency program required all applicants for the 2021-2022 Match cycle to complete an online video and text-based SJT (CASPer). We compared these results, reported as z-scores, with self-identified race/ethnicity, sex, United States vs international medical school attendance, and United States Medical Licensing Examination (USMLE) Step 1 scores. Results Of the 1245 applicants who completed CASPer, 783 identified as male. The racial/ethnic distribution was 512 White, 412 Asian, 106 Black, 126 Hispanic, and 89 Other/No Answer. CASPer z-scores did not differ by sex. White candidates scored higher than Black (0.18 vs -0.57, P&amp;lt;.001) and Hispanic (0.18 vs -0.52, P&amp;lt;.001) candidates. Applicants attending US medical schools scored higher than those attending international medical schools (z-scores: 0.15 vs -0.68, P&amp;lt;.001). There was no correlation between CASPer z-scores and USMLE Step 1 scores. Conclusions Our results suggest that CASPer scores favor White applicants over Black and Hispanic ones and applicants attending US medical schools over those attending international medical schools.

Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis.

The United States Medical Licensing Examination (USMLE) has been critical in medical education since 1992, testing various aspects of a medical student's knowledge and skills through different steps, based on their training level. Artificial intelligence (AI) tools, including chatbots like ChatGPT, are emerging technologies with potential applications in medicine. However, comprehensive studies analyzing ChatGPT's performance on USMLE Step 3 in large-scale scenarios and comparing different versions of ChatGPT are limited. This paper aimed to analyze ChatGPT's performance on USMLE Step 3 practice test questions to better elucidate the strengths and weaknesses of AI use in medical education and deduce evidence-based strategies to counteract AI cheating. A total of 2069 USMLE Step 3 practice questions were extracted from the AMBOSS study platform. After including 229 image-based questions, a total of 1840 text-based questions were further categorized and entered into ChatGPT 3.5, while a subset of 229 questions were entered into ChatGPT 4. Responses were recorded, and the accuracy of ChatGPT answers as well as its performance in different test question categories and for different difficulty levels were compared between both versions. Overall, ChatGPT 4 demonstrated a statistically significant superior performance compared to ChatGPT 3.5, achieving an accuracy of 84.7% (194/229) and 56.9% (1047/1840), respectively. A noteworthy correlation was observed between the length of test questions and the performance of ChatGPT 3.5 (ρ=-0.069; P=.003), which was absent in ChatGPT 4 (P=.87). Additionally, the difficulty of test questions, as categorized by AMBOSS hammer ratings, showed a statistically significant correlation with performance for both ChatGPT versions, with ρ=-0.289 for ChatGPT 3.5 and ρ=-0.344 for ChatGPT 4. ChatGPT 4 surpassed ChatGPT 3.5 in all levels of test question difficulty, except for the 2 highest difficulty tiers (4 and 5 hammers), where statistical significance was not reached. In this study, ChatGPT 4 demonstrated remarkable proficiency in taking the USMLE Step 3, with an accuracy rate of 84.7% (194/229), outshining ChatGPT 3.5 with an accuracy rate of 56.9% (1047/1840). Although ChatGPT 4 performed exceptionally, it encountered difficulties in questions requiring the application of theoretical concepts, particularly in cardiology and neurology. These insights are pivotal for the development of examination strategies that are resilient to AI and underline the promising role of AI in the realm of medical education and diagnostics.

The Associations Between United States Medical Licensing Examination Performance and Outcomes of Patient Care.

The United States Medical Licensing Examination (USMLE) comprises a series of assessments required for the licensure of U.S. MD-trained graduates as well as those who are trained internationally. Demonstration of a relationship between these examinations and outcomes of care is desirable for a process seeking to provide patients with safe and effective health care. This was a retrospective cohort study of 196,881 hospitalizations in Pennsylvania over a 3-year period (January 1, 2017 to December 31, 2019) for 5 primary diagnoses: heart failure, acute myocardial infarction, stroke, pneumonia, or chronic obstructive pulmonary disease. The 1,765 attending physicians for these hospitalizations self-identified as family physicians or general internists. A converted score based on USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores was available, and the outcome measures were in-hospital mortality and log length of stay (LOS). The research team controlled for characteristics of patients, hospitals, and physicians. For in-hospital mortality, the adjusted odds ratio was 0.94 (95% confidence interval [CI] = 0.90, 0.99; P < .02). Each standard deviation increase in the converted score was associated with a 5.51% reduction in the odds of in-hospital mortality. For log LOS, the adjusted estimate was 0.99 (95% CI = 0.98, 0.99; P < .001). Each standard deviation increase in the converted score was associated with a 1.34% reduction in log LOS. Better provider USMLE performance was associated with lower in-hospital mortality and shorter log LOS for patients, although the magnitude of the latter is unlikely to be of practical significance. These findings add to the body of evidence that examines the validity of the USMLE licensure program.

Unintended Consequence: Diversity as a Casualty of Eliminating United States Medical Licensing Examination Step 1 Scores

PurposeThe purpose of this study was to use a discrete-choice experiment to model the trade-offs evaluators make between academic attributes and demographics when the United States Medical Licensing Examination (USMLE) Step 1 switches to pass/fail. MethodsA discrete-choice experiment was administered to faculty members from a geographically diverse mix of 14 academic and community radiology departments in the United States from August through November 2020. Reviewers reviewed 10 applicant pairs with numeric Step 1 scores (part 1) and 10 applicant pairs with a pass Step 1 result (part 2). Applicant attributes included medical school rank, gender, race/ethnicity, USMLE Step 1 score, USMLE Step 2 score, class rank, clerkship honors, and publications. Conditional logistic regression modeled the influence of attribute levels. ResultsTwo hundred twelve evaluators completed the study (response rate 59%). The most influential attribute was Step 1 score in part 1 and medical school rank in part 2. The relative importance of race/ethnicity and gender decreased by 25% and 29%, respectively, when Step 1 switches to pass/fail. Evaluators weigh race/ethnicity the strongest when applicants have the same Step 1 score (preference weights of 0.85 for African American, 1.42 for Hispanic, and 0 for White and Asian applicants). Race/ethnicity is relatively more important when Step 1 scores are higher (preference weights of 1.58 for African American, 0.90 for Hispanic, and 0 for White and Asian applicants). ConclusionsThe loss of numeric Step 1 scores reduced the residency evaluator preference for diversity. Reviewers prioritize underrepresented-in-medicine applicants when Step 1 scores are higher and comparable with White and Asian applicants.

Key factors for residency interview selection from the National Resident Matching Program: analysis of residency Program Director surveys, 2016-2020.

As the number of medical school graduates continues to outpace the available residency training positions, applying for residency in the United States has become a highly competitive process, often associated with a low rate of selection and invitation for interview. The National Resident Matching Program (NRMP) Program Director survey provides data assessing factors considered by Program Directors (PD) in selecting and inviting candidates for interview. Assessing the evolution of these factors over time is efficacious to inform and guide prospective applicants toward improving preparation for residency application. We aim to synthesize NRMP data showing factors that PDs reported and rated as important in their decision to select and invite applicants for interview. Data from residency PD surveys from 2008 to 2021were accessed, but after applying inclusion/exclusion criteria, only the data from 2016 to 2020 were reviewed and analyzed. The NRMP survey reports provided two metrics that characterized PDs' evaluation of the residency factors for interview, namely, "percent citing factor" and "average rating" on a 0 to 5 Likert-type scale. These two metrics were combined into an aggregate measure of importance (AI), and another measure of relative importance (RI) was constructed from normalizing the AI of each individual factor to the sum of the AI within each survey year. The top ranked factors were United States Medical Licensing Examination (USMLE) Step 1/Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 1, Letter of Recommendation (LOR) in the specialty, Medical Student Performance Evaluation (MSPE/Dean's Letter), and USMLE Step 2 Clinical Knowledge (CK)/COMLEX Level 2 Cognitive Exam (CE) score, any failed attempt in USMLE/COMLEX, and perceived commitment to specialty. Factors rising in importance were Audition Elective/Rotation Within Your Department, Personal Statement (PS), Perceived Commitment to Specialty, Perceived Interest in Program, LOR in the Specialty, Other Life Experience, and Personal Prior Knowledge of the Applicant. Factors with declining importance were Interest in Academic Career, Awards or Special Honors in Basic Sciences, Graduate of Highly Regarded US Medical School, Awards or Special Honors in Clinical Clerkships, Lack of Gaps in Medical Education, Awards or Special Honors in Clerkship in Desired Specialty, and Consistency of Grades. Compared to the 2021PD survey, our findings show continued predictive consistency, particularly related to specialty and program commitment. The factors identified for the selection of medical school graduates for interview into a residency program reveal that PDs move toward a more integrated approach. Specifically, PDs are placing increasing emphasis on factors that border on subjective qualities more so than the more traditional, quantitative, and objective metrics. Medical students and educators need to continually apprise themselves of the NRMP data to inform students' preparation endeavors throughout medical school to strengthen their application portfolios and enhance their competitiveness for the matching process.

USMLE Step 2 CK Scores and Radiology Residency Applications: Does it Affect Diversity in Selection of Trainees?

As residency programs in diagnostic radiology aspire to broaden trainee diversity, reliance on certain criteria may affect the selection of candidates from underrepresented groups. With the conversion of reporting of the United States Medical Licensing Examination (USMLE) Step 1 scores to pass/fail, programs may rely more on numerical USMLE Step 2 Clinical Knowledge (CK) scores. The purpose of our investigation is to assess the effects of Step 2 CK scores on the selection of underrepresented minority (URM) and female candidates. Applications from United States senior allopathic medical students to a radiology residency program from the 2021-2023 National Residency Matching Program cycles were analyzed. Subjects were classified as male or female and URM or non-URM by self-identification. Step 2 CK scores were compared and the use of cutoff scores was examined for disparate effects. 1017 subjects fulfilled the entry criteria. There were 721 males and 296 females, with 164 URM and 853 non-URM candidates. Comparing males to females, there was no significant difference in the mean score (p=0.21) and no disparate effects of cutoff scores. There was a significant difference between the mean score of URM versus non-URM candidates of eight points (p<0.00011). The use of cutoffs showed a disparate effect on URM candidates with a cutoff score of 250 (average score for 2022 matched applicants) excluding 71% of URM candidates while excluding only 46% of non-URM candidates. Reliance on USMLE Step 2 CK scores to screen applications for radiology residency can disadvantage URM candidates. Females are not adversely affected.

The Matthew Effect: Prevalence of Doctor and Physician Parents among Ophthalmology Applicants.

Objective  This article determines the prevalence of physician parents among ophthalmology residency applications. Design  Retrospective, single-center cohort study. Subjects  All applicants to the University of Kentucky Ophthalmology Residency between 2018 and 2023. Methods  Residency applications were reviewed, with data collection including applicant gender, self-identified Under-Represented in Medicine (URiM) status, United States Medical Licensing Examination (USMLE) Step 1 score, USMLE Step 2 score, and whether the application identified a doctor or physician as a parent. Doctor was defined as a profession requiring a doctorate degree, and similarly, physician as a profession requiring a medical degree. Results  A total of 2,057 applications were reviewed, representing 54% of all match participants during the study period. Fourteen percent (296) of applications indicated a parent was a doctor and 12% (253) a parent was a physician. There were no differences between gender, URiM, USMLE Step 1, and Step 2 scores between applicants indicating a doctor or physician as a parent and those that did not ( p all > 0.4 and Cohen's d all < 0.02). Of the type of doctors, 85% (253) were physicians, 6% (17) optometrists, 6% (17) Doctors of Philosophy, 3% (8) dentists, 1% (1) pharmacist, and 1% (1) veterinarian. Eighty-six percent (217) of applications with a physician parent provided the type of physician, with ophthalmologist the most common (93, 43%). Ninety-eight percent (249) of applications with a physician parent provided the gender of the parent, with father (168, 68%) more common than mother (42, 17%) or both parents (39, 16%). Conclusion  Physician parents are substantially overrepresented in ophthalmology residency applicants. This raises concerns regarding diversity and inclusion efforts for recruitment in medicine.

Medical Students' Socioeconomic Status and Academic Performance in Medical School.

Background Students from lower socioeconomic groups tend to underestimate their chances of acceptance to medical school and their likelihood of success once admitted. Objective The objective of this study is to determine if socioeconomic disadvantage status is linked to lower medical college admission test (MCAT) scores and academic performance in medical school. Methods Using the Association of American Medical Colleges (AAMC) education/occupation (EO) indicator, we compared economically disadvantaged students to students with no financial disadvantage on the MCAT, Phase 1 National Board of Medical Examiners (NBME), United States Medical Licensing Examination (USMLE) Step 1, Phase 2 NBME, and USMLE Step 2 test scores. Results Medical students in the disadvantaged group scored significantly lower on the MCAT than students with no financial disadvantage. The disadvantaged group showed a non-significant lower trend in performance until USMLE Step 2. Conclusions Applicants from lower socioeconomic backgrounds may perform lower on their MCAT and early medical school benchmark exams, but they appear to catch up with and even surpass their peers on their USMLE Step 2 examination.

Increasing Diversity in the Physician Workforce: Pathway Programs and Predictive Analytics.

Lack of diversity in the physician workforce has well-documented negative impacts on health outcomes. Evidence supports the use of pathway or pipeline programs to recruit underrepresented in medicine students. However, data on how a pathway program should deliver instruction are lacking. This report describes a multiyear project to build such a system with the goal of increasing diversity within medical school cohorts and ultimately the physician workforce. In the 2015-2016 academic year, the Ponce Health Sciences University started a 3-phase project to create a data-driven medical school feeder system by coupling a pathway program with predictive analytics. Phase 1 launched the pathway program. Phase 2 developed and validated a predictive model that estimates United States Medical Licensing Examination (USMLE) Step 1 performance. Phase 3 is underway and focuses on adoption, implementation, and support. Data analysis compared 2 groups of students (pathway vs direct) across specific factors, including Medical College Admission Test (MCAT) score, undergraduate grade point average (GPA), first-generation status, and Step 1 exam performance. Statistically significant differences were found between the 2 groups on the MCAT exam and undergraduate GPA; however, no significant differences were found between groups for first-generation status and performance on the Step 1 exam. This finding supports the authors' hypothesis that although pathway students have significantly lower mean MCAT exam scores compared with direct students, pathway students perform just as well on the USMLE Step 1 exam. Next steps include expanding the project to another campus, adding more socioeconomic status and first-generation data, and identifying best curricular predictors. The authors recommend that medical school programs use pathway programs and predictive analytics to create a more data-centered approach to accepting students with the goal of increasing physician workforce diversity.

Medical School Admissions: Focusing on Producing a Physician Workforce That Addresses the Needs of the United States.

The aging population, burnout, and earlier retirement of physicians along with the static number of training positions are likely to worsen the current physician shortage. There is an urgent need to transform the process for selecting medical students. In this Invited Commentary, the authors suggest that to build the physician workforce that the United States needs for the future, academic medicine should focus on building capacity in 3 overarching areas. First, medical schools need to develop a more diverse pool of capable applicants that better matches the demographic characteristics of health care trainees with those of the population, and they need to nurture applicants with diverse career aspirations. Second, medical schools should recalibrate their student selection process, aligning criteria for admission with competencies expected of medical school graduates, whether they choose to become practicing clinicians, physician-scientists, members of the public health workforce, or policy makers. Selection criteria that overweight the results of standardized test scores should be replaced by assessments that value and predict academic capacity, adaptive learning skills, curiosity, compassion, empathy, emotional maturity, and superior communication skills. Finally, to improve the equity and effectiveness of the selection processes, medical schools should leverage innovations in data science and generative artificial intelligence platforms. The ability of ChatGPT to pass the United States Medical Licensing Examination (USMLE) demonstrates the decreasing importance of memorization in medicine in favor of critical thinking and problem-solving skills. The 2022 change in the USMLE Step 1 to pass/fail plus the exodus of several prominent medical schools from the U.S. News and World Report rankings have exposed limitations of the current selection processes. Newer approaches that use precision education systems to leverage data and technology can help address these limitations.

Educational Videos Versus Question Banks: Maximizing Medical Student Performance on the United States Medical Licensing Examination Step 1 Exam.

Objective The aim of this research was to determine if medical students' use of the active study strategy of working practice questions is associated with improved performance on the United States Medical Licensing Examination (USMLE) Step 1 examwhen compared to students who used the passive study strategy of watching educational videos. Methods The study used a correlational design. Participants were students from two cohorts in a United States medical school (N=164 and N=163) who completed their first two years and took the USMLEStep 1 exam. Data collected retrospectively included the number of practice questions completed, educational videos watched, Step 1 exam scores, average scores on in-class exams, and scores on the Medical College Admission Test (MCAT). Results The number of videos watched was negatively and significantly correlated with the Step 1 score for cohort 2022 (r= -0.294, α=0.01) and cohort 2023 (r= -0.175, α=0.05). The number of practice questions worked was positively and significantly correlated with the Step 1 score for cohort 2022 (r=0.176, α=0.05) and cohort 2023 (r=0.143 though not significant). The number of practice questions was a significant positive predictorof Step 1 score for cohort 2022 (β=0.141, p=0.017) and cohort 2023 (β=0.133, p=0.015). Videos were significant negative predictors for cohort 2023 (β= -0.118, p=0.034). Conclusions Answering practice questions appears to be a more effective study method than passively watching videos. Though other studies have supported the use of active learning methods, this study is unique in finding a negative correlation between test scores and the number of educational videos watched. Medical students should be urged to make the most effective use of study time by incorporating working practice questions and limiting watching educational videos.

Trends in successfully matched neurosurgery residency applicants.

The United States Medical Licensing Examination (USMLE) Step 1 recently transitioned to a pass/fail outcome, renewing interest in how programs select neurosurgical residents. This study investigates the association between match status and key academic metrics over time. Data are from the National Resident Matching Program from 2009 to 2022 for matched and unmatched US allopathic (MD) seniors. Investigated metrics included the mean number of contiguous ranks; mean number of distinct specialties ranked; mean USMLE Step 1 and Step 2 Clinical Knowledge (CK) scores; mean number of abstracts, presentations, and publications; mean number of research, work, and volunteer experiences; Alpha Omega Alpha status; attendance at a top 40 NIH-funded institution; PhD degree; and other degree. Multiple linear regression without an interaction term was used to evaluate how these have varied between the two groups during the study period and whether there is a difference between unmatched and matched MD seniors applying for a neurosurgical residency. Multiple linear regression with an interaction term was then used to test whether the difference in variables between the two groups changed over time. Regardless of match status, MD senior neurosurgical residency applicants exhibited an increase in USMLE Step 1 and 2 scores; average research experiences; abstracts, presentations, and publications; and work and volunteer experiences (p < 0.001). The percentage of applicants from a top 40 NIH-funded school decreased (p = 0.018), and the percentage who held an additional degree increased (p = 0.007). Between groups, there were significant differences in all categories except work experiences and other degree obtained. Over time, the difference between USMLE Step 2 scores between matched and unmatched seniors diminished (p = 0.027); in contrast, the difference in abstracts, presentations, and publications between the two groups increased over time (p < 0.001). From 2009 to 2022, neurosurgical residency applicants grew in their achievements across many metrics. In the advent of Step 1 becoming pass/fail, this study suggests that Step 2 is not viewed by programs as an adequate replacement. However, the Step 1 grading transition may serve as an opportunity for other factors to be considered that may better predict success in neurosurgical residency.

The Impact on Medical Student Stress in Relation to a Change in USMLE Step 1 Examination Score Reporting to Pass/Fail.

The United States Medical Licensing Examination (USMLE) Step 1 was designed to be a benchmark measure of knowledge and has been used heavily in the residency application process. Step 1 has moved from 3-digit scoring to a pass/fail scoring system, in part to decrease the stress associated with the exam. Emerging literature suggests that this transition has led to other stresses for students. Our study compared student stress levels, both overall and in relation to Step 1, leading up to the exam between a scored cohort and pass/fail cohort. We administered to each cohort a 14-item survey that included demographics, the PSS-4 stress scale, and 6 other potential stressors. Data was analyzed using two-tailed t test for independent means and analysis of variance. We found that while there was no difference in general overall stress between the students who took Step 1 for a score and students who took Step 1 pass/fail, we did see differences in stress related to the Step 1 exam. Step 1 stress was significantly lower for the pass/fail cohort than the score cohort during the second year of medical education leading up to the exam. However, this difference in Step 1 stress between the cohorts disappeared by the dedicated study period immediately before the exam. The change in scoring appears to have decreased stress specifically related to Step 1, but this reduction was not sustained as students entered their study period to prepare for Step 1.

Perceptions of the Emergency Medicine Resident Selection Process by Program Directors Following the Transition to a Pass/Fail USMLE Step 1.

Beginning January 26th, 2022, the National Board of Medical Examiners transitioned scoring of the United States Medical Licensing Examination (USMLE) Step 1 from a 3-digit score to pass/fail. In the past, the Step 1 score has been weighted heavily by program directors (PDs) as one of the most important metrics when assessing medical student's competitiveness. The objective of this study was to evaluate the perceptions of emergency medicine (EM) PDs on the transition to a pass/fail USMLE Step 1 exam, and to elicit the opinions of EM PDs on the USMLE examinations' ability to predict resident performance. A survey consisting of ranking and multiple-choice questions was sent to EM PDs. The multiple-choice questions were asked to determine EM PDs level of confidence in the ability of Step 1 and Step 2 Clinical Knowledge (CK) to predict a student's ability to succeed in residency. The ranking questions focused on assessing each program's current resident selection practices in comparison to expected selection criteria changes following a transition to pass/fail Step 1. R studio and MATLAB were used for statistical analysis, and a P value <0.05 was considered significant. The survey was completed by 57 (20.21%) EM PDs. When asked if Step 1 and Step 2 CK are accurate predictors of a resident's ability to perform clinically within EM, only 10.5% of PDs answered 'yes' to Step 1 being predictive, compared to 31.6% for Step 2 CK. Regarding selection criteria, the top quartile of attributes (standardized letters of evaluation [1st], away rotations [2nd], clerkship grades [3rd] and Step 2 CK score [4th]) remained the same following the transition. Our results indicate that the top quartile of attributes might remain the same, despite most PDs agreeing that Step 2 CK is a better predictor of a resident's performance.

The Impact of Holistic Review of Urology Residency Applications on Selection for Interview During the COVID-19 Pandemic

To investigate the impact of a holistic review of urology residency applications on interview selection at our institution during the COVID-19 pandemic. In the 2019-2020 cycle, applicants were filtered by a Step 1 score of 230 and whether they applied from selected east coast medical schools. For the 2020-2021 and 2021-2022 cycles, we implemented a scoring system which focused on desirable attributes based on our program training needs and resources. We compared applicant and interviewee demographics and United States Medical Licensing Examination (USMLE) scores using descriptive statistics and 1-way analysis of variance tests. A total of 282, 300, and 367 students applied to our residency program with 50, 45, and 52 selected for interviews during the 2019-2020, 2020-2021, and 2021-2022 cycles, respectively. Compared to 2019-2020, the 2020-2021 and 2021-2022 interviewee cohorts comprised of more non-tri-state applicants (36%, 55.6%, and 46.2%, respectively). Underrepresented minority representation increased for the 2020-2021 interviewee cohort; however, this was not observed in 2021-2022 (16%, 24.4%, 15.4%, respectively). Additionally, USMLE Step 1 and 2 scores were similar between interviewee cohorts in 2019-2020, 2020-2021 and 2021-2022, respectively (Step 1: 244.2 ± 8.8, 242 ± 12.1, 242.8 ± 12.4, P = .624) (Step 2: 249.1 ± 11.5, 251.5 ± 10.5, 254.4 ± 10.8, P = .143). Utilizing a comprehensive review resulted in a geographically diverse interview pool and no significant difference in academic performance among interviewees. Holistic review provides an alternative, balanced evaluation of residency applicants which may increase diversity in urology.

USMLE Step 1 Pass/Fail is Here: Are Plastic Surgery Applicants Really Better Off?

As of January 26, 2022, the United States Medical Licensing Examination(USMLE) Step 1 score reporting will be changed from a numeric scoring system to pass/fail. Although the new scoring policy is expected to benefit medical students, there is concern that it will also amplify preexisting disadvantages and worsen disparities for students applying in the already-competitive plastic surgery match. Whether the reporting change will tangibly benefit applicants to plastic surgery has yet to be elucidated. A cross-sectional survey was distributed to medical students and graduates via social media platforms. Data were analyzed using Student t test and Chi-squared statistic, with an alpha level set at 0.05. Data collection and analysis was performed at Johns Hopkins University School of Medicine. This study included a sample of 120 American Medical Trainees (AMTs) and International Medical Trainees (IMTs) who are interested in plastic surgery. The plurality of respondents were against the new Step 1 score reporting (AMT: 40.7%; IMT: 44.3%), but differences existed between AMTs when compared to IMTs, especially regarding opinions about the reporting change on stress levels and competitiveness of highly competitive specialties. Regardless of training status, respondents felt that the pass/fail scoring system would increase their likelihood to engage with more research, dual apply, prioritize studying for Step 2 CK, and consider a dedicated research year. While a pass/fail reporting system for Step 1 may alleviate some stress for medical trainees, other issues arise that may perpetuate disparities and bias against students with little resources in the field of plastic surgery. Residency programs should offer anticipatory guidance regarding prioritization of aspects of application to ease this psychosocial and financial pressure, as well as help students reorganize their constrained time.

Diagnostic & interventional radiology residency program directors' perspectives on the USMLE step 1 exam shift to pass/fail scoring

Rationale and objectivesAs of January 2022, Step 1 of the United States Medical Licensing Examination (USMLE) has changed to pass/fail grading. The purpose of this study was to share survey results and communicate changes Diagnostic (DR) and integrated Interventional (IR) Radiology residency program directors (PDs) will make and aspire to make, given this change. Materials and methodsAn online survey was sent to DR and IR PDs. Data was collected over four months. Custom R programming and MATLAB language scripts were used to evaluate the survey responses. Chi squared tests were used to determine statistical significance for multiple choice questions regarding PD views of Step 1 transitioning to pass-fail. Paired t-tests were used to differentiate pre- and post-values for questions in which PDs ranked criteria for resident selection. ResultsAfter USMLE Step 1 becomes pass/fail, most respondents will use Step 2 CK scores as a more important factor than previously, believe medical schools should share National Board of Medical Examiners (NBME) shelf exam scores, do not believe students will be better prepared clinically, and believe a student's medical school rank will be considered more. ConclusionThe added emphasis on Step 2 CK scores, NBME shelf exam scores, class rank, and a student's medical institution may negate the positive impacts of changing Step 1 to pass/fail. Alternatively, it may present an opportunity for programs to evaluate students more broadly.

A Survey of Vascular Surgery Program Directors: Perspectives following USMLE Step 1 Conversion to Pass/Fail and Virtual Only Interviews

Step 1 of the United States Medical Licensing Examination (USMLE), a historically important factor in the selection of vascular surgery residents and fellows, transitioned to completely pass/fail on January 26, 2022. Due to the COVID-19 pandemic, residency and fellowship interviews were conducted virtually during the 2020-21 and 2021-22 application cycles. Given these significant changes in the evaluation of candidates for residency training, we sought to understand vascular surgery program directors' (PDs) perspectives regarding the change in step 1 scoring and use of virtual interviews as well as determine which factors will assume importance when applying to vascular surgery training programs in the future. A 26-item survey questionnaire was created using Qualtrics survey tools with questions regarding attitudes toward the change in step 1 scoring and virtual interviews, the importance of additional factors utilized by programs in selecting candidates for interviews and ranking applicants for residency/fellowship selection, and programmatic demographics. This was distributed anonymously to all vascular surgery training programs over a 2-week period using a comprehensive list of 249 unique email addresses created by cross-referencing the Accreditation Council for Graduate Medical Education list of certified vascular training programs with email addresses from individual program websites identifying current program leadership. Responses were analyzed with descriptive statistics with values listed as average Likert scale weight±standard deviation (SD) or percentages. Sixty-eight of 249 (27.3%) program and associate PDs responded to the survey. Of which, 33.9% of respondents strongly disagreed with step 1 going completely pass/fail. In the absence of a scored USMLE step 1, letters of recommendation (average Likert scale weight±SD, 4.43±0.92), dedication to specialty (4.14±1.03), and USMLE step 2 CK (4.06±0.92) had the highest average scores for deciding which applicants to interview for integrated vascular surgery residency. For determining which candidates to interview for vascular surgery fellowship, letters of recommendation (4.51±0.84), dedication to specialty (4.12±0.90), and research (4.10±0.80) had the highest average scores. For ranking residency candidates, the interviewee's perceived "fit" (4.61±0.55), letters of recommendation (4.53±0.76), and an overall interview experience (4.47±0.62) had the highest average scores. Similarly, the factors with the highest average Likert scores for ranking fellowship candidates included the interviewee's perceived "fit" (4.69±0.51), letters of recommendation (4.65±0.52), and an overall interview experience (4.51±0.59). The majority (72.2%) of PDs preferred in-person interviews; however, 50% of respondents were at least "somewhat satisfied" with virtual interviews during the 2021-22 application cycle as they could judge applicants' interview skills at least "moderately well." The minority (18.8%) who preferred virtual interviews most commonly noted a "reduction of the financial burden for applicants" as the reason for this preference. Most vascular surgery program and associate PDs were dissatisfied with USMLE step 1 going pass/fail with most indicating prescreening applicants using both step 1 and step 2 clinical knowledge (CK) during the residency and fellowship selection processes. In the absence of a scored step 1, the top factors for interviewing and ranking integrated vascular surgery residency and fellowship candidates included letters of recommendation, dedication to specialty, research, USMLE step 2 CK, the interviewee's perceived "fit," and overall interview experience. Though most PDs preferred face-to-face interviews, they were overall at least "somewhat satisfied" with the virtual format that took place during the 2021-22 cycle.