Assessing the Impact of Changes to USMLE Step 1 Grading on Evaluation of Neurosurgery Residency Applicants in the United States: A Program Director Survey

Abstract

Neurosurgery (NS) is among the most selective specialties in the United States. As the United States Medical Licensing Examination (USMLE) Step 1 transitions to a binary pass/fail score, residency programs face unclear challenges in screening and evaluating applicants. The aim of this study is to provide insights into the perceived impact of changes to the USMLE Step 1 grading in the applicant selection process. We created a survey using questions regarding NS program demographics, the perceived predictive abilities of Step 1 and Step 2 clinical knowledge (CK), and several factors that programs consider when assessing applicants. We queried program directors (PDs), program coordinators (PCs), and assistant PDs at 117 NS residency programs. Respondents were asked to rank these factors in order of importance for selection at their respective training program. We used descriptive statistics and a Wilcoxon matched-pairs signed-rank test to evaluate the effects of these changes using STATA17. A total of 35 (30%) residency programs responded with 26 (74%) completing the factor ranking questions. 86% (95% confidence interval, 71.5%-94.3%) disagreed that the changes will better prepare students clinically. USMLE Step 2 CK scores, class rank, and away rotations saw significant increases in priority in the absence of a graded Step 1, whereas letters of recommendation and surrogates for research productivity saw notable, but not significant, changes after adjusting for multiple testing. Reporting binary Step 1 grades marks a significant shift in assessing applicants for NS residency by emphasizing Step 2 CK, class rank, and research productivity.