MP25-05 PROGRAM DIRECTORS' PERSPECTIVES ON THE ASSESSMENT OF RESIDENCY APPLICANTS IN THE POST-USMLE STEP 1 ERA: A STRONG CASE FOR STANDARDIZED LETTERS OF RECOMMENDATION?

Abstract

You have accessJournal of UrologyCME1 Apr 2023MP25-05 PROGRAM DIRECTORS' PERSPECTIVES ON THE ASSESSMENT OF RESIDENCY APPLICANTS IN THE POST-USMLE STEP 1 ERA: A STRONG CASE FOR STANDARDIZED LETTERS OF RECOMMENDATION? Angeline Johny, Patrick J. Shenot, Charles Green, Leah Chisholm, Steve Riggs, Stephen V. Jackman, Aqsa A. Khan, Peter N. Kolettis, Brian K. McNeil, and Wesley A. Mayer Angeline JohnyAngeline Johny More articles by this author , Patrick J. ShenotPatrick J. Shenot More articles by this author , Charles GreenCharles Green More articles by this author , Leah ChisholmLeah Chisholm More articles by this author , Steve RiggsSteve Riggs More articles by this author , Stephen V. JackmanStephen V. Jackman More articles by this author , Aqsa A. KhanAqsa A. Khan More articles by this author , Peter N. KolettisPeter N. Kolettis More articles by this author , Brian K. McNeilBrian K. McNeil More articles by this author , and Wesley A. MayerWesley A. Mayer More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003253.05AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: The USMLE Step 1 exam evolved into a key metric utilized by program directors (PDs) in assessing candidates for residency. The transition to a USMLE Step 1 binary pass/fail scoring system has resulted in a loss of an important objective assessment. With national movements towards pass/fail systems for clerkship grading and trends towards abandonment of class ranking, assessing residency applications is becoming increasingly challenging. METHODS: The Society for Academic Urologists (SAU) convened a task force to, in part, assess the perspectives of urology PDs regarding the importance of various aspects of a residency application for predicting clinical performance. An anonymous survey was disseminated to all urology program directors in the United States. Perspectives on 11 potential application predictors of clinical performance and demographics were recorded. Descriptive statistics characterized PD responses. Friedman’s Test and pairwise Wilcoxon tests were used to evaluate the relative ranks assigned to application elements by PD’s. RESULTS: There was a 60.5% response rate (89/147). Letters of recommendation (LOR) was ranked as the most important predictor, with a mean of 2.39, median of 2 (IQR range 1- 3). Clerkship grades and USMLE Step 1 were comparable and ranked second with mean scores of 4.3, and 4.62 respectively. Medical school reputation ranked the lowest with a mean score of 8.47. There was significant heterogeneity among categories, however this was less so for LORs which predominated as the most important factor among application elements (p<0.001). CONCLUSIONS: To our knowledge, this is the largest sample size assessing program director perspectives on application factors that predict clinical performance. The second (clerkship grades) and third (USLME Step 1) most important factors moving to binary pass/fail systems creates an opportunity for actionable change to improve assessment objectivity. Our data demonstrate LOR to be the most important factor of residency applications. Thus, there may be a strong argument for moving towards a standardized LOR to maximize this tool, mitigate bias, and improve inter-reviewer reliability. Source of Funding: N/A © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e344 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Angeline Johny More articles by this author Patrick J. Shenot More articles by this author Charles Green More articles by this author Leah Chisholm More articles by this author Steve Riggs More articles by this author Stephen V. Jackman More articles by this author Aqsa A. Khan More articles by this author Peter N. Kolettis More articles by this author Brian K. McNeil More articles by this author Wesley A. Mayer More articles by this author Expand All Advertisement PDF downloadLoading ...