USMLE Step 3 Scores Have Value in Predicting ABR Core Examination Outcome and Performance: A Multi-institutional Study

Abstract

We analyzed multi-institutional data to determine if Step 3 performance tiers can identify radiology residents with increased risk of Core examination failure and submean performance. We collected Step 3 scores (USMLE Step 3 or COMLEX Level 3) and American Board of Radiology (ABR) Core examination outcomes and scores for anonymized residents from 13 different Diagnostic Radiology residency programs taking the ABR Core examination between 2013 and 2019. Step 3 scores were converted to percentiles based on Z-score, with Core outcome and performance analyzed for Step 3 groups based on 50th percentile and based on quintiles. Core outcome was scored as fail when conditionally passed or failed. Core performance was measured by the percent of residents with scores below the mean. Differences between Step 3 groups for Core outcome and Core performance were statistically evaluated. Data were available for 342 residents. The Core examination failure rate for 121 residents with Step 3 scores <50th percentile was 19.8% (fail relative risk = 2.26), significantly higher than the 2.7% failure rate for the 221 other residents. Of 42 residents with Step 3 scores in the lowest quintile, the Core failure rate increased to 31.0% (fail relative risk = 3.52). Core performance improved with higher Step 3 quintiles. Step 3 licensing scores have value in predicting radiology resident performance on the ABR Core examination, enabling residency programs to target higher risk residents for early assessment and intervention.