Impact of the Step 1 Scoring Change on the Upcoming Application Cycle.

Abstract

The recent transition of United States Medical Licensing Examination Step 1 to a pass–fail system was intended to promote student wellness. However, some have raised concerns for increased emphasis on subjective measures of performance.1 Among plastic surgery residency program directors (PDs), perceptions of the change have largely been negative.2 While long-term effects have been considered, in the immediate future, a mixture of students with numeric and passing scores will create a challenge for programs comparing applicants. In total, 40% of PDs (n = 36) completed a survey about anticipated short-term effects of the scoring change. Respondants compared the use of Step 1 in applicant evaluation in the upcoming application cycle (2022–2023) with the most recent cycle (2021–2022). (See Table, Supplemental Digital Content 1, which displays the survey questions, https://links.lww.com/PRSGO/C107.) As anticipated, numeric Step 1 scores will continue to factor into evaluations in the short term, but Step 2 scores will be of increased importance. In the next cycle, more programs will give Step 2 greater or equal weight than Step 1 (19% and 53%, respectively, up from 11% and 44%) and fewer will give Step 1 greater weight than Step 2 (28% down from 44%). Notably, a minority of PDs (11%) stated that they will no longer consider numeric Step 1 scores (Table 1). Table 1. - Relative Importance of Step 1 and Step 2 in Upcoming Application Cycles Current Application Cycle (n = 36) Upcoming Application Cycles (n = 32*) Relative weight of Step 1 and Step 2 for determining interview invitations Step 1 > Step 2 16 (44%) 9 (28%) Step 2 > Step 1 4 (11%) 6 (19%) Equally weighted 16 (44%) 17 (53%) Cutoff score for Step 1 No cutoff score 18 (50%) 17 (53%) 221–230 3 (8%) 2 (6%) 231–240 6 (17%) 4 (13%) 241–250 9 (25%) 7 (22%) >250 0 (0%) 2 (6%) Cutoff score for Step 2† No cutoff score 28 (78%) 14 (45%) 221–230 1 (3%) 7 (23%) 231–240 2 (6%) 7 (23%) 241–250 4 (11%) 2 (6%) >250 1 (3%) 1 (3%) *Four PDs responded that they will no longer consider numeric Step 1 scores in evaluating applicants.†One PD did not give a cutoff score range for Step 2 in upcoming application cycles (n = 31). In the next application cycle, twice as many of the programs surveyed will institute Step 2 cutoff scores. In the most recent cycle, 22% of PDs endorsed using a Step 2 cutoff score, while 55% anticipate having a cutoff in the next cycle. The majority of anticipated cutoff scores for Step 2 were between 221–230 (23%) and 231–240 (23%) (Table 1). Historically, mean Step 2 scores have been higher than mean Step 1 scores (eg, 256 versus 249 for matched plastic surgery applicants in 2020).3 Identical Step 2 cutoffs may in practice be more inclusive than prior Step 1 cutoffs. Similar to previous reports, letters of recommendation and previous knowledge of the applicant were ranked as the most important factors for applicant evaluation. (See Table, Supplemental Digital Content 2, which displays rankings of factors, https://links.lww.com/PRSGO/C108.)4,5 Half of PDs (53%) stated that research would carry greater weight in the next cycle (Table 2). When presented with hypothetical scenarios, a strong Step 1 score remained beneficial to applicants, whereas a low Step 1 score may not be detrimental in many cases. While 69% of PDs answered that an applicant with a high Step 1 score would be at an advantage over an applicant with a pass, only 47% of PDs stated that an applicant with a low Step 1 score would be at a disadvantage compared to a pass (Table 2). Table 2. - Hypothetical Scenarios for Applicant Evaluation and Relative Importance of Research Question PD Response (%) Do you anticipate that research experiences like research fellowships will carry more significant weight than in the past? 19 (53) With comparable Step 2 scores, will an applicant with a low Step 1 score be at a disadvantage relative to an applicant with a pass for a Step 1 score? 17 (47) With comparable Step 2 scores, will an applicant with a high Step 1 score be at an advantage relative to an applicant with a pass for a Step 1 score? 25 (69) This study suggested a gradual shift in emphasis from Step 1 to Step 2, although some programs may immediately stop considering Step 1. Importantly, over half of PDs would not view a low Step 1 score negatively. For these programs, a low Step 1 score may be considered similar to a pass. Those who performed strongly on Step 1 will benefit in most cases from having taken the examination before the switch. Disclosure The authors have no financial interest to declare in relation to the content of this article.