Standardized Video Interviews Do Not Correlate to United States Medical Licensing Examination Step 1 and Step 2 Scores.

Daniel Egan,Arlene Chung,William Krauss,Lukasz Cygan,Jonathan Bronner,Abbas Husain,Matthew Silver,Kaushal Shah,Jeff Van Dermark,Ida Li,Jan Shoenberger,Michael Bond,Todd Taylor,Brahim Ardolic,William Caputo,Moshe Weizberg,Melissa White

doi:10.5811/westjem.2018.11.39730

Abstract

IntroductionIn 2017, the Standardized Video Interview (SVI) was required for applicants to emergency medicine (EM). The SVI contains six questions highlighting professionalism and interpersonal communication skills. The responses were scored (6–30). As it is a new metric, no information is available on correlation between SVI scores and other application data. This study was to determine if a correlation exists between applicants’ United States Medical Licensing Examination (USMLE) and SVI scores. We hypothesized that numeric USMLE Step 1 and Step 2 Clinical Knowledge (CK) scores would not correlate with the SVI score, but that performance on the Step 2 Clinical Skills (CS) portion may correlate with the SVI since both test communication skills.MethodsNine EM residency sites participated in the study with data exported from an Electronic Residency Application Service (ERAS®) report. All applicants with both SVI and USMLE scores were included. We studied the correlation between SVI scores and USMLE scores. Predetermined subgroup analysis was performed based on applicants’ USMLE Step 1 and Step 2 CK scores as follows: (≥ 200, 201–220, 221–240, 241–260, >260). We used linear regression, the Kruskal-Wallis test and Mann-Whitney U test for statistical analyses.Results1,325 applicants had both Step 1 and SVI scores available, with no correlation between the overall scores (p=0.58) and no correlation between the scores across all Step 1 score ranges, (p=0.29). Both Step 2 CK and SVI scores were available for 1,275 applicants, with no correlation between the overall scores (p=0.56) and no correlation across all ranges, (p=0.10). The USMLE Step 2 CS and SVI scores were available for 1,000 applicants. Four applicants failed the CS test without any correlation to the SVI score (p=0.08).ConclusionWe found no correlation between the scores on any portion of the USMLE and the SVI; therefore, the SVI provides new information to application screeners.

Highlights

In 2017, the Standardized Video Interview (SVI) was required for applicants to emergency medicine (EM)
We hypothesized that numeric United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) scores would not correlate with the SVI score, but that performance on the Step 2 Clinical Skills (CS) portion may correlate with the SVI since both test communication skills
We found no correlation between the scores on any portion of the USMLE and the SVI; the SVI provides new information to application screeners. [West J Emerg Med. 2019;20(1)87-91.]

Summary

Introduction

In 2017, the Standardized Video Interview (SVI) was required for applicants to emergency medicine (EM). As it is a new metric, no information is available on correlation between SVI scores and other application data. Emergency medicine (EM) has attempted to standardize recommendation letters and clerkship-grading transparency through the Standardized Letter of Evaluation (SLOE).[1] Without standardization, letters of recommendation showed grade inflation, lack of meaningful comparison between applicants, and the inability to use them as discriminatory tools for success in residency.[2,3,4] Inconsistencies in grades and evaluations by gender have been demonstrated in other specialties.[5] Even with standardized letters in several specialties, the use of the full spectrum of global assessments has not been found consistently nor has the accurate prediction of an applicant’s position on the rank list.[1,6,7]

Methods

Results

Discussion

Conclusion