Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience

Abstract

Purpose The aim of the study is to demonstrate whether resident performance on the United States Medical Licensing Exam (USMLE) Step 1 and the Ophthalmology Knowledge Assessment Program (OKAP) correlates to American Board of Ophthalmology Written Qualifying Examination (ABO-WQE). Design This is a retrospective cohort study. Participants Forty-one residents who completed their postgraduate training between 1999 and 2011 from our institution were included in the study. Methods Percentiles on the USMLE Step 1, OKAP, and ABO-WQE first attempt were recorded and analyzed for possible correlations among scores across examinations using Pearson rank correlation, chi-square analysis, Fisher exact test, and Student t-test. Results The percentile score on the USMLE Step 1 was the only standardized test that had a statistically significant association with the percentile score on the ABO-WQE (Pearson rank coefficient = 0.45, p = 0.01). Not achieving a score of 20th percentile on the OKAP examination among second-year residents had the highest statistical association with failing the ABO-WQE on first attempt (p = 0.001) with an odds ratio of 20.4, and the relative risk of failing the ABO-WQE was 11.3; however, the relative risk of passing if the cutoff was achieved was only 1.8, and a positive predictive value for failing the ABO-WQE on first attempt of 0.47. Conclusions Percentile scores on the OKAP are not indicative of percentile scores on the ABO-WQE. While OKAP examinations in the second and third year appear to separate out residents at risk of failing the ABO-WQE, there is a large disparity between the risk of passing and failing which is likely due to the inadequate prognostic value of the test. It is likely OKAP examinations may be a useful tool to identify residents who are at low risk for failing the ABO-WQE on first attempt, but much less useful for identifying residents who are at high risk for failing the ABO-WQE.