323 Can In-Training Exam Review Quizzes Administered via Audience Response System Predict Scores on In-Training Exam?

Abstract

Listen

Translate article

In preparation for the American Board of Emergency Medicine (ABEM) written board examination (Qualifying Examination) residents take an annual in-training exam. All emergency medicine (EM) residencies strive for a 100% pass rate of this examination. However, each residency has its own training strengths and weaknesses (eg, strong educational sessions in toxicology or high clinical exposure to trauma). To improve in-training exam scores, many residency programs provide review sessions. These serve to improve performance. One method of review is board-style questions with an audience response system which has been shown to be an effective tool for knowledge acquisition and assessment in other settings. We investigate if scores on review quizzes delivered by an audience response system correlate with in-training exam scores. Prospective observational study of EM residents at 6 ACGME-accredited EM residency programs. Subjects included residents who had taken previous in-training examinations. Subjects participated in bimonthly review sessions using Rosh Review questions and an audience response system. Twelve review quizzes were administered each consisting of 10 multiple choice questions covering a major topic of the EM in-training exam: cardiology, gastroenterology, neurology, toxicology, signs and symptoms, procedures, trauma, respiratory, obstetrics and gynecology, infectious disease, head-ears-eyes-nose-throat, and environmental disorders. Total mean scores for the review quizzes were calculated for each participant. Scores from incomplete quizzes, defined as less than 7 out of 10 questions answered, were excluded from analysis. A mixed linear model was used to analyze the data accounting for prior 2012 in-training exam scores and nesting due to institution. A total of 135 residents participated and their data analyzed. Results from the mixed linear model indicate that the total mean score on the review quizzes was a significant [t(127) = 6.68; P<.00010] predictor of the 2013 in-training exam score after controlling for the 2012 in-training exam score. Our results suggest that scores from review quizzes delivered by an audience response system correlate with in-training exam scores. This information may be useful in identifying poor performers early so that targeted educational interventions can be made.