An Online Spaced-Education Game to Teach and Assess Residents: A Multi-Institutional Prospective Trial

Abstract

While games are frequently used in resident education, there is little evidence supporting their efficacy. We investigated whether a spaced-education (SE) game can be both a reliable and valid method of assessing residents' knowledge and an effective means of teaching core content. The SE game consisted of 100 validated multiple-choice questions and explanations on core urology content. Residents were sent 2 questions each day via email. Adaptive game mechanics re-sent the questions in 2 or 6 weeks if answered incorrectly and correctly, respectively. Questions expired if not answered on time (appointment dynamic). Residents retired questions by answering each correctly twice in a row (progression dynamic). Competition was fostered by posting relative performance among residents. Main outcomes measures were baseline scores (percentage of questions answered correctly on initial presentation) and completion scores (percentage of questions retired). Nine hundred thirty-one US and Canadian residents enrolled in the 45-week trial. Cronbach alpha reliability for the SE baseline scores was 0.87. Baseline scores (median 62%, interquartile range [IQR] 17%) correlated with scores on the 2008 American Urological Association in-service examination (ISE08), 2009 American Board of Urology qualifying examination (QE09), and ISE09 (r = 0.76, 0.46, and 0.64, respectively; all p < 0.001). Baseline scores varied by sex, country, medical degree, and year of training (all p ≤ 0.001). Completion scores (median 100%, IQR 2%) correlated with ISE08 and ISE09 scores (r = 0.35, p < 0.001 for both). Seventy-two percent of enrollees (667 of 931) requested to participate in future SE games. An SE game is a reliable and valid means to assess residents' knowledge and is a well-accepted method by which residents can master core content.