Diagnostic performance 1 h after simulation training predicts learning

Abstract

Although simulation training improves post-training performance, it is unclear how well performance soon after simulation training predicts longer term outcomes (i.e., learning). Here our objective was to assess the predictive value of performance 1 h post-training of performance 6 weeks later. We trained 84 first year medical students a simulated case of chest pain due to aortic stenosis. They then received training on a case of acute onset dyspnea due to pulmonary embolism, after which we evaluated diagnostic performance on their trained murmur followed by novel murmur. We repeated the evaluation of diagnostic performance on the same murmurs 6 weeks later. One hour post-training 88.1 % of students identified the training murmur, compared to 60.7 % for the novel murmur. Six weeks after training the corresponding results were 89.3 and 65.5 %, respectively (p < 0.0001 for both time periods). The probability of students diagnosing their training murmur 6 weeks post-training if they diagnosed this after 1 h (positive predictive value) was 0.89 [0.87, 0.93], and the probability of misdiagnosing their trained murmur 6 weeks post-training if they misdiagnosed this after 1 h (negative predictive value) was 0.10 [0.01, 0.40]. The corresponding positive and negative predictive values for the novel murmur were 0.69 [0.55, 0.80] and 0.39 [0.24, 0.57], respectively. Students who successfully diagnosed a cardiac murmur 1 h after simulation training were very likely to recognize the same murmur 6 weeks later, suggesting that we can use performance 1 h post-training as a learning outcome.