Performance variability during training on simulators is associated with skill transfer

Abstract

BackgroundExpert performance is characterized by consistency. The degree of variability of performance from repetition to repetition during proficiency-based simulator training could potentially indicate acquisition of expertise. We hypothesized that learners with less variability in performance during simulator training would achieve greater performance at the end of training and improved transfer of skills to a live, anesthetized, porcine model. MethodsThe performance of 93 subjects (surgery residents and medical students) who had participated in 3 randomized controlled trials was analyzed for variability. All participants had trained in laparoscopic suturing on the Fundamentals of Laparoscopic Surgery (FLS) simulator. Their performance had been assessed on the simulator before (baseline) and after training (posttest) and on a live, anesthetized, porcine model (transfer test). We computed the coefficient of variations of suturing scores during training for each participant. Linear regression was used to assess whether variability in performance during training predicted posttest and transfer-test scores. ResultsDecreased practice variability in performance was associated with greater scores in posttests and transfer tests. For each percent decrease in variability performance, posttest scores increased by 3.8 points (P < .001) and transfer-test scores increased by 3.0 points (P < .001). Greater mean scores during practice were associated with greater scores on the transfer test (P < .001). ConclusionDecreased variability in performance during practice on simulators is associated with improved performance at the end of training and during transfer to a live, anesthetized, porcine model. These findings suggest that variability in performance during simulator training may be used to track the progress and readiness of a trainee for the clinical environment. Further studies are needed to verify the robustness of this potentially new metric of performance.