Fact or Fiction? Practice on a Simulator is Not Required to Pass the Fundamentals of Endoscopic Surgery (FES)TM Skills Exam

Abstract

Initial work on the validity evidence used to support the Fundamentals of Endoscopic Surgery (FES) performance exam as a measure of technical competency showed a strong relationship to clinical experience. Despite this evidence, there is a perception among some program directors that the exam cannot be successfully passed without practice on a simulator. We assess the validity of this perception. Deidentified data from the initial FES skills examination (prior to the 2014 FEC requirement) was reviewed, and 335 unique participants with reported simulation experience demographics were identified. Self reported data analyzed included gender, total clinical endoscopy procedure experience (1-150, 151-300, >300), and endoscopy simulator training hours (0, 1-5, 6-10, 11-20, >20). Final FES skills exam scores, and pass/fail designations for each participant were reported by the FES program staff. Continuous variables were compared between groups using one-way analysis of variance with post-hoc analysis where appropriate. Categorical variables were compared using Pearson Chi-Squared. The effect of variables on pass rate was assessed using univariate and multivariate logistic regression. Simulation training experience (SE) was categorically reported in hours(n,%): 0 (98, 29%), 1-5 (135, 40%), 6-10 (52, 16%), 11-20 (24, 7%), and >20 (26, 8%). Clinical endoscopy experience (CE), reported categorically as total cases performed (n,%), was available for 323 of 355 identified participants: 1-150 (126, 39%), 151-300 (99, 31%), >300 (98, 30%). There was no statistically discernible differences in mean FES total or task scores across the SE groups (total score 0:72 ± 15, 1-5:72 ± 13, 6-10:71 ± 14, 11-20:71 ± 16, 20:78 ± 13; p = 0.28), while both total score and task scores were discernibly higher in the more experienced CE groups (>151) compared to the least experienced group (total score; <150:67 ± 15, 151-300:75 ± 1, >300:77 ± 14; p < 0.01). Similarly, there was no statistically discernible difference in FES skills exam pass rates between SE groups (0: 80%, 1-5: 82%, 6-10: 79%, 11-20: 75%, >20: 85%; x2 = 2.5, p = 0.6), but there was a strong relationship between clinical experience and pass rate (<150: 70%, 151-300: 87%, >300: 89%; x2 = 15.8, p < 0.001). Finally, on both univariate and multivariate logistic regression, CE remained a discernible predictor of passing, even when controlling for SE (odds ratio = 2, 95% confidence interval 1.4-2.9, p < 0.001). FES skills examination data collected on participants completing the examination before the FEC requirement shows no demonstrable relationship with self-reported training experience on a simulator but confirms a strong relationship with clinical endoscopy experience. This lends further evidence to the validity of the FES exam as a marker of clinical endoscopic skill.