Measuring Competence in Surgical Training through Assessment of Surgical Entrustable Professional Activities

Abstract

To assess competency in surgical residents with bundled assessments using a surgical entrustable professional activity (SEPA) framework. A pilot study was conducted using a combination of validated assessment tools (multiple choice exam (MCE) questions from the Surgical Council on Resident Education [SCORE], the Clinical Assessment and Management Examination - Outpatient (CAMEO) form, the Virtual Surgical Patient (VSP) website, and a procedure-specific Operative Performance Rating System [OPRS]) to determine competency in surgical residents in the treatment of breast cancer and gallbladder disease, respectively. A large academic institution with a surgical training program in the Mid West of the United States. A total of 10 categorical surgical residents were invited to participate. Five completed the breast surgical EPA (SEPA) and 5 different residents completed the gallbladder SEPA. In terms of performance on the assessments, for the breast SEPA, scores did not appear to be related to PGY level, and residents' performance in general was the least strong on the MCE and the VSP case. The gallbladder SEPA showed a more expected pattern, distinguishing between junior and senior residents. As expected, all junior residents were required to remediate the OPRS assessment, while the senior residents passed. For the OPRS, senior level residents consistently were rated as "excellent" in terms of operative flow (5/5), while junior residents were all given a score of "good" (3/5). Assessing competence among surgical residents has been a discussion for several years. Varying methods of assessing competence have been proposed, but surgical competence is presently defined in a very general way through both the ACGME and American Board of Surgery (ABS). Using a SEPA format, as proposed, we could ensure specific understanding of each graduating resident's ability. These results show that the SEPA may be a valid tool for defining and capturing multiple areas of competence that are associated with different disease processes.