Correlation of Objective Assessment Data With General Surgery Resident In-Training Evaluation Reports and Operative Volumes

Abstract

Faculty evaluations, ABSITE scores, and operative case volumes often tell little about true resident performance. We developed an objective structured clinical examination called the Surgical X-Games (5 rooms, 15 minutes each, 12-15 tests total, different for each postgraduate [PGY] level). We hypothesized that performance in X-Games will prove more useful in identifying areas of strength or weakness among general surgery (GS) residents than faculty evaluations, ABSITE scores, or operative cases volumes. PGY 2 to 5 GS residents (n = 35) were tested in a semiannual X-Games assessment using multiple simulation tasks: laparoscopic skills, bowel anastomosis, CT/CXR analysis, chest tube placement, etc. over 1 academic year. Resident scores were compared to their ABSITE, in-training evaluation reports, and operating room case numbers. Academic medical center. PGY-2, 3, 4, and 5 GS residents at Mayo Clinic in Rochester, MN. Results varied greatly within each class except for staff evaluations: in-training evaluation reports medians for PGY-2s were 5.3 (range: 5.0-6.0), PGY-3s 5.9 (5.5-6.3), PGY-4s 5.6 (5.0-6.0), and PGY-5s were 6.1 (5.6-6.9). Although ABSITE and operating room case volumes fluctated greatly with each PGY class, only X-Games scores (median: PGY-2 = 82, PGY-3 = 61, PGY-4 = 76, and PGY-5 = 60) correlated positively (p < 0.05) with operative case volume and negatively (p < 0.05) with staff evaluations. X-Games assessment generated wide differentiation of resident performance quickly, inexpensively, and objectively. Although "Minnesota-nice" surgical staff may feel all GS trainees are "above average," objective assessment tells us otherwise.