Assessing intraoperative judgment using script concordance testing through the gynecology continuum of practice

Abstract

Objective: To measure surgical judgment across the Obstetrics and Gynecology (OBGYN) continuum of practice and identify factors that correlate with improved surgical judgment.Methods: A 45-item written examination was developed using script concordance theory, which compares an examinee’s responses to a series of “ill-defined” surgical scenarios to a reference panel of experts. The examination was administered to OBGYN residents, Female Pelvic Medicine and Reconstructive Surgery (FPMRS) fellows, practicing OBGYN physicians and FPMRS experts. Surgical judgment was evaluated by comparing scores against the experts. Factors related to surgical experience were measured for association with scores.Results: In total, 147 participants including 11 residents, 37 fellows, 88 practicing physicians and 11 experts completed the 45-item examination. Mean scores for practicing physicians (65.2 ± 7.4) were similar to residents (67.2 ± 7.1), and worse than fellows (72.6 ± 4.2, p < 0.001) and experts (80 ± 5, p < 0.001). Positive correlations between scores and surgical experience included: annual number of vaginal hysterectomies (r = 0.32, p = <0.001), robotic hysterectomies (r = 0.17, p = 0.048), stress incontinence (r = 0.29, p < 0.001) and prolapse procedures (r = 0.37, p < 0.001). Inverse correlation was seen between test scores and years in practice. (r = −0.19, p = 0.02).Conclusion: Intraoperative judgment in practicing OBGYN physicians appears similar to resident physicians. Practicing physicians who perform FPMRS procedures perform poorly on this examination of surgical judgment; lower performance correlates with less surgical experience and the greater amount of time in practice.