Reflection Versus Reality: Accuracy of Surgeon Self-Reflection on Hysterectomy Quality Metrics

Abstract

Study Objective Describe gynaecologic surgeons’ self-reflection accuracy for three hysterectomy quality metrics and determine whether accuracy is associated with specific surgeon or practice characteristics. Design Retrospective, cross-sectional analysis. Setting Six Ontario, Canada hospitals (3 academic, 3 community). Patients or Participants Sixty-nine gynecologic surgeons reviewing a first performance report card. Interventions Surgeons estimated their preceding six months of hysterectomy case volume, technicity (minimally-invasive rate), and complication rate prior to reviewing a personalized report card. Measurements and Main Results Agreement between estimated and actual performance was evaluated using Pearson correlation. Differences (∆) between estimated and actual performance were used to proxy “accuracy”. Means of these differences (∆mean) were compared to zero (“perfect accuracy”) using Wilcoxon signed-rank or T-tests. Surgeons were categorized by tertiles of ∆s into over-, accurate or under-estimators. Association between accuracy and surgeon gender, subspecialty training, practice duration and location were assessed using analysis-of-variance. Sixty-nine surgeons (42 generalists, 27 fellowship-trained) accessed report cards between 2016-2018. Correlation between estimated and actual performance was strong for case volume (r=0.73, p<0.001), moderate for technicity (r=0.57, p<0.001), and poor for complication rate (r=0.29, p=0.019). Surgeons systematically underestimated complication rate (∆mean: -4.7%, 95% CI -7.5% to -1.5%, p=0.005) but accurately estimated case volume (∆mean: +1.0%, 95% CI -0.5 to 2.5 cases, p=0.260) and technicity (∆mean: +0.6%, 95% CI -3.6% to 10.5%, p=0.935). Surgeons who overestimated complication rate had more years in practice (23 years) than those who accurately estimated (16 years, p=0.027) or underestimated (16 years, p=0.016) complications. Surgeons who underestimated technicity had more years in practice (22 years) than those who accurately estimated technicity (14 years, p=0.015). Accurate self-reflection was not associated with gender, fellowship training, or practice location. Conclusion Surgeons reliably reflect on case volume and technicity. However, there is a disconnect in surgeons’ reflections on complication rate compared to actual performance, underscoring the need for regular feedback through initiatives such as performance report cards. Describe gynaecologic surgeons’ self-reflection accuracy for three hysterectomy quality metrics and determine whether accuracy is associated with specific surgeon or practice characteristics. Retrospective, cross-sectional analysis. Six Ontario, Canada hospitals (3 academic, 3 community). Sixty-nine gynecologic surgeons reviewing a first performance report card. Surgeons estimated their preceding six months of hysterectomy case volume, technicity (minimally-invasive rate), and complication rate prior to reviewing a personalized report card. Agreement between estimated and actual performance was evaluated using Pearson correlation. Differences (∆) between estimated and actual performance were used to proxy “accuracy”. Means of these differences (∆mean) were compared to zero (“perfect accuracy”) using Wilcoxon signed-rank or T-tests. Surgeons were categorized by tertiles of ∆s into over-, accurate or under-estimators. Association between accuracy and surgeon gender, subspecialty training, practice duration and location were assessed using analysis-of-variance. Sixty-nine surgeons (42 generalists, 27 fellowship-trained) accessed report cards between 2016-2018. Correlation between estimated and actual performance was strong for case volume (r=0.73, p<0.001), moderate for technicity (r=0.57, p<0.001), and poor for complication rate (r=0.29, p=0.019). Surgeons systematically underestimated complication rate (∆mean: -4.7%, 95% CI -7.5% to -1.5%, p=0.005) but accurately estimated case volume (∆mean: +1.0%, 95% CI -0.5 to 2.5 cases, p=0.260) and technicity (∆mean: +0.6%, 95% CI -3.6% to 10.5%, p=0.935). Surgeons who overestimated complication rate had more years in practice (23 years) than those who accurately estimated (16 years, p=0.027) or underestimated (16 years, p=0.016) complications. Surgeons who underestimated technicity had more years in practice (22 years) than those who accurately estimated technicity (14 years, p=0.015). Accurate self-reflection was not associated with gender, fellowship training, or practice location. Surgeons reliably reflect on case volume and technicity. However, there is a disconnect in surgeons’ reflections on complication rate compared to actual performance, underscoring the need for regular feedback through initiatives such as performance report cards.