Development and validation of a procedure-specific assessment tool for hands-on surgical training in congenital heart surgery

Abstract

BackgroundHands-on surgical simulation has been sought to address training limitations within congenital heart surgery (CHS). However, there is a need for objective assessment methods to measure surgeons’ performance to justify its global adoption. This study aimed to validate a procedure-specific assessment tool for the simulation of the arterial switch operation on 3D-printed models and to evaluate the consistency of scoring among evaluators with different levels of experience in CHS. MethodsFive “expert” and 5 “junior” surgeons performed the arterial switch procedure on 3D-printed models with transposition of the great arteries during 2 hands-on surgical training courses. Their performance was retrospectively assessed by 9 evaluators with varying experience in CHS (staff surgeons, resident surgeons, and non-MD raters). Assessments were done using 2 assessment tools: the Hands-On Surgical Training–Congenital Heart Surgery (HOST-CHS) assessment tool and the global rating scale (GRS). ResultsThe HOST-CHS tool showed a higher interrater and intrarater reliability compared with the GRS. Total scores for expert surgeons were highly consistent across all evaluators. Non-MD raters’ total scores for junior surgeons were slightly higher than those of residents and staff evaluators. All grades of evaluator were able to discriminate between junior and expert surgeons. ConclusionsThis study demonstrates the development and validation of an objective, procedure-specific assessment tool for the arterial switch operation with consistency among evaluators with different experience. There is now a platform for quantifying and accurately evaluating performance, which will be highly beneficial in training and developing the next generation of congenital heart surgeons.