What errors make a laparoscopic cancer surgery unsafe? An ad hoc analysis of competency assessment in the National Training Programme for laparoscopic colorectal surgery in England.

Abstract

The National Training Programme for laparoscopic colorectal surgery in England was implemented to ensure training was supervised, structured, safe and effective. Delegates were required to pass a competency assessment (sign-off) before undertaking independent practice. This study described the types of errors identified and associated these errors with competency to progress to independent laparoscopic colorectal practice. All sign-off submissions from the start of the process in January 2008 until July 2013 were included. Content analysis was used to categorise errors. Bayes factor (BF) was used to measure the impact of individual error on assessment outcome. A smaller BF indicates that an error has stronger associations with unsuccessful assessments. Bayesian network was employed to graphically represent the reasoning process whereby the chance of successful assessment diminished with the identification of each error. Quality of the error feedback was measured by the area under the ROC curve which linked the predictions from the Bayesian model to the expert verdict. Among 370 assessments analysed, 240 passed and 130 failed. On average, 2.5 different types of error were identified in each assessment. Cases that were more likely to fail had three or more different types of error (χ(2) = 72, p < 0.0001) and demonstrated poorer technical skills (CAT score <2.7, χ(2) = 164, p < 0.0001). Case complexity or right- versus left-sided resection did not have a significant impact. Errors associated with dissection (BF = 0.18), anastomosis (BF = 0.23) and oncological quality (BF = 0.19) were critical determinants of surgical competence, each reducing the odds of pass by at least fourfold. The area under the ROC curve was 0.84. Errors associated with dissection, anastomosis and oncological quality were critical determinants of surgical competency. The detailed error analysis reported in this study can guide the design of future surgical education and clinical training programmes.