281. COMPETENCY ASSESSMENT TOOL FOR MINIMALLY INVASIVE ESOPHAGECTOMY: DEVELOPMENT, CONSTRUCT VALIDITY, RELIABILITY AND VERSION 2

Abstract

Abstract Minimally invasive esophagectomy (MIE) is a complex and technically demanding procedure and implementation is associated with substantial learning curves. In other complex minimally invasive procedures suboptimal surgical performance, assessed by video-based peer reviewing, has convincingly been related to less favorable patient outcomes. A validated procedure-specific competency assessment tool (CAT) for MIE could provide structured feedback to evaluate surgical performance and be used for training, safe implementation, learning curve assessment, research and to improve patient outcome. Based on key steps from a Delphi study and validated CATs, the MIE-CAT was composed. Final content was approved by eighteen independent international MIE experts (> 120 MIEs performed). These experts assessed 32 MIE videos from the same surgical team (2010–2020) with the MIE-CAT. For construct validity, videos were divided into groups based on the surgical team’s experience (novice, intermediate, advanced and expert). For reliability assessment, inter- and intra-rater reliability was determined. Based on discrepancies in MIE-CAT’s scores and a focus group, the MIE-CAT 2.0 was composed. MIE-CAT 2.0’s reliability was analyzed based on four experts assessing five videos each. The MIE-CAT contains eight procedural phases. Per phase four quality components are scored on a 1–4 Likert scale, with a total score between [32–128]. Eighty-four assessments were used for analysis. Median MIE-CAT scores were 80 [IQR 75–94], 91 [85–107], 101 [91–106] and 101 [96–106] for respectively novice, intermediate, advanced and expert group. Intra-rater reliability of 0.75 (ICC, 95% CI 0.64–0.83) and inter-rater reliability of 0.52 (ICC, 95% CI 0.38–0.65) was observed. Expert opinions from a utility questionnaire highlighted a useful and practical feedback tool. Results from the MIE-CAT 2.0 are expected in May 2022. The MIE-CAT was developed and validated in this study. Results indicate good reliability within observers and moderate reliability between observers. The MIE-CAT can be used for clinical, training and research purposes. Future work will focus on optimizing the MIE-CAT to increase inter-rater reliability and clinical validation by correlating MIE-CAT scores to patient outcomes.