Evaluation of AI vs. Clinical Experts SBRT-Thorax Computed Tomography OARs Delineation

Abstract

<h3>Purpose/Objective(s)</h3> Organ-at-risk (OAR) delineation is an essential step for radiotherapy treatment (RT) planning. Standard practice during the last decade referred to manual delineation of OARs from medical experts. Manual delineation is the current standard of practice and can be a tedious, time-consuming process, prone to errors due to intra and inter-observer variations. This is the case for lung stereotactic body radiation therapy (SBRT) due to the variety of structures to be contoured. The latest advances of artificial intelligence methods offer new perspectives towards a fully automatic delineation. This study aims at evaluating an AI-based auto-contouring solution (AC) and compare its clinical acceptability against contours delineated by experts (EC). <h3>Materials/Methods</h3> In this study, a CE/FDA cleared anatomically preserving ensemble deep-learning architecture contouring solution was used. The AI-solution was trained using more than 300+ fully annotated multi-centric SBRT cases according to the ESTRO guidelines (male and female) cases. A fully external cohort of 30 additional patients was considered to assess performance quantitatively and qualitatively. In terms of quantitative metrics, the Dice coefficient was used while qualitative assessment was done through a scoring mechanism: A/acceptable, B/ acceptable after minor corrections, and C/ not acceptable for clinical use. In terms of reference/comparisons with clinical experts, two independent annotations were used while treatment experts' contours were blended blindly with the AI-solution at 50/50 ratio. Random blending at the patient level was performed guaranteeing that, among contours being evaluated per patient and OAR, the 50/50 split was satisfied. <h3>Results</h3> The mean Dice coefficient between expert and AI was 88 % which dropped to 87% among experts. Overall clinical acceptability after aggregating blinded evaluations for the combined categories (A+B) was 95% for the AC which dropped to 81% for EC. When looking at the overall acceptability of contours that do not require any modifications (A), the AC (67%) outperformed the EC (48%) by significant margin. AC outperformed the EC on 14 structures. <h3>Conclusion</h3> This work reports clinical evaluation of AC solution on CT scans for Thorax-SBRT. Our results suggest that the deep learning model could be a viable clinical alternative to the human expert offering treatment standardization and potentially better outcomes.