Integrating Workshop Training and Artificial Intelligence (AI) to Contour Organs at Risk (OARs) in Thoracic Radiation Therapy

Abstract

This work aimed to quantify and explore the effects of expert training and AI on improving the consistency in Organs at Risk (OAR) delineations on the and the concordance to RTOG Atlas for future work of multi-center clinical trials. This is a result of an on-site clinical training workshop/Global Collaborative Oncology Group (GCOG) and an online cross-over study of AI application on key thoracic OARs. The participating experts were provided with the same de-identified cases of stage III Non-Small Cell Lung Cancer (NSCLC) and instructions on structure contouring and plan assessment criteria according to RTOG1106. Required OARs included lungs minus gross tumor volume, heart, spinal cord, esophagus and brachial plexus. The reference OAR contours were generated by one expert investigator of a RTOG trial on locally advanced NSCLC. The consistency between contours from the physicians and the concordance of contours to the Standard OAR contours were evaluated by the dice similarity coefficient (DSC). Post-workshop contours and AI-assistant contours were further assessed for consistency and concordance. A total of 35 centers participated in the workshop, 32 of them with complete DICOM files were eligible for the DSC analysis. Following the given instructions and atlas before training, for RTOG 1106, this experiment showed the mean DSC of the lung, heart, spinal cord, esophagus and brachial plexus were 0.91±0.15, 0.84±0.17, 0.72±0.22, 0.73±0.16 and 0.34±0.11, respectively. The DSC of the lung and heart of this case with earlier stage of disease were 0.96±0.06 and 0.88±0.07, respectively, which were better than that of the first case with mode locally advanced disease. The mean DSC of Contours after GCOG training of lung, heart, spinal cord, esophagus and brachial plexus were 0.92±0.15, 0.81±0.24, 0.70±0.26, 0.70±0.24 and 0.26±0.14, respectively. In the crossover experiment, the mean DSC of contours without AI assistance of lung, heart, spinal cord, esophagus and brachial plexus were 0.94±0.02, 0.87±0.07, 0.85±0.04, 0.81±0.04, and 0.38±0.12, respectively. The mean DSC of contours with AI assistance of lung, heart, spinal cord, esophagus and brachial plexus were 0.95±0.01, 0.91±0.04, 0.84±0.06, 0.81±0.02 and 0.40±0.05, respectively. The mean decreased time of delineating the lung, heart, spinal cord, esophagus and brachial plexus were 29 mins, 3 mins, 6 min, 4 min and 3 min, respectively, with a total of 45 mins for one case. This study demonstrated 1) remarkable variations in OAR delineation among 32 investigators, 2) notable improvement in concordance after the training workshop, and 3) significant improvement in consistency with AI assistance, along with time-saving of treating physicians. More training with better AI tools may further improve the contouring consistency, which is important for clinical practice and clinical trial.