Deep Learning-Based Quality Assurance for Auto-Segmentation Masks in Radiotherapy

Abstract

Deep learning-based auto-contouring has shown great promise in several disease sites including GU and head and neck. However, quality assurance (QA) is key to identify poor auto-contours which is time consuming. We hypothesis that training a deep learning model to predict contour quality metrics, such as Dice coefficients (DSC) and associated uncertainties for QA. We trained a 3D U-Net-based DL model for segmenting the target and three clinical-relevant OARs (bladder and rectum). To mimic the slice-by-slice review process in clinical practice, we then trained a 2D ResNet-based DL model to predict the 2D DSC for each 2D slice's contour, generated by the 3D segmentation model. Using the Monte Carlo dropout technique, we made 20 independent predictions per slice, with the final DSC calculated as their average and uncertainty estimated as 95% prediction intervals (PI). The study cohort consisted of 912 prostate cancer patients who received definitive radiotherapy. The 3D auto-segmentation model was trained on 129 patients and validated on 20, before being tested on 763 patients. The 2D DSC prediction model was trained on 293 patients with 11116 slices, validated on 73 patients with 2804 slices, and tested on 366 patients with 14117 slices. Rectum was chosen to test the 2D contour QA model as it is the most challenging OAR. We categorized 2D slices into three groups based on the lower and upper bounds of the prediction intervals. "no/minor edits" (lower bound > = 0.9), "major edits" (lower bound < 0.9 and upper bound > = 0.8), and "not acceptable" (upper bound < 0.8). The model performance was quantified by calculating correlation coefficients between predicted and ground truth DSC and the fraction of cases that were correctly identified in each category. The results of the study showed that the overall correlation coefficient between predicted, and ground truth DSC was 0.842. The model was able to correctly identify 78.3%, 60.7%, and 53.4% of the "no/minor edits", "major edits", and "not acceptable" cases, respectively. This study provides a valuable tool for clinicians in making quick decisions on the acceptance, rejection, or revision of auto-segmented masks during the radiation therapy planning process by providing quantitative results on predicted DSC and associated uncertainties.