Using Prediction Uncertainty to Identify Reliable Predictions for a Deep Learning Model that Predicts the Need for Early Feeding Tube Placement

Abstract

<h3>Purpose/Objective(s)</h3> Radiotherapy (RT) for head and neck cancer is associated with odynophagia, dysphagia, and weight loss, which compromise quality-of-life and often lead to feeding tube (FT) placement. Early identification of patients who will require a FT would reduce weight loss and its associated complications. We hypothesized a deep learning (DL) model could identify these patients better than using early weight loss trends and that uncertainty estimates could stratify reliable from unreliable predictions. <h3>Materials/Methods</h3> We obtained planning CTs, dose, and CBCTs within 6-10 days of starting RT for 271 patients treated from 2016-2020 at a single institution. Images were cropped to include the oral cavity, pharynx, and cervical esophagus and used as inputs to the model. We used a ResNet50 with pretrained weights obtained from MedicalNet further trained via transfer learning employing 5-fold cross validation (CV). We predicted a composite endpoint of FT placement or ≥10% weight loss after 20 days from starting RT. Epistemic and aleatoric uncertainty were estimated using dropout variation inference and test-time image augmentation. Three-hundred predictions for each method and image were obtained; the mean class predictions were used to calculate the informational entropy (i.e., uncertainty value). Median values obtained from the validation set were used to stratify performance on the test set. 95% CI for AUCs obtained from ROC curves were calculated and unpaired Delong test was used to compare performance (R-4.1.2). <h3>Results</h3> Our cohort was 81% male, 37% non-smokers, 82% oropharynx/larynx, 82% received chemoRT. The endpoint was seen in 42% of patients. An AUC of 0.72 (95% CI: 0.66-0.78; sensitivity/specificity: 0.69/0.69) was obtained on the test set. Weight loss >5% at day 10 obtained an AUC of 0.54. An AUC of 0.78 (0.70-0.85; 0.87/0.63) was obtained on the epistemic "certain" patients and an AUC of 0.62 (0.52-0.72; 0.42/0.85) was obtained on the "uncertain" patients (p=0.01). An AUC of 0.78 (0.71-0.86; 0.69/0.77) was obtained on the aleatoric "certain" patients and an AUC of 0.65 (0.55-0.74; 0.63/0.70) was obtained on the "uncertain" patients (p=0.03). Table 1 reports per fold test set performance in 5-fold CV. <h3>Conclusion</h3> Despite a relatively subjective endpoint of "needing a FT", we showed our DL model provided early FT predictions greatly improved over using weight loss alone. We showed that epistemic and aleatoric uncertainty estimates identified a cohort of patients where our model was more accurate. These uncertainty techniques are applicable to similar image-based DL models and may have utility when any such models are deployed in clinics.