Differentiation of glioblastoma from solitary brain metastasis using deep ensembles: Empirical estimation of uncertainty for clinical reliability

Abstract

Background and ObjectivesTo develop a clinically reliable deep learning model to differentiate glioblastoma (GBM) from solitary brain metastasis (SBM) by providing predictive uncertainty estimates and interpretability. MethodsA total of 469 patients (300 GBM, 169 SBM) were enrolled in the institutional training set. Deep ensembles based on DenseNet121 were trained on multiparametric MRI. The model performance was validated in the external test set consisting of 143 patients (101 GBM, 42 SBM). Entropy values for each input were evaluated for uncertainty measurement; based on entropy values, the datasets were split to high- and low-uncertainty groups. In addition, entropy values of out-of-distribution (OOD) data from unknown class (257 patients with meningioma) were compared to assess uncertainty estimates of the model. The model interpretability was further evaluated by localization accuracy of the model. ResultsOn external test set, the area under the curve (AUC), accuracy, sensitivity and specificity of the deep ensembles were 0.83 (95 % confidence interval [CI] 0.76–0.90), 76.2 %, 54.8 % and 85.2 %, respectively. The performance was higher in the low-uncertainty group than in the high-uncertainty group, with AUCs of 0.91 (95 % CI 0.83–0.98) and 0.58 (95 % CI 0.44–0.71), indicating that assessment of uncertainty with entropy values ascertained reliable prediction in the low-uncertainty group. Further, deep ensembles classified a high proportion (90.7 %) of predictions on OOD data to be uncertain, showing robustness in dataset shift. Interpretability evaluated by localization accuracy provided further reliability in the “low-uncertainty and high-localization accuracy” subgroup, with an AUC of 0.98 (95 % CI 0.95–1.00). ConclusionsEmpirical assessment of uncertainty and interpretability in deep ensembles provides evidence for the robustness of prediction, offering a clinically reliable model in differentiating GBM from SBM.