Abstract
Abstract In this work, we discuss epistemic uncertainty estimation obtained by Bayesian inference in diagnostic classifiers and show that the prediction uncertainty highly correlates with goodness of prediction. We train the ResNet-18 image classifier on a dataset of 84,484 optical coherence tomography scans showing four different retinal conditions. Dropout is added before every building block of ResNet, creating an approximation to a Bayesian classifier. Monte Carlo sampling is applied with dropout at test time for uncertainty estimation. In Monte Carlo experiments, multiple forward passes are performed to get a distribution of the class labels. The variance and the entropy of the distribution is used as metrics for uncertainty. Our results show strong correlation with ρ = 0.99 between prediction uncertainty and prediction error. Mean uncertainty of incorrectly diagnosed cases was significantly higher than mean uncertainty of correctly diagnosed cases. Modeling of the prediction uncertainty in computer-aided diagnosis with deep learning yields more reliable results and is therefore expected to increase patient safety. This will help to transfer such systems into clinical routine and to increase the acceptance of machine learning in diagnosis from the standpoint of physicians and patients.
Highlights
Computer-aided diagnosis (CAD) based on deep learning has been demonstrated to achieve a performance similar to that of human experts in classification tasks in medical imaging [2]
Center for Image Guided Innovation and Therapeutic Intervention (CIGITI), The Hospital for Sick Children, 555 University Ave, Toronto, ON M5G 1X8, Canada, Department of Mathematical and Computational Sciences, University of Toronto Mississauga, 3359 Mississauga Rd, Mississauga, ON L5L 1C6, Canada lutional neural networks (CNN), which was trained on a large database of more than 84,000 retinal OCT images of four different disease states [9]
We show that the prediction uncertainty correlates with accuracy, enabling the identification of false predictions or unknown cases, which were not covered during training
Summary
Computer-aided diagnosis (CAD) based on deep learning has been demonstrated to achieve a performance similar to that of human experts in classification tasks in medical imaging [2]. Lutional neural networks (CNN), which was trained on a large database of more than 84,000 retinal OCT images of four different disease states [9]. The performance in classifying retinal conditions was comparable to that of trained physicians. The main contributions of the work are the integration of uncertainty estimation with Bayesian inference into diagnostic classifiers with an exemplary dataset of retinal OCT scans and extensive experiments regarding the effect of uncertainty estimation to classification accuracy. This paper extends our previous work published at Medical Imaging with Deep Learning (MIDL) 2019 with formal background and more experiments [11]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
