Estimating Ensemble Bias using Bayesian Convolutional Neural Network

Abstract

Convolutional neural networks (CNN) are powerful tools in many medical imaging applications including denoising. However, a major concern in deploying a CNN in safety-critical areas is to access its prediction accuracy on out-of-distribution test samples. Our previous study showed that while a CNN could generate similar ensemble standard deviation levels on inputs with different noise levels, the magnitude and location of bias could be substantially different. These observations imply that although CNNs can generate visually high-quality images from each individual dataset, the bias might be higher in one image than the other, which might lead to misdiagnosis. Therefore, it is crucial for us to understand when and where a CNN is uncertain about its prediction in real clinical datasets. In this study, we investigate the use of a Bayesian Convolutional Neural Network (BCNN) to estimate an uncertainty map of its prediction, and assess its correlations with the ensemble bias of its own and a conventional CNN's prediction ensemble bias, respectively. The BCNN is implemented by adding Monte Carlo dropout layers after each convolution and ReLU layer, which is equivalent to imposing a Bernoulli prior distribution on the network weights. At the testing stage, we acquire multiple stochastic forward passes through the network to generate the uncertainty map. We applied BCNN and CNN on two 18F-FDG scans, with one whole-body scan which is considered as an in-distribution test sample, and one brain only study which is considered as an out-of-distribution sample. For the whole-body scan, we generated 10 noise realizations for ensemble analysis using bootstrap. Our results demonstrate that BCNN can generate a prediction uncertainty map that highly correlates with the ensemble bias map. This technique could provide important guidance on interpreting the denoising results from a neural network where ground truth is unknown.