Ranking Information Extracted from Uncertainty Quantification of the Prediction of a Deep Learning Model on Medical Time Series Data

Ruxandra Stoean,Roberto Rodríguez-Labrada,Gonzalo Joya,Miguel Atencia,Catalin Stoean

doi:10.3390/math8071078

Abstract

Uncertainty quantification in deep learning models is especially important for the medical applications of this complex and successful type of neural architectures. One popular technique is Monte Carlo dropout that gives a sample output for a record, which can be measured statistically in terms of average probability and variance for each diagnostic class of the problem. The current paper puts forward a convolutional–long short-term memory network model with a Monte Carlo dropout layer for obtaining information regarding the model uncertainty for saccadic records of all patients. These are next used in assessing the uncertainty of the learning model at the higher level of sets of multiple records (i.e., registers) that are gathered for one patient case by the examining physician towards an accurate diagnosis. Means and standard deviations are additionally calculated for the Monte Carlo uncertainty estimates of groups of predictions. These serve as a new collection where a random forest model can perform both classification and ranking of variable importance. The approach is validated on a real-world problem of classifying electrooculography time series for an early detection of spinocerebellar ataxia 2 and reaches an accuracy of 88.59% in distinguishing between the three classes of patients.

Highlights

Deep learning (DL) has uncovered substantial findings in supporting decision making in medicine.The high accuracy of its predictions has triggered its increasing use for a variety of clinical classification tasks [1]
There is a large number of papers in this uncertainty quantification (UQ) direction in the application of DL for medical image processing, the area related to biomedical time series data is not so dense in Monte Carlo dropout (MCD)
The registers from the third class, sick, are easier to distinguish, since a majority of saccades have a non-healthy appearance. It is not the first time UQ is applied for the current task, a previous study led to the results in Reference [4]

Summary

Introduction

Deep learning (DL) has uncovered substantial findings in supporting decision making in medicine.The high accuracy of its predictions has triggered its increasing use for a variety of clinical classification tasks [1]. In order to be truly effective in practice, the confidence in its output must be expressed beyond the resulting probabilities for the classes of the problem. Other factors that derive from the experience of the clinician and influence the diagnosis are not encompassed within the data. This epistemic uncertainty together with the inherent aleatory unpredictability of the process can be tackled through methods for uncertainty quantification (UQ). The two best-known current practices in UQ are the use of Monte Carlo dropout (MCD) [2] at the prediction phase and the constitution of deep ensembles [3]

Methods

Results

Discussion

Conclusion