Interpretable uncertainty estimation for automated triage of 12-lead electrocardiogram using deep convolutional neural networks

Abstract

Abstract Background Interpretation of electrocardiograms (ECGs) using artificial intelligence (AI) has shown very promising results recently, such as comprehensive triage of ECGs. Despite this progress, remarkably few of these AI algorithms have been implemented in clinical practice. Mayor challenges before safe widespread implementation is possible, are uncertainty estimation and interpretability. Without these, algorithms are forced to always provide a diagnosis or prediction. When the algorithm has never seen such an ECG before or when it is noisy or contains a difficult or ambiguous case, this could lead to unrecognized erroneous results. Purpose We propose a novel deep neural network (DNN) architecture that is able to overcome both the interpretability and uncertainty estimation challenges in automated diagnosis of ECGs. Methods A dataset with 320.000 raw 10s 12-lead ECGs was used to train and validate the DNN in separate 90:10 splits. All ECGs were interpreted by a physician and these free-text annotations were translated into 6 categories from normal to acute based on how quickly a cardiologist needs to be consulted. Using a novel variational autoencoder DNN, the ECG is compressed into a latent space with 64 parameters. In the compressed space, each category has a separate distribution, instead of a single distribution for the whole dataset. For clinical use, the ECG is represented in the compressed space and the category prior distribution that is the most likely to produce the input ECG representation is the predicted category. The smaller the probability of generating the data point under any of the known categories, the more uncertain the prediction. The compressed space is reduced using principal component analysis (PCA) to visualize where the input is represented in the compressed landscape. Performance of the uncertainty estimation is assessed in the validation set by ranking the uncertainty in 10 bins from highest to lowest and comparing with the accuracy. Results We demonstrate that the accuracy of the algorithm decreases for ECGs where the model is more uncertain about its predictions (Figure 1). Prediction accuracies range from 90% for the ECGs where the algorithm is most certain, to 50% where it is most uncertain. The PCA of the latent space appears to be able to visually represent the categories in the latent space (Figure 2). Moreover, it indicates whether the new ECG belongs to a known category, the algorithm is doubting between categories or the ECG belongs to none of the categories. Conclusion The proposed novel method is promising in estimating uncertainty in an interpretable way, which could allow for safer implementation of AI-based ECG interpretation. Further research is needed to improve the baseline performance of the model, to compare to existing methods and to examine the detection of noisy or new ECGs. The interpretability can be further extended to highlight the uncertain regions on the ECGs. Funding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): ZonMw Figure 1. Uncertainty versus accuracyFigure 2. Representation of ECG