An interpretable model for incident heart failure prediction with uncertainty estimation

Abstract

Abstract Background Forecasting incident heart failure is a critical demand for prevention. Recent research suggested the superior performance of deep learning models on the prediction tasks using electronic health records. However, even with a relatively accurate predictive performance, the major impediments to the wider use of deep learning models for clinical decision making are the difficulties of assigning a level of confidence to model predictions and the interpretability of predictions. Purpose We aimed to develop a deep learning framework for more accurate incident heart failure prediction, with provision of measures of uncertainty and interpretability. Methods We used a longitudinal linked electronic health records dataset, Clinical Practice Research Datalink, involving 788,880 patients, 8.3% of whom had an incident heart failure diagnosis. To embed the uncertainty estimation mechanism into the deep learning models, we developed a probabilistic framework based on a novel transformer deep learning model: deep Bayesian Gaussian processes (DBGP). We investigated the performance of incident heart failure prediction and uncertainty estimation for the model and validated it using an external held-out dataset. Diagnoses, medications, and age for each encounter were included as predictors. By comparing the uncertainty, we investigated the possibility of identifying the correct predictions from wrong ones to avoid potential misclassification. Using model distillation meant to mimic a well-trained complex model with simple models, we investigated the importance of associations between diagnoses, medications and heart failure with an interpretable linear regression component learned from DBGP. Results The DBGP achieved high precision with 0.941 as AUROC for external validation. More importantly, it showed the uncertainty information could distinguish the correct predictions from wrong ones, with significant difference (p-value with 500 samples) between distribution of uncertainties for negative predictions (3.21e-69 between true negative and false negative), and positive predictions (3.39e-22 between true positive and false positive). Utilising the distilled model, we can specify the contribution of each diagnosis and medication to heart failure prediction. For instance, Losartan/Fosinopril, Bisoprolol and Left bundle-branch block showed strong association to heart failure incidence with coefficient 0.11 (95% CI: 0.10, 0.12), 0.09 (0.08, 0.11) and 0.09 (0.07, 0.11) respectively; Peritoneal adhesions, Trochanteric bursitis and Galactorrhea showed strong disassociations with coefficient −0.07 (−0.09, −0.05), −0.07 (−0.09, −0.04) and −0.06 (−0.08, −0.04) individually. Conclusions Our novel probabilistic deep learning framework adds a measure of uncertainty the prediction and helps to mitigate misclassification. Model distillation provides an opportunity to interpret deep learning models and offers a data-driven perspective for risk factor analysis. Funding Acknowledgement Type of funding source: Public Institution(s). Main funding source(s): Oxford Martin School,University of Oxford; NIHR Oxford Biomedical Research Centre, University of Oxford