PMHnet-alpha: development and validation of a neural network based discrete-time survival model for mortality prediction in ischemic heart disease

Abstract

Abstract Background Current risk prediction models in ischemic heart disease (IHD) use a small set of well-known risk factors, have limited predictive capabilities, and are largely the same as they were twenty years ago. We developed and externally validated PMHnet-alpha, a neural-network based survival model for risk-stratification in ischemic heart disease that leverages the multitude of clinical features available in modern electronical health records. Methods We included 39,746 IHD patients from the regional Heart Registry that had been subjected to a coronary angiography between 2006 and 2017 with confirmed coronary artery disease. Clinical data was extracted from the Danish National Patient Registry, and electronic health records. 595 different features, consisting of diagnosis codes, procedure codes, biochemical test results, and clinical measurements were used as model inputs. Prior to model development, patients were randomly divided into a training set (n=34,746) and a tesing set (n=5,000). The testing set was not used for model development. Model performance was evaluated at six months, one years, three-, and five years of follow-up using time-dependent ROC curve analysis and Harrels' C-index. Lastly, we also assessed the calibration of the model. We benchmarked the performance of PMHnet-alpha against the GRACE Risk Score 2.0, which is widely considered to the best-performing model in current clinical use. We explored the importance of individual features using SHAP values on the trained models. Findings PMHnet-alpha had very high model discrimination on the testing data with time-dependent AUCs of 0.88 (95% CI 0.86–0.90) at six months, 0.88 (95% CI 0.86–0.90) at one year, 0.84 (95% CI 0.82–0.86) at three years, and 0.82 (95% CI 0.80–0.84) at five years. The discrimination of the benchmark model GRACE2.0 on the same data was considerably lower, 0.77 (95% CI 0.73–0.80) at six months, 0.77 (95% CI 0.74–0.80) at one year, and 0.73 (95% CI 0.70–0.75) at three years. PMHnet-alpha is undergoing external validation in other nordic countries. We identified that on-average, age, coronary pathology and smoking status were the most impactful features. Interpretation Here we present a significant improvement of the state of the art in cardiac risk prediction. PMHnet-alpha supports better and optimized use of available healthcare data, signified by the vast improvement compared to GRACE2.0. This also signifies an important paradigm shift in which data-driven strategies are necessary to transform the increasing amount of data generated in the modern healthcare system into evidence-based clinical decision making. Funding Acknowledgement Type of funding sources: Foundation. Main funding source(s): The Novo Nordisk Foundation, NordForsk