Using machine learning to improve risk prediction in durable left ventricular assist devices.

Arman Kilic,Artur Dubrawski,Rema Padman,James K Miller,Daniel Dochtermann,Vakhtang Tchantchaleishvili

doi:10.1371/journal.pone.0247866

Arman Kilic, Artur Dubrawski + Show 4 more

Open Access

https://doi.org/10.1371/journal.pone.0247866

Copy DOI

Abstract

Risk models have historically displayed only moderate predictive performance in estimating mortality risk in left ventricular assist device therapy. This study evaluated whether machine learning can improve risk prediction for left ventricular assist devices. Primary durable left ventricular assist devices reported in the Interagency Registry for Mechanically Assisted Circulatory Support between March 1, 2006 and December 31, 2016 were included. The study cohort was randomly divided 3:1 into training and testing sets. Logistic regression and machine learning models (extreme gradient boosting) were created in the training set for 90-day and 1-year mortality and their performance was evaluated after bootstrapping with 1000 replications in the testing set. Differences in model performance were also evaluated in cases of concordance versus discordance in predicted risk between logistic regression and extreme gradient boosting as defined by equal size patient tertiles. A total of 16,120 patients were included. Calibration metrics were comparable between logistic regression and extreme gradient boosting. C-index was improved with extreme gradient boosting (90-day: 0.707 [0.683–0.730] versus 0.740 [0.717–0.762] and 1-year: 0.691 [0.673–0.710] versus 0.714 [0.695–0.734]; each p<0.001). Net reclassification index analysis similarly demonstrated an improvement of 48.8% and 36.9% for 90-day and 1-year mortality, respectively, with extreme gradient boosting (each p<0.001). Concordance in predicted risk between logistic regression and extreme gradient boosting resulted in substantially improved c-index for both logistic regression and extreme gradient boosting (90-day logistic regression 0.536 versus 0.752, 1-year logistic regression 0.555 versus 0.726, 90-day extreme gradient boosting 0.623 versus 0.772, 1-year extreme gradient boosting 0.613 versus 0.742, each p<0.001). These results demonstrate that machine learning can improve risk model performance for durable left ventricular assist devices, both independently and as an adjunct to logistic regression.

Highlights

Recent studies have demonstrated an increasing number of durable left ventricular assist devices (LVADs) being implanted in the United States with improving outcomes [1,2,3]
A prior analysis evaluating the predictive performance of the HeartMate II Risk Score, the Model for End-Stage Liver Disease, and the Destination Therapy Risk Score in estimating 90-day mortality risk after durable LVAD implantation demonstrated area under receiveroperating-characteristic curves (AUROC) or c-indices of only 0.59–0.64 in validation cohorts, for example [4]
Internal cardioverter defibrillators were in place in the majority at the time of LVAD implantation (78.4%)

Summary

Introduction

Recent studies have demonstrated an increasing number of durable left ventricular assist devices (LVADs) being implanted in the United States with improving outcomes [1,2,3]. Despite these trends, there remains no widely utilized risk stratification tool for LVAD therapy. A prior analysis evaluating the predictive performance of the HeartMate II Risk Score, the Model for End-Stage Liver Disease, and the Destination Therapy Risk Score in estimating 90-day mortality risk after durable LVAD implantation demonstrated area under receiveroperating-characteristic curves (AUROC) or c-indices of only 0.59–0.64 in validation cohorts, for example [4]. The aim of this study was to evaluate whether ML can improve risk prediction in patients undergoing LVAD implantation

Objectives

Methods

Results

Discussion

Conclusion