Abstract 14312: Comparison of Machine Learning and Conventional Statistical Modeling for Predicting Readmissions Following Acute Heart Failure Hospitalization

Abstract

Introduction: Developing accurate models for predicting risk of 30-day readmission has been a major healthcare interest. Evidence suggests that models developed using machine learning (ML) may have better discrimination than conventional statistical models (CSM), but the calibration of such models is unclear. Objectives: To compare models developed using CSM or ML to predict 30-day readmission for cardiovascular and non-cardiovascular causes in HF patients. Methods: We studied 10,919 patients with HF (> 18 years) discharged alive from a hospital or emergency department (2004-2007) in Ontario, Canada, linked to administrative databases for hospitalization and vital status resulting in complete follow-up. The study sample was randomly divided into training and validation sets in a 2:1 ratio. CSMs to predict 30-day readmission were developed using Fine-Gray subdistribution hazards regression (treating death as a competing risk), and the ML algorithm employed random survival forests. Models were evaluated in the validation set using both discrimination and calibration metrics. Results: In the validation sample of 3602 patients (median age 76 [IQR, 67-82] years, 46.6% females), Random Survival Forests (c-statistic = 0.620) showed similar discrimination to the Fine-Gray competing risk model (c-statistic= 0.621) for 30-day cardiovascular readmission. In contrast, for 30-day non-cardiovascular readmission, the Fine-Gray model (c-statistic= 0.641) slightly outperformed the random survival forests model (c-statistic = 0.632). For both outcomes, The Fine-Gray model displayed better calibration than random survival forests when deciles of observed vs. predicted risks were compared (Panels A-D). Conclusions: In HF patients, time-to-event analysis of outcomes using Fine-Gray models had similar discrimination but superior calibration to ML model, highlighting the importance of reporting calibration metrics for ML-based prediction models.