Machine Learning Approaches Incorporating High-Dimensional Longitudinal Data Improve the Prediction of Survival after Allogeneic Hematopoietic Cell Transplantation

Abstract

Introduction: The European Society for Blood and Marrow Transplantation (EBMT) risk score and the Hematopoietic Cell Transplant Comorbidity Index (HCT-CI) remain suboptimal for predicting mortality after allogeneic hematopoietic cell transplantation (HCT) in pediatric patients. These risk scores are calculated based on a small number of variables collected before HCT. Their predictive performance is limited because of the inherent procedural uncertainties, the statistical methodologies used, and the limited number and quality of variables included. By combining multi-institutional, comprehensive, and patient-centric data, machine learning (ML)-based techniques provide an opportunity to develop more accurate risk prediction algorithms by incorporating longitudinal data collected before and after HCT. To our knowledge, no risk prediction algorithms incorporate longitudinal data collected after HCT into the prediction of overall survival (OS) for pediatric patients. We hypothesized that by including high-dimensional longitudinal data such as objective clinical and laboratory assessments from electronic health records (EHRs), models built with ML could help identify patients at high risk of mortality. To test this hypothesis, we developed an ML model incorporating such longitudinal data, which were readily available from EHRs, to predict the 1-year OS among pediatric patients undergoing allogeneic HCT, and we compared its prediction performance with that of a model involving only the baseline variables normally used in standard risk prediction models. Patients and Methods: This study included a cohort of 770 pediatric patients who underwent their first allogeneic HCT at St. Jude Children's Research Hospital between 2000 and 2020. Combining the baseline data (e.g., disease, donor, and graft characteristics) and the longitudinal evaluation data (e.g., blood cell counts, serum chemistries, measurable residual disease) collected within 30 days before or after HCT, we built a predictive model that uses penalized logistic regression to identify patients at high risk for mortality within 1 year after HCT. We compared the proposed ML model incorporating longitudinal measurements with a model trained only on baseline data and analyzed the importance of the different input features. Results: The area under the curve (AUC), either the receiver operating characteristic (ROC) curve or the precision-recall (PR) curve, measures the prediction accuracy of the estimated model. The AUC ranges from 0.0 and 1.0, with a higher AUC indicating a more accurate prediction. The proposed ML model has achieved an ROC AUC of 0.760 (95% confidence interval [CI]: 0.739-0.779), significantly outperforming the baseline model, for which the ROC AUC is 0.698 (95% CI: 0.675-0.718). This predictive advantage of the proposed ML model is also reflected in the PR AUC of 0.680 (95% CI: 0.651-0.712), as compared with 0.513 (95% CI: 0.482-0.546). The feature importance analysis suggests that in prediction modeling, the learned model relies heavily on the longitudinal evaluations, including the concentrations of serum magnesium, aspartate aminotransferase (AST), and ferritin, and the platelet count before and after HCT in addition to certain baseline variables such as the underlying diagnosis and graft type. Conclusion: Leveraging readily available longitudinal data from EHRs, we developed an ML model for 1-year OS prediction in pediatric patients undergoing allogeneic HCT. Our model shows that the results of frequently monitored laboratory and clinical tests performed within 30 days before or after HCT can potentially identify patients with a high risk of mortality after HCT. We are currently validating this model by using external datasets. Such ML models could be included in the EHR system as clinical decision support tools (see an R Shiny app created for data analysis and an illustration of the results at https://sjbiostat.shinyapps.io/pedsHCT), enabling better dynamic stratification of patient risk, optimization of clinical care, and improved therapeutic outcomes. The findings of this study reflect the fact that ML approaches are hugely augmentative and vital to efforts to transform large pools of data into valuable information as a result of their ability to extract predictive features efficiently from high-dimensional and heterogeneous data sources. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal