EXPLAINABLE MACHINE LEARNING MODEL FOR PREDICTING GI BLEED MORTALITY IN THE ICU

Abstract

SESSION TITLE: Critical Care Posters SESSION TYPE: Original Investigation Posters PRESENTED ON: October 18-21, 2020 PURPOSE: Acute upper and lower gastrointestinal (GI) bleed is a common reason for hospitalization with 2-10% risk of mortality. Several scoring systems for mortality prediction in GI bleed have been developed, however, none of these are considered part of the standard of care. In this study we used machine learning to develop a model to calculate the risk of mortality in critically ill patients admitted for GI bleed and compared it to the traditionally used mortality risk assessment score, APACHE IVa. Since one of the limitations for using machine learning models in practical setting has been lack of physician trust in the model, we utilized explainable methods to provide insight into the model's prediction and outcome. METHODS: We analyzed patient data in the eICU Collective Research Database consisting of critically ill patients and extracted data for 5691 patients (mean age = 67.4 years; 61% males) admitted with GI bleed (upper-, lower- and unknown GI bleed). The data was utilized in training a gradient-boosting machine learning model to identify patients who died in the ICU. We compared the predictive performance of the machine learning model with the APACHE IVa score, a widely accepted risk scoring system for critically ill patients. Performance was measured by area under receiver operating characteristic curve (AUC) analysis. This study also utilizes explainable machine learning methods to provide insights into the model’s outcome prediction using the SHAP (SHapley Additive exPlanations) method. RESULTS: The machine learning model performed better than APACHE IVa risk score in correctly classifying patients who are low risk. The ML model had a specificity of 55% (95% CI: 52 – 64) at sensitivity of 100% compared with APACHE score which had a specificity of 4% (95% CI: 3 –43) at sensitivity of 100%. The model identified patients who died with an AUC of 0.90 (95% CI: 0.93 - 0.86) in the internal validation set while the APACHE IVa clinical scoring systems identified patients who died with AUC values of 0.82 (95% CI: 0.88 - 0.76) with p-value <0.001. CONCLUSIONS: We developed a machine learning model that identifies patients with GI bleed who are at risk of dying with a greater accuracy than current risk scoring systems. We also provide insights into why the model is predicting a certain outcome in order to build trust with physician. CLINICAL IMPLICATIONS: We developed a machine learning model that predicts mortality in patients with GI bleed with a greater AUC than that of the current scoring system. This model could help with identification of critically ill GI bleed patients who are at low risk of mortality and can be moved out of the ICU into a step-down unit for monitoring. By making machine learning model explainable, clinicians would be better able to understand the reasoning behind the outcome and thus would be more likely to implement it in clinical practice to make patient-centered decisions. DISCLOSURES: No relevant relationships by Farah Deshmukh, source=Web Response No relevant relationships by Shamel Merchant, source=Web Response