Machine-learning-derived online prediction models of outcomes for patients with cholelithiasis-induced acute cholangitis: development and validation in two retrospective cohorts

Abstract

Cholelithiasis-induced acute cholangitis (CIAC) is an acute inflammatory disease with poor prognosis. This study aimed to create machine-learning (ML) models to predict the outcomes of patients with CIAC. In this retrospective cohort and ML study, patients who met the both diagnosis of 'cholangitis' and 'calculus of gallbladder or bile duct' according to the International Classification of Disease (ICD) 9th revision, or met the diagnosis of 'calculus of bile duct with acute cholangitis with or without obstruction' according to the ICD 10th revision during a single hospitalization were included from the Medical Information Mart for Intensive Care database, which records patient admissions to Beth Israel Deaconess Medical Center, MA, USA, spanning June 1, 2001 to November 16, 2022. Patients who were neither admitted in an emergency department nor underwent biliary drainage within 24h after admission, had an age of less than 18, or lost over 20% of the information were excluded. Nine ML methods, including the Logistic Regression, eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine, Adaptive Boosting, Decision Tree, Gradient Boosting Decision Tree, Gaussian Naive Bayes, Multi-Layer Perceptron, and Support Vector Machine were applied for prediction of in-hospital mortality, re-admission within 30 days after discharge, and mortality within 180 days after discharge. Patients from Zhongda Hospital affiliated to Southeast University in China between January 1, 2019 and July 30, 2023 were enrolled as an external validation set. The area under the receiver operating characteristic curve (AUROC) was the main index for model performance assessment. A total of 1156 patients were included to construct models. We performed stratified analyses on all patients, patients admitted to the intensive care unit (ICU) and those who underwent biliary drainage during ICU treatment. 13-16 features were selected from 186 variables for model training. The XGBoost method demonstrated the most optimal predictive efficacy, as evidenced by training set AUROC of 0.996 (95% CI NaN-NaN) for in-hospital mortality, 0.886 (0.862-0.910) for re-admission within 30 days after discharge, and 0.988 (0.982-0.995) for mortality within 180 days after discharge in all patients, 0.998 (NaN-NaN), 0.933 (0.909-0.957), and 0.988 (0.983-0.993) in patients admitted to the ICU, 0.987 (0.970-0.999), 0.908 (0.873-0.942), and 0.982 (0.971-0.993) in patients underwent biliary drainage during ICU treatment, respectively. Meanwhile, in the internal validation set, the AUROC reached 0.967 (0.933-0.998) for in-hospital mortality, 0.589 (0.502-0.677) for re-admission within 30 days after discharge, and 0.857 (0.782-0.933) for mortality within 180 days after discharge in all patients, 0.963 (NaN-NaN), 0.668 (0.486-0.851), and 0.864 (0.757-0.970) in patients admitted to the ICU, 0.961 (0.922-0.997), 0.669 (0.540-0.799), and 0.828 (0.730-0.925) in patients underwent biliary drainage during ICU treatment, respectively. The AUROC values of external validation set consisting of 61 patients were 0.741 (0.725-0.763), 0.812 (0.798-0.824), and 0.848 (0.841-0.859), respectively. The XGBoost models could be promising tools to predict outcomes in patients with CIAC, and had good clinical applicability. Multi-center validation with a larger sample size is warranted. The Technological Development Program of Nanjing Healthy Commission, and Zhongda Hospital Affiliated to Southeast University, Jiangsu Province High-Level Hospital Construction Funds.