82 A Comparison of Machine Learning and Logistic Regression Performance for the Prediction of In-Hospital Mortality Among Acute Biliary Pancreatitis Patients

Abstract

INTRODUCTION: For population databases, traditional multivariate analysis is accepted standard for predictive modelling of mortality in acute biliary pancreatitis (ABP). Machine Learning (ML) and artificial intelligence (AI) are novel methodologies to analyze databases. The gradient boosting machine (GBM) is a tree-based ML approach. Tree-based algorithms partition the data based on predictors in an attempt to accurately classify the outcome. The gradient boosting approach employs successive trees to improve upon misclassifications of the earlier trees. Alternatively, logistic regression is a statistical approach, in which predicted probabilities are generated based on parameters estimated by maximum likelihood estimation. With the advent of universal electronic medical records, patient databases are more complex, large, and interlinked to multiple institutions. Hence, we sought to compare traditional statistical methods and ML in building prediction models for mortality in ABP. METHODS: Using the Nationwide Readmission Database (2010–2014), we identified all patients (age > 18 years) with index admissions for ABP. This data was randomly divided into a training set (70%) and a test set (30%), stratified by outcome in order to get similar event rates in both sets. The same training and test sets were used for GBM learning and logistic regression. The area under the ROC curve (AUROC) was calculated for both GBM and logistic regression modelling for predicting mortality in ABP. RESULTS: A total of 97,027 patients were hospitalized for ABP from 2010–2014 in the US. The in-hospital mortality rate was 0.97% (n = 944). Using the same set of predictor variables (demographic, hospital, etiological, comorbidities, severity of acute pancreatitis (AP), procedures, and duration of stay), ML and logistic regression performed similarly in predicting mortality with GBM achieving an AUROC of 0.955 and logistic regression AUROC 0.948 (Figure 1). Key predictors of mortality for AP using logistic regression included: Severe AP, sepsis, increasing age, and failure to perform cholecystectomy. Foremost predictors using GBM included length of stay, age, severe AP and income quartile. CONCLUSION: Notwithstanding emergence of novel AI methodologies, traditional logistic regression is comparable to novel ML algorithms in the predictive modelling of hospital outcomes in ABP and can continue to be utilized in large population databases.