Development and validation of a machine learning–based, point-of-care risk calculator for post-ERCP pancreatitis and prophylaxis selection

Abstract

A robust model of post-ERCP pancreatitis (PEP) risk is not currently available. We aimed to develop a machine learning-based tool for PEP risk prediction to aid in clinical decision making related to periprocedural prophylaxis selection and postprocedural monitoring. Feature selection, model training, and validation were performed using patient-level data from 12 randomized controlled trials. A gradient-boosted machine (GBM) model was trained to estimate PEP risk, and the performance of the resulting model was evaluated using the area under the receiver operating curve (AUC) with 5-fold cross-validation. A web-based clinical decision-making tool was created, and a prospective pilot study was performed using data from ERCPs performed at the Johns Hopkins Hospital over a 1-month period. A total of 7389 patients were included in the GBM with an 8.6% rate of PEP. The model was trained on 20 PEP risk factors and 5 prophylactic interventions (rectal nonsteroidal anti-inflammatory drugs [NSAIDs], aggressive hydration, combined rectal NSAIDs and aggressive hydration, pancreatic duct stenting, and combined rectal NSAIDs and pancreatic duct stenting). The resulting GBM model had an AUC of 0.70 (65% specificity, 65% sensitivity, 95% negative predictive value, and 15% positive predictive value). A total of 135 patients were included in the prospective pilot study, resulting in an AUC of 0.74. This study demonstrates the feasibility and utility of a novel machine learning-based PEP risk estimation tool with high negative predictive value to aid in prophylaxis selection and identify patients at low risk who may not require extended postprocedure monitoring.