Automated machine learning (AutoML) with AutoGluon to predict post-operative surgical site infection (SSI) using the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) database

Abstract

Introduction: Risk calculators to predict post-operative surgical site infections (SSI) often rely on logistic regression (LR) analysis. Machine learning models are able incorporate a greater number of input variables by identifying non-linear relationships. Automated machine learning (AutoML) processes regularly outperform regular machine learning (ML) and LR methods for predictive accuracy outside of healthcare settings. AutoML systems have not yet been applied to predict post-operative surgical site infections. Methods: We used an AutoML system developed and released by Amazon in 2020, AutoGluon v0.3.1, to predict post-operative SSI in the 2019 ACS NSQIP database. A total of 3,049,617 patients and 79 pre-operative variables were included. Post-operative SSI was defined as a superficial, deep or organ-space infection that occurred within 30 days of the surgery and was not present prior to surgery. Models were trained for four hours to optimize performance on the Brier score, with lower being better. Validation of all performance metrics was done using the 2019 ACS NSQIP database. Results: 2.41% of the patients (n = 73,581) developed post-operative SSI. Brier scores were calculated for each model with the top performing model being an ensembled XGBoost model having a Brier score of 0.00287 on the validation set. The corresponding AUROC and AUC-PR was 0.768 and 0.110 respectively (Figure). Conclusions: Automated machine learning models offer similar discriminatory characteristics to existing postoperative SSI calculators. Future post-operative SSI models may benefit from AutoML analysis.