Abstract 15199: Automated Machine Learning (AutoML) Algorithms to Predict Post-Operative Venous Thromboembolism (VTE) Using the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Database

Abstract

Introduction: Post-operative patients are at increased risk for venous thromboembolism (VTE). Given the morbidity and mortality associated with VTE, risk stratification calculators have been developed based on pre-operative patient characteristics. Most risk calculators rely on logistic regression (LR) analysis. However, automated machine learning (AutoML) programs consistently outperform standard LR models in non-medical contexts. This study aims to investigate the utility of novel methods in developing a model for post-operative VTE after non-cardiac and cardiac surgeries. Hypothesis: We hypothesized that AutoML models would be superior to logistic regression models in predicting post-operative VTE. Methods: We used an AutoML system developed and released by Amazon in 2020, AutoGluon v0.3.1, to predict post-operative VTE using the 2016-2018 ACS NSQIP database. A total of 3,049,617 patients and 79 pre-operative variables were included. Post-operative VTE was defined as a deep venous thrombosis (DVT) or a pulmonary embolism (PE) within 30 days of the surgery. Models were trained for four hours to optimize performance on the Brier score, with lower being better. Validation of all performance metrics was done using the 2019 ACS NSQIP database. Results: 0.79% of the patients (n = 23,974) developed post-operative VTE. Brier scores were calculated for each model with the top performing model being an ensembled neural net model having a Brier score of 0.00758 on the validation set. The corresponding AUROC and AUC-PR was 0.784 and 0.035 respectively (Figure). Conclusions: The models generated via AutoML to predict post-operative VTE had similar discriminatory characteristics to those reported in the literature. Future post-operative VTE models may benefit from AutoML analysis.