Can We Improve Prediction of Adverse Surgical Outcomes? Development of a Surgical Complexity Score Using a Novel Machine Learning Technique

Abstract

An optimal method to quantify surgical complexity using patient comorbidities derived from administrative billing data is lacking. We sought to develop a novel, easy-to-use surgical Complexity Score to accurately predict adverse outcomes among patients undergoing elective surgery. A novel surgical Complexity Score was developed using 100% Medicare Inpatient and Outpatient Standard Analytic Files (SAFs) from years 2012 to 2016 (n= 1,049,160). Comorbid conditions were entered into a machine learning algorithm to assign weights to maximize the correlation with multiple postoperative outcomes including morbidity, readmission, mortality, and postoperative super-use. Predictive ability was compared against 3 of the most commonly used risk adjustment indices: the Charlson Comorbidity Index (CCI), Elixhauser Comorbidity Index (ECI), and the Centers for Medicare and Medicaid Service's Hierarchical Condition Category (CMS-HCC). Patients underwent colectomy (12.6%), abdominal aortic aneurysm repair (4.4%), coronary artery bypass grafting (13.0%), total hip replacement (22.0%), total knee replacement (43.0%), or lung resection (5.0%). The Complexity Score had a good to very good predictive ability for all adverse outcomes. The Complexity Score had the highest accuracy in predicting perioperative morbidity (area under the curve [AUC]: 0.868, 95% CI 0.866 to 0.869); this performed better than the CCI (AUC: 0.717, 95% CI 0.715 to 0.719), ECI (AUC: 0.799, 95% CI 0.797 to 0.800), and similar to the CMS-HCC (AUC: 0.862, 95% CI 0.861 to 0.863). Similarly, the Complexity Score outperformed each of the 3 other comorbidity indices in predicting 90-day readmission (AUC: 0.707, 95% CI 0.705 to 0.709), 30-day readmission (AUC: 0.717, 95% CI 0.715 to 0.720), and postoperative super-use (AUC: 0.817, 95% CI 0.814 to 0.820). Compared with the most commonly used comorbidity and surgical risk scores, the novel surgical Complexity Score outperformed the CCI, ECI, and CMS-HCC in predicting postoperative morbidity, 30-day readmission, 90-day readmission, and postoperative super-use.