Validation of the American College of Surgeons surgical risk calculator for thoracic surgery.

Abstract

Advances in medicine and surgical techniques make it possible to operate on selected comorbid elderly patients for whom risk assessment is essential. We aimed to validate the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) surgical risk calculator specifically for thoracic surgery. This study retrospectively included 283 consecutive patients who all underwent various thoracic surgeries at our center. Considering "serious complication" as the most important outcome, we compared the predicted risk scores with the observed incidence of 30-day morbidity and mortality. We calculated the area under the receiver operating characteristic curve (AUROC) with 95% confidence intervals for each outcome and utilized the Brier score to check the calibration and complication odds ratios above vs. below average risk in all score outcomes with the number of occurred events. In our study population, most patients were <65 years old (48%), predominantly male (63%), and overweight or obese (48%). In addition, 13% had severe chronic obstructive pulmonary disease (COPD), and 75% were categorized as American Society of Anesthesiologists (ASA) class III or higher. For "serious complication", AUROC was 59%, and events were equal in patients with above or below average risk scores (P=0.96). AUROC was 67% for "any complication" and 58% for "return to OR", expressing no useful predictive value. The Brier score and odds ratios were low for all outcomes. Dyspnea, ASA class, COPD, and body mass index as single postoperative risk predictors significantly improved the basic model consisting of the logit of the risk calculator alone. Thus, the calculator alone did not perform as well as these single variables did. The ACS NSQIP surgical risk calculator exhibited low sensitivity, specificity, and low AUROC for postoperative 30-day morbidity and mortality in our study cohort. Therefore, we think it cannot be considered as valid risk estimation tool for general thoracic surgery.