Risk Prediction Accuracy Differs for Transferred and Nontransferred Emergency General Surgery Cases in the ACS-NSQIP

Abstract

BackgroundRisk prediction accuracy of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) Surgical Risk Calculator has been shown to differ between emergency and elective surgery. Benchmarking methods of clinical performance require accurate risk estimation, and current methods rarely account for admission source; therefore, our goal was to assess whether the ACS-NSQIP predicts mortality comparably between transferred and nontransferred emergency general surgery (EGS) cases. Materials and methodsThis is a retrospective study using the ACS-NSQIP database from 2005 to 2014including all inpatients who underwent one of seven previously described EGS procedures. The admission source was classified as directly admitted versus transferred from an outside emergency room or an acute care facility. We compared the accuracy of ACS-NSQIP–predicted mortality probabilities using the observed-to-expected (O:E) ratio and Brier score. A subgroup analysis was performed to compare accuracy of high-risk and low-risk procedures. ResultsA total of 206,103 EGS admissions were identified, of which 6.97% were transfers. Overall mortality was 3.26% for the entire cohort and 10.24% within the transfer group. The O:E ratios generated by ACS-NSQIP models differed between transferred patients (O:E = 1.0, 95% confidence interval = 0.97-1.02) and nontransferred patients (O:E = 1.12, 95% confidence interval = 1.09-1.14). The Brier score for transferred patients was greater than that for nontransferred patients (0.063 versus 0.018, respectively) showing higher accuracy for nontransferred patients. ConclusionsThe ACS-NSQIP risk estimates used for benchmarking differ between transferred and nontransferred EGS cases. Analyses of the Brier score by the ACS-NSQIP risk calculator demonstrated inferior prediction for transferred patients. This increased burden on accepting institutions will have an impact on quality metrics and should be considered for benchmarking of clinical performance.