Assessing the predictive accuracy of the American College of Surgeons National Surgical Quality Improvement Project Surgical Risk Calculator in open ventral hernia repair

Abstract

BackgroundPreoperative surgical risk assessment continues to be a critical component of clinical decision-making. The ACS Universal Risk Calculator estimates risk for several outcomes based on individual risk profiles. Although this represents a tremendous step toward improving outcomes, studies have reported inaccuracies among certain patient populations. This study aimed to assess the predictive accuracy of the American College of Surgeons' (ACS) Risk Calculator in patients undergoing open ventral hernia repair (VHR). MethodsA review of patients undergoing open, isolated VHR between 7/1/2007 and 7/1/2014 by a single surgeon was performed. Risk factors and outcomes were collected as defined by National Surgical Quality Improvement Project. Thirty-day outcomes included serious complication, venous thromboembolism, medical morbidity, surgical site infection (SSI), unplanned reoperation, mortality, and length of stay (LOS). Patient profiles were entered into the ACS Surgical Risk Calculator and outcome-specific risk predictions recorded. Prediction accuracy was assessed using the Brier score and receiver-operator area under the curve (AUC). ResultsOne hundred forty-two patients undergoing open VHR were included. ACS predictions were accurate for cardiac complications (Brier = .02), venous thromboembolism (Brier = .08), reoperation (Brier = .10), and mortality (Brier = .01). Significantly, underestimated outcomes included SSI (Brier = .14), serious complication (Brier = .30), and any complication (Brier = .34). Discrimination ranged from highly accurate (mortality, AUC = .99) to indiscriminate (SSI, AUC = .57). Predicted LOS was 3-fold shorter than observed (2.4 vs 7.4 days, P <.001). ConclusionsThe ACS Surgical Risk Calculator accurately predicted medical complications, reoperation, and 30-day mortality. However, SSIs, serious complications, and LOS were significantly underestimated. These findings suggest that additional considerations are needed to better estimate complications after open VHR.