Predicting Adverse Outcomes After Complex Cancer Surgery

Abstract

Included in this issue of the Annals of Surgical Oncology is a comprehensive analysis of risk factors and outcomes in patients who underwent resection of thoracic and abdominal malignancies.1 The dataset that the authors chose to analyze is the deidentified Participant Use Data File (PUF) of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP). As the authors comment in their manuscript, this is a validated dataset compiled by trained nurse reviewers at more than 200 U.S. hospitals.2 There are three main strengths of this dataset. First, each of the multiple preoperative risk factors, intraoperative risk factors, and postoperative complications (termed “occurrences” in the ACS-NSQIP) are strictly defined, allowing consistency across institutions and personnel. Second, nurse reviewers complete a 30-day postoperative data collection, accounting for occurrences and deaths that occurred during the early postdischarge period. Third, the timeliness and size of the dataset (including 635,265 cases from 2005 to 2008) adds power to detailed outcomes analyses. The manuscript leverages each of these factors to comment on the ability to prognosticate complications after a mixed group of complex surgical oncology operations from pneumonectomy to pelvic exenteration. In their multivariate analyses, they identify multiple important risk factors for postoperative outcomes, such as older age, performance status, ASA score, serum albumin, and intraoperative transfusion. These findings validate multiple previous ACS-NSQIP risk models, indicating that the core set of risk factors for poor outcome in all general surgery patients also apply to patients who require complex cancer operations. The authors then rate the ability of their cancer procedure-specific multivariate models to account for all of the adverse outcomes that occurred. Using the c-index statistic as their metric, the authors find that, despite the power of the large number of cases analyzed, their models were only able to predict a portion of the complications and deaths that occurred. The important question then becomes “Why can’t we use the ACS-NSQIP database (arguably the largest, most complete, objective, and accurate compilation of surgical risk factors and outcomes) to identify the subset of risk factors that perfectly correlates with postoperative adverse outcome?” The authors offer several explanations for the discrepancy between the quality of the dataset and the imperfection of its predictive models. First, they argue that missing data confounded the predictive power of the models, citing the finding that 13% of the 398 patients recorded as 30-day postoperative mortalities had no recorded “major complications” (the subset of occurrences defined by the authors as “major complications” is detailed in the Methods section). Although we cannot comment on the exact subset of patients that the authors chose to analyze, by using a definition of major complications similar to theirs (which we have previously published3) an audit finds that 3,247 of the 635,265 cases (0.51%) in the PUF are recorded as 30-day mortalities without major complication. In 1,024 of these cases (31.5%) with apparent missing data, the death occurred after hospital discharge. In this scenario, a nurse reviewer may have a patient with no inpatient postoperative complications, who then has a sudden death event after discharge. Without medical input concerning the exact cause of death, this case would be coded as a 30-day mortality without major complication. Admittedly a potential weakness of the database, crucial missing data appears to be a rare event that should not adversely impact risk modeling. The second major reason that the authors use to explain the discrepancy between data quality and predictive ability is lack of definitions and recording of operation-specific outcomes. This point is well recognized and well taken. Given the current ACS-NSQIP occurrence definitions, complications, such as bile leak after liver resection and pancreatic fistula after pancreaticoduodenectomy, may be captured in such categories as deep space surgical site infection, but this is not certain. During the past 2 years, multiple ACS-NSQIP–engaged cancer surgeons have focused on this issue. One immediate remedy is that each ACS-NSQIP institution has a set of open data fields available to collect such specific data. As well, collaboration between ACS-NSQIP and subspecialty surgical societies is active to create consensus on definitions of such occurrences, facilitating uniform entry into the national data collection algorithm.4 To that end, the Society of Surgical Oncology membership has an opportunity to propose cancer-specific risk factors that likely contribute to adverse postoperative outcomes. For example, the ACS-NSQIP definition of “preoperative chemotherapy” is neoadjuvant treatment within 1 month of surgical resection. Based on this definition, a patient who received 6 cycles of FOLFOX with Avastin for colorectal liver metastases, whose therapy was discontinued 6 weeks before surgery to reduce the risk (perceived or real) of complications, would not be coded as having preoperative chemotherapy in the current ACS-NSQIP lexicon. Given that this patient’s risk for preoperative anemia, and therefore perioperative transfusion requirements, among several other adverse outcomes, is likely higher than a chemotherapy-naive patient,5 a more complete definition of the term “preoperative chemotherapy” may empower future ACS-NSQIP–derived risk models to better predict outcomes in cancer patients. Pathologic stage of cancer is another example of a cancer patient-specific variable that may impact perioperative outcomes but is not currently recorded in the ACS-NSQIP. One final point to consider is that when we speak about “major cancer surgery,” we immediately encounter two features that automatically add variability to any prognostic model. As the complexity of an operation increases, the prognostic weight of any one or any combination of preoperative risk factors is diminished. Likewise, cancer in and of itself, likely through effects on the immune system, increases the rate of apparently random events that occur during the perioperative period. These factors conspire to put a ceiling on the maximal c-index that can be derived from any risk model. This does not mean that we should stop trying to identify perioperative risk factors for adverse postoperative outcomes in cancer surgery; instead this presents an opportunity to refine surgical outcomes databases, such as the ACS-NSQIP PUF. This would allow us to better understand practice patterns and systems of care that are rigid enough to reliably manage the known risk factors and flexible enough to respond to the random unpredictable events.6 This hybrid ability is where the best final outcomes in major cancer surgery will be found.