COMPARISON OF CARDIAC SURGERY MORTALITY REPORTS USING RISK ADJUSTMENTS FROM ADMINISTRATIVE VERSUS CLINICAL DATA SOURCES

Abstract

Quality of care outcomes for cardiac surgery are increasingly being publically reported. These reports employ regression models and use data from either administrative sources or clinical registries for risk-adjustment and to account for differences in case mix. However, the impact of the type of data used in risk-adjustment models on accuracy of outcomes reporting and perceived performance of a given operator has not been fully examined. In the same patient cohort, we compared the performance of two different risk-adjustment models using administrative or clinical data sources. The administrative-only risk-adjustment model which uses the International Classification of Diseases diagnosis codes (CIHI Cardiac Care Quality Indicator [CCQI]) and the Society of Thoracic Surgeons [STS] risk-adjustment model using clinical data were tested on patients undergoing cardiac surgery at a single centre between 2013-2016 (N=1,635). Comparing both risk-adjustment models, the STS model included more data on cardiac function/status prior to the procedure and clinical cardiovascular conditions while the CIHI model included more variables pertaining to non-cardiac comorbidities. The primary outcome was in-hospital mortality within 30 days of the operation. Model performance was established by comparing predicted and observed mortality, model calibration and handling of critical covariates. Observed mortality rate was 1.96% (95%CI: 1.40-2.75%) which was similar to STS predicted mortality (1.96%) but significantly higher than CCQI-predicted mortality (1.03%). Despite both models having similar c-statistics (0.756 CCQI; 0.758 STS), the CCQI model showed significant underestimation of probability of mortality at the higher end of the risk spectrum. There was significant miscalibration (underestimation in the CCQI model) of risk, which was largely driven by seven important pre-operative covariates: NYHA class IV; prior congestive heart failure; left ventricular ejection fraction <20%, prior atrial fibrillation; acute coronary insufficiency; cardiac compromise (defined by the presence of shock, myocardial infarction <24 hours, intra-aortic balloon pump in situ, cardiac resuscitation prior to surgery, or pre-procedure circulatory support); and creatinine concentration ≥100mg/dL. Together, these factors accounted for 84% of the variation in predicted mortality between the CCQI and STS models. Risk prediction using administrative data underestimated mortality risk compared to a validated clinical model. The use of administrative data sources for risk-adjusted mortality reporting may inflate observed to predicted mortality ratios at hospitals with patients who are more ill, more complex, or are at the higher end of the clinical risk spectrum. Caution is warranted when hospital outcomes reports of cardiac surgery are based on administrative data alone.