Composite Measures for Profiling Hospitals on Surgical Morbidity

Abstract

Although risk-adjusted morbidity is widely used as a surgical quality indicator, it may not always be a reliable indicator of hospital quality. In this study, we assess the value of a novel composite measure for improving the reliability of hospital morbidity rankings. Using data from the American College of Surgeons' National Surgical Quality Improvement Program (ACS-NSQIP), we studied all patients undergoing 4 surgical procedures (2008-2009): colectomy, ventral hernia repair, abdominal aortic aneurysm repair, and lower extremity bypass surgery. For these procedures, we created a composite measure by combining quality indicators from several distinct domains of quality: morbidity, reoperation, length of stay, and morbidity with other potentially related procedures. We empirically weighted each measure and adjusted for reliability using empirical Bayes techniques. To validate this approach, we assessed how well composite measures from 1 year (2008) predict morbidity in the next year (2009) compared with the standard ACS-NSQIP approach for assessing hospital rates of risk-adjusted morbidity. For all 4 operations, the composite measures explained a higher proportion of hospital-level variation in morbidity than the standard approach: ventral hernia repair (58% for the composite vs 8% for the standard approach), colon resection (33% vs 14%), abdominal aortic aneurysm repair (51% vs 38%), and lower extremity bypass surgery (32% vs 3%). When evaluating the ability to discriminate future performance, the composite approach performed best for ventral hernia repair. For this procedure, the bottom 20% of hospitals based on the composite approach had nearly threefold higher (odds ratio: 2.65; 95% confidence interval: 1.83-3.85) morbidity rates than the top 20% of hospitals. However, when using the standard approach, there was only a 1.3-fold difference (odds ratio: 1.30; 95% confidence interval: 0.87-1.96). Although the differences were smaller in magnitude, the composite measure also outperformed the standard approach for the other 3 procedures. Composite measures better reflect hospital quality than simple rates of risk-adjusted morbidity. In the context of ACS-NSQIP, composite measures would give hospitals a better sense of where they stand and help identify truly exemplary hospitals for benchmarking.