Investigating associations between ICU level and quality of care in the Netherlands: reporting only SMRs is not the whole story.

Abstract

Dear Editor, With interest we read the paper by Kluge et al. [1] in which they compared standardized mortality ratios (SMRs) in different levels of intensive care in the Netherlands. They clearly showed that the SMR was equally low in all levels of intensive care, suggesting an overall good performance of the Dutch intensive care system [2]. This, however, does not necessarily mean that the different levels of intensive care perform the same and, therefore, they should not conclude that their finding ‘is in contrast with some earlier studies suggesting a volume–outcome relationship’. Kluge et al. [1] found very different case-mixes in the three levels of intensive care, with sicker patients being more prevalent in higher-level intensive care units (ICUs) than in lower-level ones. SMRs can be confounded by case-mix differences since the SMR is calculated using a logistic regression model, and a wellknown and previously reported serious problem of this model is that it underestimates mortality in the more sick patient and overestimates mortality in the less sick patient [2]. Furthermore, notwithstanding the fact that these authors did take 90-day mortality into account, the transfer of patients must have contaminated their findings. Thus, we suggest that at best Kluge et al. [1] can conclude that SMRs are similar in different levels of intensive care in the Netherlands when the data are not corrected for important differences in case-mix and not corrected for the effects of patients who were transferred from ICUs providing a lower level of care to those providing a higher level of care. Obviously, a case–control design study in which patient groups are matched and transferred patients excluded is preferred. As highlighted very recently, performance assessment requires better case-mix adjustment than is currently available in the APACHE IV model, especially given the growing role of outcomes in driving prospective payment patient referral and public reporting [3]. This is not to say that SMR is invalid per se. Indeed, large differences in SMRs should prompt discussions between ICUs as part of their quality improvement cycles. However, we regret that Kluge et al. [1] over-interpreted—or possibly misinterpreted—their findings, which should be corrected.