Effect of Varying the Case Mix on the Standardized Mortality Ratio and W Statistic: A Simulation Study

Abstract

To evaluate the validity of using the standardized mortality ratio (SMR) and the W statistic as risk-adjusted measures of hospital mortality to judge ICU performance. APACHE (acute physiology and chronic health evaluation) II data were collected prospectively from the surgical ICU (SICU) at a single institution using all adult admissions (n = 6806) over an 8-year period (excluding cardiac surgical patients, burn patients, and patients under 16 years of age). Using a computer simulation technique, virtual ICUs (VICUs) with mortality rates between 5% and 16% were constructed. After first dividing the original data set into deciles of risk, each VICU was constructed by randomly resampling between 10 and 680 patients from each decile. The SMR, W statistic, and Z statistic were calculated for 10,000 different case mixes. The SICU at a 450-bed teaching hospital. A group of 6,806 adult patient admissions, excluding cardiac surgical patients and burn patients. VICUs were created from a data set of actual patients treated at one institution in order to test the hypothesis that the SMR and W statistic would remain invariant when applied to subsets of patients from a single institution. Instead, the SMR and W statistic were found to be very sensitive to changes in case mix. The SMR and W statistic were linear functions of the simulated ICU mortality rate. This simulation demonstrates that the SMR and the W statistic based on APACHE II cannot be used to compare outcomes of ICUs. We have proposed a revision of the SMR that eliminates the effect of case mix and allows for more accurate comparisons of ICU performance.