High-mortality Outliers Research Articles

Comprehensive analysis of combat casualty outcomes in US service members from the beginning of World War II to the end of Operation Enduring Freedom.

Advances in combat casualty care have improved combat survivability over the past two decades. However, these outcomes remain incompletely framed in the broader context of combat casualty outcomes over the past eighty years. We hypothesized that starting with World War II, combat survival worsened at the beginning of each new conflict but then improved over time. To evaluate long-term trends in combat casualty outcomes, monthly combat injuries and deaths during World War II, the Korean conflict, the Vietnam conflict, Operation Iraqi Freedom (OIF), and Operation Enduring Freedom (OEF) were collated. From these numbers, we calculated the monthly case fatality rate (CFR), the killed in action rate (%KIA), and the died of wounds rate (%DOW). We analyzed these metrics for significant trends during and between each conflict using linear and Loess regression. We then simulated alternate outcome scenarios by eliminating outcome variability. In this comprehensive analysis, CFR decreased over the study period in parallel with a decrease in %KIA. When examining individual conflicts, however, several unfavorable trends emerged including a spike in all fatality measures at the end of Vietnam and a rise in %DOW over the course of Korea and OIF. In comparing CFR at the beginning of each conflict to the best CFR from the prior conflict, high mortality outliers occurred in every conflict after a period of relative peace, and a clear peacetime effect occurred in both World War II and Vietnam. Eliminating these negative trends and the attendant preventable deaths would have reduced combat fatalities over the course of eighty years by 107,256 (39.7%). In summary, although combat mortality rates have generally improved since World War II, closer examination indicates several unfavorable trends both during and between conflicts. Identifying factors behind these trends will reveal further opportunities to improve combat casualty outcomes in the future. LEVEL OF EVIDENCE: III, Epidemiological.

Hospital Variation in Geriatric Surgical Safety for Emergency Operation

The American College of Surgeons maintains that surgical care in the US has not reached optimal safety and quality. This can be driven partially by higher-risk, emergency operations in geriatric patients. We therefore sought to answer 2 questions: First, to what degree does standardized postoperative mortality vary in hospitals performing nonelective operations in geriatric patients? Second, can the differences in hospital-based mortality be explained by patient-, operative-, and hospital-level characteristics among outlier institutions? Patients 65 years and older who underwent 1 of 8 common emergency general surgery operations were identified using the California State Inpatient Database (2010 to 2011). Expected mortality was obtained from hierarchical, Bayesian mixed-effects logistic regression models. A risk-adjusted hospital-level standardized mortality ratio (SMR) was calculated from observed-to-expected in-hospital deaths. "Outlier" hospitals had an SMR 80% CI that did not cross the mean SMR of 1.0. High-mortality (SMR >1.0) and low-mortality (SMR <1.0) outliers were compared. We included 24,207 patients from 107 hospitals. SMRs varied widely, from 2.3 (highest) to 0.3 (lowest). Eleven hospitals (10.3%) were poor-performing high-SMR outliers, and 10 hospitals (9.3%) were exceptional-performing low-SMR outliers. SMR was 3 times worse in the high-SMR compared with the low-SMR group (1.7 vs 0.6; p < 0.001). Patient-, operation-, and hospital-level characteristics were equivalent among outlier-hospitals. Significant hospital variation exists in standardized mortality after common general surgery operations done emergently in older patients. More than 10% of institutions have substantial excess mortality. These findings confirm that the safety of emergency operation in geriatric patients can be significantly improved by decreasing the wide variability in mortality outcomes.

The value of failure to rescue in determining hospital quality for pediatric trauma

In adult trauma patients, high- and low-mortality trauma hospitals have similar rates of major complications but differ based on failure to rescue (mortality following a major complication), which has become a marker of hospital quality. The aim of this study is to examine whether failure to rescue is also an appropriate hospital quality indicator in pediatric trauma. Children younger than 15 years were identified in the 2007 to 2014 National Trauma Databank research data sets. Hospitals were classified as a high, average or low mortality based on risk-adjusted observed-to-expected in-hospital mortality ratios using the modified Trauma Mortality Probability Model. Regression modeling was used to explore the impact of hospital quality ranking on the incidence of major complications and failure to rescue. Of 125,057 children, 31,600 were treated at low-mortality outlier hospitals, and 7,014 at high-mortality outlier hospitals. Low-mortality hospitals had a lower rate of major complications compared with high-mortality hospitals (0.5% [low] vs. 0.8% [high]; adjusted odds ratio [OR], 0.71; 95% confidence interval [CI], 0.61-0.83; p < 0.01) and a lower failure-to-rescue rate (17.6% [low] vs. 24.1% [high]; adjusted OR, 0.53 [high; 95% CI 0.34-0.83; p < 0.01]). When patients who died within 48 hours were excluded, low-mortality hospitals had a lower complication rate (OR, 0.81; 95% CI, 0.68, 0.96; p = 0.02), but similar failure-to-rescue rate compared to high-mortality hospitals. There was no correlation between trauma verification level and hospital mortality status based on the model. For pediatric trauma patients, mortality is more strongly associated with major complication rate than with failure to rescue. Thus, failure to rescue does not appear to be the key driver of hospital quality in this population as it does in the adult trauma population. Prognostic and epidemiological, level III.

Observed to expected or logistic regression to identify hospitals with high or low 30-day mortality?

IntroductionA common quality indicator for monitoring and comparing hospitals is based on death within 30 days of admission. An important use is to determine whether a hospital has higher or lower mortality than other hospitals. Thus, the ability to identify such outliers correctly is essential. Two approaches for detection are: 1) calculating the ratio of observed to expected number of deaths (OE) per hospital and 2) including all hospitals in a logistic regression (LR) comparing each hospital to a form of average over all hospitals. The aim of this study was to compare OE and LR with respect to correctly identifying 30-day mortality outliers. Modifications of the methods, i.e., variance corrected approach of OE (OE-Faris), bias corrected LR (LR-Firth), and trimmed mean variants of LR and LR-Firth were also studied.Materials and methodsTo study the properties of OE and LR and their variants, we performed a simulation study by generating patient data from hospitals with known outlier status (low mortality, high mortality, non-outlier). Data from simulated scenarios with varying number of hospitals, hospital volume, and mortality outlier status, were analysed by the different methods and compared by level of significance (ability to falsely claim an outlier) and power (ability to reveal an outlier). Moreover, administrative data for patients with acute myocardial infarction (AMI), stroke, and hip fracture from Norwegian hospitals for 2012–2014 were analysed.ResultsNone of the methods achieved the nominal (test) level of significance for both low and high mortality outliers. For low mortality outliers, the levels of significance were increased four- to fivefold for OE and OE-Faris. For high mortality outliers, OE and OE-Faris, LR 25% trimmed and LR-Firth 10% and 25% trimmed maintained approximately the nominal level. The methods agreed with respect to outlier status for 94.1% of the AMI hospitals, 98.0% of the stroke, and 97.8% of the hip fracture hospitals.ConclusionWe recommend, on the balance, LR-Firth 10% or 25% trimmed for detection of both low and high mortality outliers.

Association of hospital structures with mortality from ruptured abdominal aortic aneurysm.

There is significant variation in the mortality rates of patients with a ruptured abdominal aortic aneurysm (rAAA) admitted to hospital in England. This study sought to investigate whether modifiable differences in hospital structures and processes were associated with differences in patient outcome. Patients diagnosed with rAAA between 2005 and 2010 were extracted from the Hospital Episode Statistics database. After risk adjustment, hospitals were grouped into low-mortality outlier, expected mortality and high-mortality outlier categories. Hospital Trust-level structure and process variables were compared between categories, and tested for an association with risk-adjusted 90-day mortality and non-corrective treatment (palliation) rate using binary logistic regression models. There were 9877 patients admitted to 153 English NHS Trusts with an rAAA during the study. The overall combined (operative and non-operative) mortality rate was 67·5 per cent (palliation rate 41·6 per cent). Seven hospital Trusts (4·6 per cent) were high-mortality and 15 (9·8 per cent) were low-mortality outliers. Low-mortality outliers used significantly greater mean resources per bed (doctors: 0·922 versus 0·513, P < 0·001; consultant doctors: 0·316 versus 0·168, P < 0·001; nurses: 2·341 versus 1·770, P < 0·001; critical care beds: 0·045 versus 0·019, P < 0·001; operating theatres: 0·027 versus 0·019, P = 0·002) and performed more fluoroscopies (mean 12·6 versus 9·2 per bed; P = 0·046) than high-mortality outlier hospital Trusts. On multivariable analysis, greater numbers of consultants, nurses and fluoroscopies, teaching status, weekday admission and rAAA volume were independent predictors of lower mortality and, excluding rAAA volume, a lower rate of palliation. The variability in rAAA outcome in English National Health Service hospital Trusts is associated with modifiable hospital resources. Such information should be used to inform any proposed quality improvement programme surrounding rAAA.

American College of Surgeons trauma center verification versus state designation

Single-center experience has shown that American College of Surgeons (ACS) trauma verification can improve outcomes. The current objective was to compare mortality between ACS-verified and state-designated centers in a national sample. Subjects 16 years or older from ACS-verified or state-designated Level I and II centers were identified in the National Trauma Databank 2007 to 2008. A predictive mortality model was constructed using Trauma Quality Improvement Project methodology. Imputation was used for missing data. Probability of mortality in the model determined expected deaths. Observed-to-expected (O/E) mortality ratios with 90% confidence interval (CI) and outliers (90% CI more than or less than 1.0) were compared across ACS and state Level I and II centers. The mortality model was repeated with ACS versus state included. There were 900,274 subjects. The model had an area under the curve of 0.92 to predict death. Level I ACS centers had a lower median O/E ratio compared with state centers (0.95 [interquartile range, 0.82-1.05] vs. 1.02 [interquartile range, 0.87-1.15]; p < 0.01), with no difference in Level II centers. Level II state centers had more high O/E outliers. ACS verification was an independent predictor of survival in Level II centers (odds ratio, 1.26; 95% CI, 1.20-1.32; p < 0.01) but not in Level I centers (p = 0.84). Level II centers have a disproportionate number of high mortality outliers, and ACS verification is a predictor of survival. Level I ACS centers have lower O/E ratios overall, but no difference in outliers. ACS verification seems beneficial. These data suggest that Level II centers benefit most, and promoting Level II ACS verification may be an opportunity for improved outcomes. Prognostic study, level III.

Hospital performance reports based on severity adjusted mortality rates in patients with cirrhosis depend on the method of risk adjustment

Background. Hospital outcome report cards are used to judge provider performance, including for liver transplantation. We aimed to determine the impact of the choice of risk adjustment method on hospital rankings based on mortality rates in cirrhotic patients.Material and methods. We identified 68,426 cirrho-tic patients hospitalized in the Nationwide Inpatient Sample database. Four risk adjustment methods (the Charlson/Deyo and Elixhauser algorithms, Disease Staging, and All Patient Refined Diagnosis Related Groups) were used in logistic regression models for mortality. Observed to expected (O/E) death rates were calculated for each method and hospital. Statistical outliers with higher or lower than expected mortality were identified and rankings compared across methods.Results. Unadjusted mortality rates for the 553 hospitals ranged from 1.4 to 30% (overall, 10.6%). For 163 hospitals (29.5%), observed mortality differed significantly from expected when judged by one or more, but not all four, risk adjustment methods (25.9% higher than expected mortality and 3.6% lower than expected mortality). Only 28% of poor performers and 10% of superior performers were consistently ranked as such by all methods. Agreement between methods as to whether hospitals were flagged as outliers was moderate (kappa 0.51-0.59), except the Charlson/Deyo and Elixhauser algorithms which demonstrated excellent agreement (kappa 0.75).Conclusions. Hospital performance reports for patients with cirrhosis require sensitivity to the method of risk adjustment. Depending upon the method, up to 30% of hospitals may be flagged as outliers by one, but not all methods. These discrepancies could have important implications for centers erroneously labeled as high mortality outliers.

Single measures of performance do not reflect overall institutional quality in colorectal cancer surgery

ObjectiveTo evaluate overall performance of English colorectal cancer surgical units identified as outliers for a single quality measure—30 day inhospital mortality.Design144 542 patients that underwent primary major colorectal cancer resection...

Impact of Public Reporting of Coronary Artery Bypass Graft Surgery Performance Data on Market Share, Mortality, and Patient Selection

The impact of publicly reporting risk-adjusted outcomes for hospitals and surgeons remains controversial, with particular concern about unintended consequences. We evaluated the impact of 3 reports from the voluntary California CABG Mortality Reporting Program (CCMRP) on hospital market share, hospital mortality, and patient selection for coronary artery bypass graft (CABG) surgery. We analyzed data from January 2000 to December 2005 for all patients receiving isolated CABG surgery in California. We compared hospital groups based on their quality classification, including low-mortality outliers ("better"), high-mortality outliers ("worse"), and nonoutliers, as well as participation in the CCMRP. We compared changes in market share, risk-adjusted mortality, and hospital caseload of high-risk patients for isolated CABG surgeries before and after the public release of 3 CCMRP reports (July 2001, August 2003, and February 2005). Low-mortality outlier hospitals experienced significantly increased market share for isolated CABG surgery in the first 6 months after the public release of the CCMRP reports (relative change in adjusted mean market share=8.9%, P=0.002). We found no evidence to suggest reduced risk adjusted mortality after the release of the CCMRP reports, but high-mortality outlier hospitals, on average, operated on less sick patients (relative change in mean expected mortality=25%, P=0.02). The release of public CABG hospital performance reports in California was associated with increased volume at low-mortality hospitals, and may have reduced referrals of high-risk patients to high-mortality hospitals (or risk avoidance).

Mortality After Cardiac Bypass Surgery

Risk-adjusted outcome rates frequently are used to make inferences about hospital quality of care. We calculated risk-adjusted mortality rates in veterans undergoing isolated coronary artery bypass surgery (CABS) from administrative data and from chart-based clinical data and compared the assessment of hospital high and low outlier status for mortality that results from these 2 data sources. We studied veterans who underwent CABS in 43 VA hospitals between October 1, 1993, and March 30, 1996 (n=15,288). To evaluate administrative data, we entered 6 groups of International Classification of Diseases (ICD)-9-CM codes for comorbid diagnoses from the VA Patient Treatment File (PTF) into a logistic regression model predicting postoperative mortality. We also evaluated counts of comorbid ICD-9-CM codes within each group, along with 3 common principal diagnoses, weekend admission or surgery, major procedures associated with CABS, and demographic variables. Data from the VA Continuous Improvement in Cardiac Surgery Program (CICSP) were used to create a separate clinical model predicting postoperative mortality. For each hospital, an observed-to-expected (O/E) ratio of mortality was calculated from (1) the PTF model and (2) the CICSP model. We defined outlier status as an O/E ratio outside of 1.0 (based on the hospital's 90% confidence interval). To improve the statistical and predictive power of the PTF model, selected clinical variables from CICSP were added to it and outlier status reassessed. Significant predictors of postoperative mortality in the PTF model included 1 group of comorbid ICD-9-CM codes, intraortic balloon pump insertion before CABS, angioplasty on the day of or before CABS, weekend surgery, and a principal diagnosis of other forms of ischemic heart disease. The model's c-index was 0.698. As expected, the CICSP model's predictive power was significantly greater than that of the administrative model (c=0.761). The addition of just 2 CICSP variables to the PTF model improved its predictive power (c=0.741). This model identified 5 of 6 high mortality outliers identified by the CICSP model. Additional CICSP variables were statistically significant predictors but did not improve the assessment of high outlier status. Models using administrative data to predict postoperative mortality can be improved with the addition of a very small number of clinical variables. Limited clinical improvements of administrative data may make it suitable for use in quality improvement efforts.

Do well-publicized risk-adjusted outcomes reports affect hospital volume?

Report cards on hospital performance are common but have uncertain impact. The objective of this study was to determine whether hospitals recognized as performance outliers experience volume changes after publication of a report card. Secondary objectives were to test whether favorable outliers attract more patients with related conditions, or from outside their catchment areas; and whether disadvantaged groups are less responsive to report cards. We used a time-series analysis using linear and autoregressive models. We studied patients admitted to nonfederal hospitals designated as outliers in reports on coronary bypass surgery (CABG) mortality in New York, acute myocardial infarction (AMI) mortality in California, and postdiskectomy complications in California. We studied observed versus expected hospital volume for topic and related conditions and procedures, by month/quarter after a report card, with and without stratification by age, race/ethnicity, insurance, and catchment area. Potential confounders included statewide prevalence, prereport hospital volume and market share, and unrelated volume. In California, low-mortality and high-mortality outliers did not experience changes in AMI volume after adjusting for autocorrelation. Low-complication outliers for lumbar diskectomy experienced slightly increased volume in autoregressive models. No other cohorts demonstrated consistent trends. In New York, low-mortality outliers experienced significantly increased CABG volume in the first month after publication, whereas high-mortality outliers experienced decreased volume in the second month. The strongest effects were among white patients and those with HMO coverage in California, and among white or other patients and those with Medicare in New York. Volume effects were modest, transient, and largely limited to white Medicare patients in New York.

Procedural volume as a marker of quality for cabg surgery

THE ASSOCIATION BETWEEN HOSpitals’ coronary artery bypass graft (CABG) surgery volume and outcome has been the subject of multiple investigations. Based on these studies, the Center for Medicare & Medicaid Services (CMS) and the Leapfrog Group have proposed using hospital volume as an indicator of CABG quality. Prior studies of the association between volume and outcome, however, generally have been based on administrative data sources, have reflected selected patient populations, and often have not adequately accounted for patient case mix, patient clustering, and other methodological concerns. Before hospital surgery volume is accepted as a standard quality metric, further research into issues related to both analysis and policy implication is warranted. We undertook a contemporary examination of the association between hospital CABG procedural volume and outcome using clinical data available from the Society of Thoracic Surgeons (STS) National Cardiac Database. Specifically, we considered whether hospital CABG volume was associated with operative mortality after accounting for patient case mix. Second, we examined the extent to which patient clustering within centers and site variance issues affected this association. Third, we determined how the association varied as a function of patient age and predicted surgical risk. Fourth, we determined whether the association between hospital volume and outcome was influenced by individual surgeon volume. Finally, we investigated the potential health policy implications of using hospital volume as a quality indicator. This included determining the ability of hospital volume to discriminate high-mortality outliers, as well as investigating the potential number of lives saved if low-volume centers were systematically closed. METHODS Data Source The STS National Cardiac Database was established in 1989 to report surgical outcomes following cardiothoracic surgical procedures. The database cur-

Analyzing hospital mortality. The consequences of diversity in patient mix

Consumers and payers increasingly demand data with which to evaluate health care providers. While publication of risk-adjusted hospital-specific death rates is one response, debate continues over whether higher than predicted mortality is a warning about quality of care or rather a reflection of a hospital's atypical patient population. To help inform this debate, we compared the characteristics of Medicare patients discharged from 187 hospitals that the Health Care Financing Administration (HCFA) had labeled "high-mortality outliers" with those of Medicare patients from 5373 hospitals not so designated. Hospitals were most likely to be flagged as high-mortality outliers by HCFA when they had large shares of very elderly patients (age greater than or equal to 85 years), patients with high-risk diagnoses, or patients requiring nursing home care. After adjustments were made to compensate for these biases, nearly half the hospitals flagged as outliers by HCFA were no longer so designated. Statistics purporting to measure effectiveness of care from hospital death rates should be modified to account for diversity in patient mix.

Analyzing Hospital Mortality

Consumers and payers increasingly demand data with which to evaluate health care providers. While publication of risk-adjusted hospital-specific death rates is one response, debate continues over whether higher than predicted mortality is a warning about quality of care or rather a reflection of a hospital's atypical patient population. To help inform this debate, we compared the characteristics of Medicare patients discharged from 187 hospitals that the Health Care Financing Administration (HCFA) had labeled "high-mortality outliers" with those of Medicare patients from 5373 hospitals not so designated. Hospitals were most likely to be flagged as high-mortality outliers by HCFA when they had large shares of very elderly patients (age ≥85 years), patients with high-risk diagnoses, or patients requiring nursing home care. After adjustments were made to compensate for these biases, nearly half the hospitals flagged as outliers by HCFA were no longer so designated. Statistics purporting to measure effectiveness of care from hospital death rates should be modified to account for diversity in patient mix. (<i>JAMA</i>. 1991;265:1849-1853)

Flaws in Mortality Data

Recent public releases of hospital mortality data have sparked debate over methods to identify poor-quality care. We examined variations among hospitals in patient characteristics known independently to affect the risk of adverse outcomes and focused on patient comorbidity, defined as the state of health at admission apart from the primary diagnosis. Data from a study of 2935 incident cancer patients treated in seven Southern California hospitals revealed substantial variations among hospitals in age, cancer stage, and the burden of comorbid conditions. In the highest-ranked hospital, 17.9% of patients had high levels of comorbidity, compared with 9.3% in the lowest-ranked hospital. The three hospitals with the highest comorbidity were also identified as high-mortality outliers in a recent California report on hospital mortality rates. We conclude that comorbidity must be considered in any hospital quality assessment method based on patient outcome. If it is not considered, variations in referral and admission patterns may be misinterpreted as differences in hospital quality.