Ordinal outcome analysis improves the detection of between-hospital differences in outcome

I E Ceyisakar,Ewout W Steyerberg,Diederik W J Dippel,H F Lingsma,N Van Leeuwen

doi:10.1186/s12874-020-01185-7

I E Ceyisakar, Ewout W Steyerberg + Show 3 more

Open Access

https://doi.org/10.1186/s12874-020-01185-7

Copy DOI

Abstract

BackgroundThere is a growing interest in assessment of the quality of hospital care, based on outcome measures. Many quality of care comparisons rely on binary outcomes, for example mortality rates. Due to low numbers, the observed differences in outcome are partly subject to chance. We aimed to quantify the gain in efficiency by ordinal instead of binary outcome analyses for hospital comparisons. We analyzed patients with traumatic brain injury (TBI) and stroke as examples.MethodsWe sampled patients from two trials. We simulated ordinal and dichotomous outcomes based on the modified Rankin Scale (stroke) and Glasgow Outcome Scale (TBI) in scenarios with and without true differences between hospitals in outcome. The potential efficiency gain of ordinal outcomes, analyzed with ordinal logistic regression, compared to dichotomous outcomes, analyzed with binary logistic regression was expressed as the possible reduction in sample size while keeping the same statistical power to detect outliers.ResultsIn the IMPACT study (9578 patients in 265 hospitals, mean number of patients per hospital = 36), the analysis of the ordinal scale rather than the dichotomized scale (‘unfavorable outcome’), allowed for up to 32% less patients in the analysis without a loss of power. In the PRACTISE trial (1657 patients in 12 hospitals, mean number of patients per hospital = 138), ordinal analysis allowed for 13% less patients. Compared to mortality, ordinal outcome analyses allowed for up to 37 to 63% less patients.ConclusionsOrdinal analyses provide the statistical power of substantially larger studies which have been analyzed with dichotomization of endpoints. We advise to exploit ordinal outcome measures for hospital comparisons, in order to increase efficiency in quality of care measurements.Trial registrationWe do not report the results of a health care intervention.

Highlights

There is a growing interest in assessment of the quality of hospital care, based on outcome measures
One of the most commonly used clinical measures is the mortality ratio (SMR) [8, 9], which has a variety of disadvantages and methodological issues when used as a quality of care measure [10,11,12]
The databases consisted of hospital data of patients with either traumatic brain injury (TBI) in the International Mission on Prognosis And Clinical Trial Design in Traumatic Brain Injury (IMPACT) study [22], and stroke patients in the PRomoting ACute Thrombolysis in Ischemic StrokE (PRACTISE) trial [23]

Summary

Introduction

There is a growing interest in assessment of the quality of hospital care, based on outcome measures. Many quality of care comparisons rely on binary outcomes, for example mortality rates. The observed differences in outcome are partly subject to chance. The observed differences in outcome between hospitals are often partly due to chance [4] and are only partly explained by actual differences in the quality of care [5]. Lack of power to detect differences between hospitals is a common problem for several clinically relevant outcome indicators. The main issue being that mortality is an especially rare outcome in many patient groups, leading to low power when trying to detect hospitals with aberrant outcomes [13]

Objectives

Methods

Results

Conclusion