Abstract
When a ranking of institutions such as medical centers or universities is based on a numerical measure of performance provided with a standard error, confidence intervals (CIs) should be calculated to assess the uncertainty of these ranks. We present a novel method based on Tukey's honest significant difference test to construct simultaneous CIs for the true ranks. When all the true performances are equal, the probability of coverage of our method attains the nominal level. In case the true performance measures have no exact ties, our method is conservative. For this situation, we propose a rescaling method to the nominal level that results in shorter CIs while keeping control of the simultaneous coverage. We also show that a similar rescaling can be applied to correct a recently proposed Monte-Carlo based method, which is anticonservative. After rescaling, the two methods perform very similarly. However, the rescaling of the Monte-Carlo based method is computationally much more demanding and becomes infeasible when the number of institutions is larger than 30-50. We discuss another recently proposed method similar to ours based on simultaneous CIs for the true performance. We show that our method provides uniformly shorter CIs for the same confidence level. We illustrate the superiority of our new methods with a data analysis for travel time to work in the United States and on rankings of 64 hospitals in theNetherlands.
Highlights
Estimation of ranks is an important statistical problem which appears in many applications in healthcare, education and social services [Goldstein and Spiegelhalter, 1996] to compare the performance of medical centers, universities or more generally institutions
We show that Tukey’s honest significant difference test (HSD) can be used to produce simultaneous confidence intervals for ranks with simultaneous coverage of at least 1 − α
Assume that we found the following 90% confidence intervals for the differences from Tukey’s HSD μA − μB ∈ [−2, −1], μA − μC ∈ [−3, −2] μB − μA ∈ [1, 2], μB − μC ∈ [−1, 1] μC − μA ∈ [2, 3], μC − μB ∈ [−1, 1]
Summary
Estimation of ranks is an important statistical problem which appears in many applications in healthcare, education and social services [Goldstein and Spiegelhalter, 1996] to compare the performance of medical centers, universities or more generally institutions. This paper presents a method to produce simultaneous CIs at a prespecified joint level 1 − α for the ranks with correct coverage of the true ranks. We mention funnel plots, see Tekkis et al [2003], Spiegelhalter [2005] among others These latter two approaches have been considered in comparing institutions, they do not aim to build (simultaneous) CIs for ranks. Holm [2012] (see Bie [2013]) calculated a Z-score, but he applied Holm’s sequential algorithm to correct for multiple comparisons on the institution level, that is for each institution he corrects for comparisons with other institutions This is only sufficient if we are interested in one of the institutions, but it is not sufficient to produce simultaneous confidence intervals for the ranks of the institutions. Software for the methods presented in this paper is available in package ICRanks downloadable from CRAN
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
