Prevalence-free utility-respecting summary indices of diagnostic power do not exist

Abstract

One would like to have a ranking method for diagnostic tests that justifiably makes no assumptions about the local 'disease mix', that is, about the pre-test probabilities of the diagnostic alternatives. Assuming the simplest possible clinical context we therefore examine the following question: can a summary index of diagnostic test performance be devised which is loyal to the expected utility paradigm and, at the same time, is incidence-free (prevalence-free)? In other words, regardless of pre-test probabilities, the index should rank rival tests in a way which agrees with how they might be ranked by expected utility calculations. From a practical point of view, the answer is negative; it is proved that such indices do exist, but they necessarily impose a very restrictive mathematical form on the utility framework and, what is worse, invariably go against what is medically appropriate by violating a boundedness requirement.