The bias of linear rank tests when testing for stochastic order in ordered categorical data

Abstract

In many hypothesis testing problems, the alternative hypothesis is characterized by one or several restrictions arising from a natural ordering among the outcome levels. It is known that tests which ignore the ordering may lack adequate statistical power for the alternatives which are of the most interest. Considerably less, however, is known about developing tests which conform to a natural ordering. Stochastic order is an objective and compelling characterization of the superiority of one treatment over another. Consequently, testing for stochastic order is of considerable importance in applications involving the comparison of one treatment to another on the basis of ordered categorical data (e.g., in clinical trials). With few exceptions (that we identify), there is no optimal test for this problem, so it is reasonable to consider the class of tests that are simultaneously unbiased and admissible. There are known necessary and sufficient conditions for admissibility, but unbiasedness is more elusive. It is known that a necessary condition for unbiasedness is exactness conditionally on the margins. Further, one can construct a test which is both admissible and unbiased by repeatedly improving the trivially unbiased “ignore-the-data” test. This complicated approach, however, does not, in general, lead to a nested family of tests. We provide a new necessary condition for unbiasedness, and show that linear rank tests, which are widely used and locally most powerful, fail this condition for certain sets of margins. For such margins, linear rank tests are severely biased (with power as low as zero to detect certain alternatives of interest) and least stringent.