An Efficient Alternative to Average Ranks for Testing with Incomplete Ranking Data

Abstract

In this paper we consider the setting where a group of n judges are to independently rank a series of k objects, but the intended complete rankings are not realized and we are faced with analyzing randomly incomplete ranking vectors. In this paper we propose a new testing procedure for dealing with such data realizations. We concentrate on the problem of testing for no differences in the objects being ranked (i.e., they are indistinguishable) against general alternatives, but our approach could easily be extended to restricted (e.g., ordered or umbrella) alternatives. Using an improvement of a preliminary screening approach previously proposed by the authors, we present an algorithm for computation of the relevant Friedman-type statistic in the general alternatives setting and present the results of an extensive simulation study comparing the new procedure with the standard approach of imputing average within-judge ranks to the unranked objects.