A new nonparametric approach to the problem of agreement between two groups of judges

Abstract

The problem of agreement between two or more groups of judges has received considerable attention during the past decade, beginning with the papers by Schucany & Frawley(1973) and Hollander & Sethuraman(1978). Two very different definitions of agreement have emerged in the literature. The first states that two groups of judges agree if they use the same probability distribution to select their rank vectors, while the second defines agreement in terms of corre-lation coefficients. We utilize the first definition in developing a new approach to solving this agreement problem, relying heavily on computer-generated tables.