ON THE EQUIVALENCE OF SOME INDICES OF SIMILARITY: IMPLICATION FOR BINARY PRESENCE/ABSENCE DATA

Abstract

Summary Cohen’s kappa, a special case of the weighted kappa, is a chance-corrected index used extensively to quantify inter-rater agreement in validation and reliability studies. In this paper, it is shown that in inter-rater agreement for 2 × 2 tables, for two raters having the same number of opposite ratings, the weighted kappa, Cohen’s kappa, Peirce, Yule, Maxwell and Pilliner and Fleiss indices are identical. This implies that the weights in the weighted kappa are less important under such assumptions. Equivalently, it is shown that for two partitions of the same data set, resulting from two clustering algorithms having the same number of clusters with equal cluster sizes, these similarity indices are identical. Hence, an important characterisation is formulated relating equal numbers of clusters with the same cluster sizes to the presence/absence of a trait in a reliability study. Two numerical examples that exemplify the implication of this relationship are presented.