Generalized Adjusted Rand Indices for cluster ensembles

Abstract

In this paper, Adjusted Rand Index (ARI) is generalized to two new measures based on matrix comparison: (i) Adjusted Rand Index between a similarity matrix and a cluster partition (ARImp), to evaluate the consistency of a set of clustering solutions with their corresponding consensus matrix in a cluster ensemble, and (ii) Adjusted Rand Index between similarity matrices (ARImm), to evaluate the consistency between two similarity matrices. Desirable properties of ARI are preserved in the two new measures, and new properties are discussed. These properties include: (i) detection of uncorrelatedness; (ii) computation of ARImp/ARImm in a distributed environment; and (iii) characterization of the degree of uncertainty of a consensus matrix. All of these properties are investigated from both the perspectives of theoretical analysis and experimental validation. We have also performed a number of experiments to show the usefulness and effectiveness of the two proposed measures in practical applications. ► We propose new evaluation measures which generalize the Adjusted Rand Index. ► Desirable properties of ARI are preserved in the new measures. ► These measures are applied to evaluate different cluster ensemble approaches. ► We study their properties based on theoretical analysis and simulations. ► We also present a number of practical applications based on these measures.