Unsupervised evaluation of cluster ensemble solutions

Abstract

Recently, a novel family of unsupervised learning techniques, which is referred to as cluster ensemble, attracts great interest from computational intelligence communities. Cluster ensemble techniques combine multiple individual clustering solutions into a consensus one, and can provide more robust and frequently more accurate partitions when comparing to individual clustering methods. However, although a number of cluster ensemble solution methods have been proposed, the selection of suitable cluster ensemble methods for specific data in an unsupervised manner is still an open problem. This problem becomes more critical before the phase of cluster quality evaluation since there is no group truth information at hand. Cluster ensemble solutions chosen from specific data at random thus could be subjective and moreover probably unsuitable. In view of these problems, in this paper, we propose a new unsupervised evaluation for different cluster ensemble methods based on the consensus affinity of cluster ensembles. Benefiting from the consensus affinity of a cluster ensemble, our proposed approach provides significant improvement beyond the average level of investigated cluster ensemble solution methods. We also propose to adopt our approach for partition selection. Studies with experimental validation shows the effectiveness of our proposed approach.