Parallel hierarchical architectures for efficient consensus clustering on big multimedia cluster ensembles

Abstract

Consensus clustering is a useful tool for robust or distributed clustering applications. However, given the fact that time complexities of the consensus functions scale linearly or quadratically with the number of combined clusterings, execution can be slow or even impossible when operating on big cluster ensembles, a situation encountered when we pursue robust multimedia data clustering. This work introduces hierarchical consensus architectures, an inherently parallel approach based on the divide-and-conquer strategy for computationally efficient consensus clustering, in a bid to make faster, more effective consensus clustering possible in big multimedia cluster ensemble scenarios. Moreover, we define a specific implementation of hierarchical architectures, including a theoretical analysis of its fully parallel implementation computational complexity. In experiments conducted on unimodal and multimedia data sets involving small and big cluster ensembles, we find parallel hierarchical consensus architectures variants perform faster than traditional flat consensus in 75% of the experiments on small cluster ensembles, a percentage that rises to 100% on unimodal and multimedia big cluster ensembles, achieving an average speedup ratio of 30.5. Moreover, depending on the consensus function employed, the quality of the obtained consensus partitions ensures robust clustering results.