Improving the quality of clustering using cluster ensembles

Abstract

Clustering is a very widely used data mining task practiced to partition data in to similar groups/clusters. Data clustering can be challenging, when it has to be done for huge data sets as no single clustering algorithm proves to give optimal results. Clustering ensembles is an emerging solution to the above issue for improving robustness, stability and accuracy of unsupervised clustering. The clustering ensembles combine multiple partitions generated by different clustering algorithms into a single clustering solution. One of the major problems in clustering ensembles is in deriving the appropriate consensus function. The focus of this project is to implement cluster ensemble algorithm to improve the accuracy and efficiency of clustering. The first method used in this work is Co-association method where pair wise comparison is made and weight factor decides the data labelling. Second method is Normalized Mutual information where information shared between two clusters are measured to aggregate the clusters. It's observed that there is 20% increase in accuracy when cluster ensemble method was used.