Clustering Large Databases in Distributed Environment

Abstract

In this article, a distributed clustering technique, that is suitable for dealing with large data sets, is presented. This algorithm is actually a modified version of the very common k-means algorithm with suitable changes for making it executable in a distributed environment. For large input size, the running time complexity of k-means algorithm is very high and is measured as O(TKN), where K is the number of desired clusters, T is the number of iterations, and N is the number of input patterns. The high time complexity of the serial k-means can be heavily reduced by executing it on a distributed parallel environment. Here, we shall describe a new distributed clustering algorithm and compared its performance with some other existing algorithms. Results of experiments show that this distributed approach can provide higher speedups and at the same time maintains all necessary characteristics of the serial k-means algorithm. We have successfully applied the new algorithm for clustering a number of data sets including a large satellite image data.