Dynamic load balancing on GPU clusters for large-scale K-Means clustering

Abstract

K-Means is the clustering algorithm which is widely used in many areas such as information retrieval, computer vision and pattern recognition. With the recent advance in General Purpose Graphics Processing Unit (GPGPU), we can use a modern GPU which is capable to do computation up to Tflops to calculate K-Means clustering on average problems. However, due to the exponential growth of data, the K-Means clustering on a single GPU will not be adequate for large datasets in the near future. In this paper, we present the design and implementation of an efficient large-scale parallel K-Means on GPU clusters. We utilize the massive parallelism in GPUs to speed up the most time consuming part of K-Means clustering in each node. We employ the dynamic load balancing to distribute workload equally on different GPUs installed in the clusters so as to improve the performance of the parallel K-Means at the inter-node level. We also take advantage from software distributed shared memory to simplify the communication and collaboration among nodes. The result of the evaluation shows the performance improvement of the parallel K-Means by maintaining load balance on GPU clusters.