Improving the offline clustering stage of data stream algorithms in scenarios with variable number of clusters

Abstract

Many data stream clustering algorithms operate in two well-defined steps: (i) online statistical data collection stage; and (ii) offline macro-clustering stage. The well-known k-means algorithm is often employed for performing the offline macro-clustering step. The conventional k-means algorithm assumes that the number of clusters (k) is defined a priori by the user. Given the difficulty of defining the value of k a priori in real-world problems, we describe a new approach that allows estimating k dynamically from streams with variable number of clusters, which is a common scenario in data with a non-stationary distribution. In addition, we combine our dynamic approach with two different strategies for initializing the centroids during the offline clustering. Analysis of results suggest that, using the dynamic approach, the method k-means++ for centroids initialization present better results.