Mining underlying correlated-clusters in high-dimensional data streams

Abstract

High-dimensional data streams pose challenges to traditional clustering algorithm, due to their inherent sparsity and data tend to cluster in different subspaces of the entire feature space. In this paper, we resolve the subspace clustering problem by mining correlated-clusters, in which selected features are correlated with each other. Moreover, taking data evolution in data streams into account, we propose methods to mine correlations of features incrementally and adaptively. At each time tick t, according to our proposed multiple regression measurement, we cluster the newly arrived data sample to one of correlated-clusters whose local correlations fit to the data sample and also update the local correlations adaptively, based on an incremental principal component analysis technology. The results of experiments on high-dimensional synthetic data and real data demonstrate that our methods can achieve higher accuracy of query than related work and perform much more efficiently. Additionally, our proposed methods are able to forecast missing values in streaming data successfully.