Clustering algorithm based on optimal intervals division for high-dimension data streams

Abstract

Clustering for high-dimension data streams is a main focus in the field of clustering research. In order to optimize the clustering process, especially for the large number of candidate subspaces generated in it, optimal segmentation section technology and FP-tree structure are introduced, based on which, DOIC (Dynamic optimal intervals-based cluster) algorithm is proposed. In this paper, the memory-based data partition and optimal intervals division are defined to generate high-density grids for each dimension, which are stored in a High-Density Unit tree (HDU). The HDU-tree is built according to the principle that high-density grids for the same interval in every dimension are stored in the same branch. Thus the process of clustering high-dimension data streams is transformed into that of searching for dense grids in the HDU-tree. By merging HDU-trees, new data streams is inserted and historical data streams is decayed, then the updating of data streams is achieved. The clustering result is returned in the form of DNF expressions timely as requests. The experimental results demonstrate that DOIC has better space scalability and higher clustering quality compared with traditional clustering algorithms.