HSCFC: High-dimensional streaming data clustering algorithm based on feedback control system

Abstract

At present, few studies have considered the clustering analysis of high-dimensional data collected by many sensors in a real-time streaming environment. Existing clustering analysis algorithms for high-dimensional data are primarily based on batch processing models, and most of them cannot meet the requirements of incremental high-dimensional data streams, which are extremely common in practical applications. To address the aforementioned problems, this paper focuses on the study of high-dimensional data clustering based on stream processing, and proposes a high-dimensional data stream clustering algorithm based on a feedback control system, which comprises three stages: window principal component analysis, feedback stream clustering, and feedback controller. The classic exponentially weighted attenuation function is used in the window principal component analysis to avoid concept drift in the data stream, and incremental feature extraction executes through the sliding window to improve the iterative efficiency of the data in the window. To minimize the errors caused by variability in projection angles during dimensionality reduction, a feedback stream clustering stage is designed with alternating iterations of window clustering and cluster aggregation. Aiming at the problems caused by manually adjusting the hyperparameters used in high-dimensional data stream clustering, a feedback controller is developed to adjust the hyperparameters in the two other stages by analyzing the clustering results in real time and using a discriminant score to adopt corresponding feedback strategies. The experimental comparisons between the proposed and the traditional algorithms on multiple datasets demonstrate the effectiveness of the former.