Scale Variable Dynamic Stream Anomaly Detection with Multi-chain Queue Structure

Abstract

Most existing data stream anomaly detection algorithms do not involve in the tackling of multi-scale characteristic of stream data, a multi-chain queue based data storage structure that is especially suitable for the analysis of multi-scale stream data is designed, and a corresponding algorithm to identify the multi-scale stream anomaly is proposed. The algorithm employs an iteration strategy and takes 3θ as the criteria for discriminating the anomaly, to minimize each anomaly’s effect to its neighbors, to detect simultaneously the anomalies in the data sequences that are at the same time and the anomalies in different times. Meanwhile, the increase of a new data sample and the delete of an obsolete observation data are implemented effectively through the operation of a queue, Hence, a better result of stream data anomaly mining is obtained with the changing of mining scale. Finally, through the experiments on a real stream dataset, the proposed algorithm is shown to be capable of finding out some true anomalies in different scales with a higher accuracy rate, when compared with the traditional sliding window based algorithms and the machine learning based algorithms.