SeqAD: An Unsupervised and Sequential Autoencoder Ensembles based Anomaly Detection Framework for KPI

Abstract

Key Performance Indicator (KPI), a kind of time-series data, its anomalies are the most intuitive characteristics when failures occurred in IT systems. KPI anomaly detection is increasingly critical to provide reliable and stable services for IT systems. Unsupervised learning is a promising method because of lacking labels and the unbalance in KPI samples. However, existing unsupervised KPI anomaly detection methods suffer from high false alarm rates. They handle KPI sequence as non-sequential data and ignore the time information, which is an essential KPI character. To this end, in this paper, we propose an unsupervised and sequential autoencoder ensembles based anomaly detection framework called SeqAD. SeqAD inherits the advantages both from the sequence-to-sequence model and autoencoder ensembles. SeqAD reduces the KPI over-fitting problem effectively by introducing autoencoder ensembles. In order to better capture the time information of KPI, we propose a random step connection based recurrent neural network (RSC-RNN) to train the KPI sequence, which can provide random connections to construct autoencoders with different structures and retain time information to the most extent. Extensive experiments are conducted on two public KPI data-sets from real-world deployed systems to evaluate the efficiency and robustness of our proposed SeqAD framework. Results show that SeqAD is able to smoothly capture most of the characteristics in all KPI data-sets, as well as to achieve a high F1 score between 0.93 and 0.98, which is better than the state-of-art unsupervised KPI anomaly detection methods.