Semi-Supervised Range-Based Anomaly Detection for Cloud Systems

Abstract

The inherent characteristics of cloud systems often lead to anomalies, which pose challenges for high availability, reliability, and high performance. Detecting anomalies in cloud key performance indicators (KPI) is a critical step towards building a secure and trustworthy system with early mitigation features. This work is motivated by (i) the efficacy of recent reconstruction-based anomaly detection (AD), (ii) the misrepresentation of the accuracy of time series anomaly detection because point-based Precision and Recall are used to evaluate the efficacy for range-based anomalies, and (iii) detects performance and security anomalies when distributions shift and overlaps. In this paper, we propose a novel semi-supervised dynamic density-based detection rule that uses the reconstruction error vectors in order to detect anomalies. We use long short-term memory networks based on encoder-decoder (LSTM-ED) architecture to reconstruct the normal KPI time series. We experiment with both testbed and a diverse set of real-world datasets. The experimental results show that the dynamic density approach exhibits better performance compared to other detection rules using both standard and range-based evaluation metrics. We also compare the performance of our approach with state-of-the-art methods, outperforms in detecting both performance and security anomalies.