Multi-source data based anomaly detection through temporal and spatial characteristics

Abstract

Anomaly detection is vital in complex distributed systems. However, existing methods did not take into full account the temporal and spatial characteristics of data from multiple sources in the system. That will lead to less accurate of anomaly detection. To address this limitation, in this paper we propose a multi-source data based anomaly detection method through temporal and spatial characteristics. Specifically, we first considered the spatial characteristics. We propose a Transformer encoder to parse log templates and output template vectors, then an attention-based CNN is introduced to obtain the spatial characteristics from traces. Next, we considered the temporal characteristics. We propose Bi-LSTM network to obtain the temporal characteristics of multi-source data of the distributed systems, followed by anomaly detection. Finally, extensive comparative experiments verify the effectiveness of our method, the F1-score reaches 0.859 and improves by 2.14% compared to state-of-the-art methods.