An artificial intelligence-based real-time monitoring framework for time series

Abstract

In the era of data technology, data growth is occurring at an unprecedented scale. Business data and information are among the most valuable assets. Massive data analysis now drives nearly every aspect of society and can facilitate informed decision-making by businesses. Fully automated data flow detection of anomalies plays a crucial role in maintaining data service stability and preventing malicious attacks. This paper presents an extensible and generic real-time monitoring system framework (EGRTMS) for large-scale time-series data. EGRTMS employs a prediction module and an anomaly detection module within an anomaly filtering layer for the accurate identification of anomalies. Moreover, the alarm module and anomaly handling module within an anomaly trace processing layer enables the system to respond swiftly to the detected threats. Our solution does not rely on the labelling of anomalies; instead, a predictor module with a deep learning attention-based mechanism learns the normal behaviour of the data, and an anomaly handling module determines the dynamic alarm-threshold by utilizing a sliding window. The results of this study demonstrate that our framework significantly outperforms other anomaly detection systems on most real and synthetic datasets.