QLLog: A log anomaly detection method based on Q-learning algorithm

Abstract

Most of the existing log anomaly detection methods suffer from scalability and numerous false positives. Besides, they cannot rank the severity level of abnormal events. This paper proposes a log anomaly detection based on Q-learning, namely QLLog, which can detect multiple types of system anomalies and rank the severity level of abnormal events. We first build a mathematical model of log anomaly detection, proving that log anomaly detection is a sequential decision problem. Second, we use the Q-learning algorithm to build the core of the anomaly detection model. This allows QLLog to automatically learn directed acyclic graph log patterns from normal execution and adjust the training model according to the reward value. Then, QLLog combines the advantages of the Q-learning algorithm and the specially designed rules to detect anomalies when log patterns deviate from the model trained from log data under normal execution. Besides, we provide a feedback mechanism and build an abnormal level table. Therefore, QLLog can adapt to new log states and log patterns. Experiments on real datasets show that the method can quickly and effectively detect system anomalies. Compared with the state of the art, QLLog can detect numerous real problems with high accuracy 95%, and its scalability outperforms other existing log-based anomaly detection methods.