Anomaly detection of user behavior based on DTMC with states of variable-length sequences

Abstract

In anomaly detection, a challenge is how to model a user's dynamic behavior. Many previous works represent the user behavior based on fixed-length models. To overcome their shortcoming, we propose a novel method based on discrete-time Markov chains (DTMC) with states of variable-length sequences. The method firstly generates multiple shell command streams of different lengths and combines them into the library of general sequences. Then the states are defined according to variable-length behavioral patterns of a valid user, which improves the precision and adaptability of user profiling. Subsequently the transition probability matrix is created. In order to reduce computational complexity, the classification values are determined only by the transition probabilities, then smoothed with sliding windows, and finally used to discriminate between normal and abnormal behavior. Two empirical evaluations on datasets from Purdue University and AT&T Shannon Lab show that the proposed method can achieve higher detection accuracy and require less memory than the other traditional methods.