Network intrusion detection based on system calls and data mining

Abstract

Anomaly intrusion detection is currently an active research topic in the field of network security. This paper proposes a novel method for detecting anomalous program behavior, which is applicable to host-based intrusion detection systems monitoring system call activities. The method employs data mining techniques to model the normal behavior of a privileged program, and extracts normal system call sequences according to their supports and confidences in the training data. At the detection stage, a fixed-length sequence pattern matching algorithm is utilized to perform the comparison of the current behavior and historic normal behavior, which is less computationally expensive than the variable-length pattern matching algorithm proposed by Hofmeyr et al. At the detection stage, the temporal correlation of the audit data is taken into account, and two alternative schemes could be used to distinguish between normalities and intrusions. The method gives attention to both computational efficiency and detection accuracy, and is especially suitable for online detection. It has been applied to practical hosted-based intrusion detection systems, and has achieved high detection performance.