Analyzing Log Files for Postmortem Intrusion Detection

Abstract

Upon an intrusion, security staff must analyze the IT system that has been compromised, in order to determine how the attacker gained access to it, and what he did afterward. Usually, this analysis reveals that the attacker has run an exploit that takes advantage of a system vulnerability. Pinpointing, in a given log file, the execution of one such an exploit, if any, is very valuable for computer security. This is both because it speeds up the process of gathering evidence of the intrusion, and because it helps taking measures to prevent a further intrusion, e.g., by building and applying an appropriate attack signature for intrusion detection system maintenance. This problem, which we call postmortem intrusion detection, is fairly complex, given both the overwhelming length of a standard log file, and the difficulty of identifying exactly where the intrusion has occurred. In this paper, we propose a novel approach for postmortem intrusion detection, which factors out repetitive behavior, thus, speeding up the process of locating the execution of an exploit, if any. Central to our intrusion detection mechanism is a classifier, which separates abnormal behavior from normal one. This classifier is built upon a method that combines a hidden Markov model with <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -means. Our experimental results establish that our method is able to spot the execution of an exploit, with a cumulative detection rate of over 90%. In addition, we propose an entropy-based approach that speeds up the construction of a profile for ordinary system behavior.