Context Sensitive Host-Based IDS Using Hybrid Automaton

Abstract

A key function of a host-based intrusion detection system is to monitor program execution.Models constructed based on static analysis have the advantage of not producing false alarms;still, they can not be put into practice due to imprecision or inefficiency of the model.The prior work has shown a trade-off between efficiency and precision.In particular, models based upon non-deterministic finite state automaton(DFA) are efficient but lack precision.More accurate models based upon pushdown automaton(PDA) are very inefficient to operate due to non-determinism in stack activity.DYCK model, VPStatic model and IMA use some subtle approaches to achieve more determinism by extracting information about stack activity of the program or inserting code to expose program state or just inline the local automaton but still can not solve the problem of indirect call/JMP.This paper presents a new training-free model(hybrid finite automaton, HFA) to gain more determinism and resolves indirect call/JMP through static-dynamic hybrid approach.The results show that in run-time, these models slowed the execution of the test programs by 5% to 10%.This paper also formally compares HFA with some typical models and proves that HFA is more accurate than others and it is more suitable for dynamic linked applications.Some technical details of the protocol type system on Linux and experimental results are also presented in the paper.