A Model-Free Approach to Intrusion Response Systems

Abstract

With the rising number of data breaches, denial of service attacks and general malicious activity facing modern computer networks, there is an increasing need to quickly and effectively respond to attacks. Intrusion Detection Systems provide an automated method of identifying malicious activity within a network however the development of an Intrusion Response System which can automatically respond to these alerts is non-trivial. Current research in IRS proposes model-based methods for identifying possible routes a malicious actor may take when attacking a network and use subjective performance values for the cost and benefit of a response, both of which can be invalidated by the increasingly dynamic nature of network topologies and system configurations. The IRS proposed in this work utilises a Model-free Reinforcement Learning approach and evaluates the Reinforcement Learning agent's performance in stopping two distinct multi-stage attack scenarios on a virtualised testbed. Experimentation demonstrates that the agent can successfully halt both attack scenarios and find responses which have minimal impact on normal network operation based on experience gained through training. A further contribution is the novel use of a virtualised environment that demonstrates Intrusion Response Reinforcement Learning performance in a more realistic environment than simulated tasks common to previous literature.