Attention based multi-agent intrusion detection systems using reinforcement learning

Abstract

Designing an effective network intrusion system (IDS) is a challenging problem because of the emergence of a large number of novel attacks and heterogeneous network applications. The existing IDSs fail to adapt to the changing attack patterns and unseen attacks that lead to inaccurate detection of network vulnerabilities and system performance degradation. Therefore, there is a need to design robust, scalable, efficient, and adaptive IDS for networks. This paper presents a novel deep reinforcement learning-based IDS that employs Deep Q-Network logic in multiple distributed agents and uses attention mechanisms to efficiently detect and classify advanced network attacks. Our proposed multi-agent IDS is designed as a distributed attack detection platform where agents work in a coordinated manner to provide scalable, fault-tolerant, multi-view architecture guided security system. We have tested our model with extensive experimentation on two benchmark datasets: NSL-KDD and CICIDS2017. It shows improved performance in terms of higher accuracy, precision, recall, F1-Score, and low false-positive rate (FPR) in comparison to the state-of-the-art IDS works. On the other hand, many machine learning systems are found vulnerable to adversarial attacks. Thus, we evaluated our model’s robustness against a practical black-box adversarial attack and observed only a little degradation in performance. We integrated the concept of denoising autoencoder (DAE) with our model to further improve its robustness. Finally, we discuss the usability of our system in real-life applications against zero-day attack patterns.