GAIL-PT: An intelligent penetration testing framework with generative adversarial imitation learning

Abstract

Penetration testing (PT) is an efficient tool for network testing and vulnerability mining by simulating the hackers’ attacks to obtain valuable information applied in operating and database systems. Most of the traditional manual solutions are strongly relying on the domain knowledge of human experts with high penetration costs. Therefore, solutions based on the artificial intelligent algorithm such as reinforcement learning (RL) and deep reinforcement learning (DRL), with less time-consuming and lower labor costs, become a great solution to address the challenge. However, there are still a few challenges for RL/DRL-based PT in real penetration scenarios, such as the large dimension size of the agent’s discrete action space usually causing difficulties in convergence. To address the above issue, this paper proposes a novel framework named Generative Adversarial Imitation Learning based intelligent Penetration Testing (GAIL-PT), which utilizes expert knowledge base and GAIL network to guide the policy generation of RL/DRL agents with lower costs. Specifically, we first construct the expert knowledge bases by collecting state-action pairs from the successful exploitations of pre-trained RL/DRL models. Secondly, we feed the expert knowledge bases generated by different RL/DRL models online into the discriminator of GAIL-PT to guide its training process. Besides, we integrate the losses of the generator and the discriminator in GAIL-PT to optimize the overall objective and use the discriminator’s discounted rewards for policy generation. The extensive experiments conducted on the practical target hosts and simulated network scenarios demonstrate that GAIL-PT achieves outstanding performance, and outperforms the state-of-art method DeepExploit in exploiting Metasploitable2 and Q-learning in different scale networks. It also verified that GAIL-PT is a general leading framework suitable for RL/DRL-based methods. The code of GAIL-PT is open-sourced at https://github.com/Shulong98/GAIL-PT//.