Network Security Defense Decision-Making Method Based on Stochastic Game and Deep Reinforcement Learning

Zenan Wu,Yi Zhang,Yuquan Du,Yan Wang,Jianfei Xie,Liqin Tian,Yanhui Guo

doi:10.1155/2021/2283786

Abstract

Aiming at the existing network attack and defense stochastic game models, most of them are based on the assumption of complete information, which causes the problem of poor applicability of the model. Based on the actual modeling requirements of the network attack and defense process, a network defense decision-making model combining incomplete information stochastic game and deep reinforcement learning is proposed. This model regards the incomplete information of the attacker and the defender as the defender’s uncertainty about the attacker’s type and uses the Double Deep Q-Network algorithm to solve the problem of the difficulty of determining the network state transition probability, so that the network system can dynamically adjust the defense strategy. Finally, a simulation experiment was performed on the proposed model. The results show that, under the same experimental conditions, the proposed method in this paper has a better convergence speed than other methods in solving the defense equilibrium strategy. This model is a fusion of traditional methods and artificial intelligence technology and provides new research ideas for the application of artificial intelligence in the field of cyberspace security.

Highlights

In recent years, with the rapid development of information technology, network attacks have increased
Due to the complexity of the network system and the concealment and stochasticity of the attack means, it is hard for existing network defense technology to meet the security requirements of the network system, which makes it harder for the defender of the network system to guarantee the absolute security of the system. erefore, there is a need for a new technology that can analyze network attack and defense events, so that network system defenders can implement dynamic and adaptive adjustment of defense strategies [3]
In response to the above problems and to improve the applicability of the stochastic game model in the analysis of network offensive and defensive events, this paper proposes a defense strategy selection model based on incomplete information stochastic game

Summary

Introduction

With the rapid development of information technology, network attacks have increased. In order to analyze the influence of bounded rationality on the stochastic game of network offense and defense, Zhang and Liu [16] aimed at the problem of state explosion when the number of network nodes increases; an offensive graph and a defensive graph have been designed to compress the state space and extract the network state and defense strategy On this basis, the introduction of intelligent learning algorithms and the design of defense decision-making algorithms with online learning capabilities would help to select the optimal defense strategy with the greatest benefit from the set of candidate strategies. In response to the above problems and to improve the applicability of the stochastic game model in the analysis of network offensive and defensive events, this paper proposes a defense strategy selection model based on incomplete information stochastic game.

Stochastic Game Model with Incomplete Information

Deep Reinforcement Learning and Bayesian Equilibrium Solution

Simulation Experiment and Analysis

Example Calculation