AI-Aided Game: Enhancing the Defense Performance of Scale-Free Network via Deep Reinforcement Learning

Abstract

Scale-free network, as a type of complex network, is prone to various target attacks. Some researches characterize the attack and defense problem of complex network as a game problem, and design the optimal defense strategy from the perspective of achieving the game equilibrium. However, such a game problem in the scale-free network space is very difficult to solve because the strategy space is too large. An effective way to achieve the equilibrium is to train a growing population of player strategies and use heuristic methods to guide the population evolution. The deep reinforcement learning (DRL) has been used to guide the population evolution due to its strong ability to explore new strategies. Inspired by this fact, in this paper we apply an AI-aided game framework, Policy Space Response Oracle (PSRO), to solve the above game problem. We first abstract the scale-free network as a game environment and design a set of game rules. The confrontation game occurs between two agents acting as the attacker and the defender in the network, respectively. We then use the convolutional neural network (CNN) to evaluate values of actions in deep Q-network (DQN) and extract game states into tensor forms as CNN’s inputs. To tackle the targeted attacks against the scale-free networks, we carefully design a specific action value for the defender combining the node weight and network connectivity parameters of a node in CNN. The simulation results show that the defender agent using CNN as strategies can delay the attacker’s intrusions under equilibrium, and achieve a balance between protecting high-weight nodes and keeping the network connectivity.