Reinforcement-learning-based dynamic defense strategy of multistage game against dynamic load altering attack

Abstract

As the current power grid is highly interconnected and more information and communication technologies (ICTs) are being deployed recently, it could be the target of malicious cyber-physical attacks. Dynamic load altering attacks (D-LAAs), as a special case of load altering attacks, could be performed to interfere the demand response and ultimately force certain generators off-line. Cascading failures due to transmission line overloads may also be triggered. In this paper, we propose a new dynamic defense strategy against D-LAAs through a multistage game between the attacker and the defender which is solved by minimax-q learning. Different from the static game, the multistage game considers the attacker and defender’s action sequences and the optimal strategies at each state are learned. After each time step, the cascading failure is measured, and the load shedding is used as the feedback for the attacker to generate the next action strategy. The performance of the proposed model is evaluated on the IEEE 39-bus system. Comparisons between the dynamic defense strategy and the passive defense strategy are conducted, and the results verify the advantage of the proposed dynamic defense strategy. To improve the power system resilience, this defense strategy can be deployed in advance when such cyber-physical attacks are anticipated.