Markov game based on reinforcement learning solution against cyber–physical attacks in smart grid

Abstract

The rapid expansion of information networks has amplified vulnerabilities in cyberspace, particularly within the power grid and communication layers of smart grid systems. Safeguarding smart grid security has become imperative for users concerned about both online and offline privacy. Detecting attacks in smart grids is crucial for ensuring the safety and continuity of the electrical network. This paper introduces a novel stealth attack-defense game that examines the impact of these invasions on the availability, confidentiality, and integrity of phasor measurement units. The proposed system considers both attack and defense perspectives. Attackers focus on key parameters to covertly infiltrate system layers while defending the smart grid system requires advanced knowledge of data flow and critical insights to establish robust protections. To address this cybersecurity challenge, we propose a game approach based on a partially observed Markov game with an improved Shapley Q-value and a multi-agent reinforcement learning framework, implemented in cooperative, communicative, and dynamic environments. The evaluation results demonstrate the robustness of the proposed approach against coordinated attacks in smart grid environments. Additionally, the results emphasize the significance of attack distribution as a crucial trace for identifying the attack source. Our approach shows promise for addressing various types of attacks against smart grid systems.