Power System Security Under False Data Injection Attacks With Exploitation and Exploration Based on Reinforcement Learning

Abstract

The false data injection (FDI) attack is a potential threat to the security of smart grids, and therefore, such threats should be assessed carefully. This paper proposes a self-governing FDI attack method with exploitation and exploration mechanisms and then evaluates its threat to power systems. The attack is executed by viruses embedded in monitoring systems in substations. First, the FDI attack is formulated as a partially observable Markov decision process. Then, an improved online reinforcement learning method called nearest sequence memory Q-learning is adopted to make the attack more effective. Finally, propagation, an inherent property of viruses, is described using a propagation-evolution model that serves as the exploration mechanism for the proposed FDI attack. To validate the proposed attack method, cosimulations of daily operations of the IEEE 39-bus system are performed in which both the automatic voltage control system and the proposed FDI attack are modeled. Test results show that the proposed FDI method can cause voltage collapse even if only a few substations are infected.