Evaluation of Reinforcement Learning-Based False Data Injection Attack to Automatic Voltage Control

Abstract

False data injection (FDI) attacks intend to threaten the security of power systems. In this paper, a novel strategy of FDI attacks is proposed, which aims to distort normal operation of a power system regulated by automatic voltage controls (AVCs). Such attacks can be launched from a single substation by the attacker who has little knowledge of the whole power grid. The optimal attack strategy is modeled as a partial observable Markov decision process (POMDP). Then, a $\mathcal {Q}$ -learning algorithm with nearest sequence memory is adopted to enable on-line learning and attacking. Stealthy attack strategies are also developed and incorporated into the POMDP model. Various tests are performed upon the IEEE 39-bus systems. Corresponding results verify the efficacy of the proposed attack strategies. The feasibility of independent and data-driven FDI attacks is investigated. Moreover, a bad data detection and correction method are presented based on kernel density estimation to mitigate the disruptive impacts of the proposed FDI attacks. Test results show that this defensive method can help maintain the security of the AVC system, even under heavy system loading.