Defending Against Data Integrity Attacks in Smart Grid: A Deep Reinforcement Learning-Based Approach

Dou An,Yang Zhang,Qingyu Yang,Wenmao Liu

doi:10.1109/access.2019.2933020

Dou An, Yang Zhang + Show 2 more

Open Access

https://doi.org/10.1109/access.2019.2933020

Copy DOI

Abstract

State estimation plays a critical role in monitoring and managing operation of smart grid. Nonetheless, recent research efforts demonstrate that data integrity attacks are able to bypass the bad data detection mechanism and make the system operator obtain the misleading states of system, leading to massive economic losses. Particularly, data integrity attacks have become critical threats to the power grid. In this paper, we propose a deep-Q-network detection (DQND) scheme to defend against data integrity attacks in alternating current (AC) power systems. DQND is a deep reinforcement learning scheme, which avoids the problem of curse of dimension that conventional reinforcement learning schemes have. Our strategy in DQND applies a main network and a target network to learn the optimal defending strategy. To improve the learning efficiency, we propose the quantification of observation space and utilize the concept of slide window as well. The experimental evaluation results show that the DQND outperforms the existing deep reinforcement learning-based detection scheme in terms of detection accuracy and rapidity in the IEEE 9, 14, and 30 bus systems.

Highlights

As a typical energy Cyber-physical Systems (CPS), the smart grid is designed to effectively monitor and control the twoway power and information flow between consumers and the grid by integrating advanced sensing, control and measurement technologies [1]–[5], [48], [54]
Delay-alarm error rate (DAE): We define the size of delay-alarm error as t − λ, and the delay-alarm error rate is defined as the sum of delay-alarm error in all episodes divided by the number of tests. t is the time when the agent detects the attack and λ is the time when the attack is launched initially
False-alarm error rate (FAE): We define the size of false-alarm error as λ − t, and the false-alarm error rate is denoted as the sum of false-alarm error in all episodes divided by the number of tests

Summary

Introduction

As a typical energy Cyber-physical Systems (CPS), the smart grid is designed to effectively monitor and control the twoway power and information flow between consumers and the grid by integrating advanced sensing, control and measurement technologies [1]–[5], [48], [54]. Depending on the target of the adversary, data integrity attacks fall into the following categories: attacks against state information (i.e., measurements) [16], [20], [32], [33], attacks against interactive electricity information (i.e., load demands, electricity price) [22], [42]. Regarding to the attack against state information, Sandberg et al proposed security parameters to quantify the minimum number of measurements to compromise so that an attack could be successfully launched, and utilized graph theory to derive data integrity attack vector [33]. Denote that there are M smart meters under an N -bus AC power system model. YM,t ], where yj,t is denoted as the measurement of smart meter j at time t. It is worth mentioning that we have M > N that assures the robustness of measurement system

Objectives

Results

Discussion

Conclusion