Active power correction strategies based on deep reinforcement learning-part I: A simulation-driven solution for robustness

Abstract

This paper addresses the active power corrective control of modern power systems by adopting deep reinforcement learning. The strategy aims to minimize the joint effect of operation cost and blackout penalty, while robustness and adaptability of the control agent are studied. In Part I of this paper, we consider the robustness case, where the agent is developed to deal with unexpected incidents and guide the stable operation of power grids. A simulation-driven graph attention reinforcement learning (SGA-RL) method is proposed to perform robust active power corrective control. The graph attention networks are introduced to learn the representation of power system states considering topological features. Monte Carlo tree search is utilized to select eligible actions from large action space, including generator redispatch and topology control actions. Finally, driven by simulation, a guided training mechanism and a long-short term action deployment strategy are designed to help the agent better evaluate the action set while training and operate more stably while deploying. The effectiveness of the proposed method is demonstrated in 2020 Learning to Run a Power Network — Neurips Track 1 global competition and relevant cases. In Part II of this paper, we address the adaptability case, where the agent is established to adapt to the grid with an increasing share of renewable energies over years.