A Multi-Agent Reinforcement Learning based Frequency Control Method with Data-Enabled Predictive Control Guided Policy Search

Abstract

In this paper, we propose a data-enabled predictive control (DeePC) guided multi-agent reinforcement learning (MARL) control algorithm, to solve the load frequency control (LFC) problem of power systems with multiple control areas. In the proposed algorithm, the offline training phase of MARL is guided by the DeePC algorithm. Moreover, a multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm is applied for LFC to reduce the overestimation bias of the Q-value in the existing multi-agent deep deterministic policy gradient (MADDPG) algorithm. Finally, the online control signals are generated directly by the trained policy neural networks. Simulation results on a two-area power system show that with the proposed DeePC-guided MATD3 algorithm: for one thing, the sample efficiency can be significantly improved, and thus the offline training phase can be stabilized and accelerated; for another thing, effective online cooperation of LFC between different control areas can be achieved.