A multi-agent deep reinforcement learning-based “Octopus” cooperative load frequency control for an interconnected grid with various renewable units

Abstract

With the aim of improving the frequency regulation performance of load frequency control (LFC) in an interconnected power system as well as reducing wastage of frequency regulation resources, an “octopus” cooperative load frequency control (OC-LFC) method is proposed. This method addresses the secondary frequency regulation coordination problem affecting different areas, as well as the coordination problem arising between the controller and power distributor within the same area. In addition, for this framework a tracking-explorer-based distributed multi-agent deep deterministic policy gradient (TED-MADDPG) algorithm is proposed. This algorithm imitates the edge-cloud cooperation framework, akin to an octopus. In the performance-based frequency regulation market, the controller and power distributor in each area are regarded as independent agents, which output the total generated power command and the generated power commands of different units. In the proposed method, centralized training and distributed implementation are employed so that a global optimal decision with higher robustness can be obtained. Furthermore, this algorithm also employs a tracking-explorer strategy which incorporate multiple learning techniques and in turn enhance the robustness of the proposed method. The effectiveness of the proposed method is demonstrated using a China Southern Grid (CSG) four-area load frequency control model.