Multi-Agent Correlated Equilibrium Q(λ) Learning for Coordinated Smart Generation Control of Interconnected Power Grids

Abstract

This paper proposes an optimal coordinated control methodology based on the multi-agent reinforcement learning (MARL) for the multi-area smart generation control (SGC) under the control performance standards (CPS). A new MARL algorithm called correlated Q(λ) learning (CEQ(λ)) is presented to form an optimal joint equilibrium strategy for the coordinated load frequency control of interconnected control areas, and a SGC framework is proposed to facilitate information sharing and strategic interaction among multi-areas so as to enhance the overall long-run performance of the control areas. Furthermore, a novel time-varying equilibrium factor is introduced into the equilibrium selection function to identify the optimum equilibrium policies in various power system operation scenarios. The performance of CEQ(λ) based SGC strategy has been fully tested and benchmarked on a two-area power system and the China Southern Power Grid. Comparative studies have not only demonstrated the superior equilibrium optimization and dynamic performance of the proposed SGC strategy but also confirmed its fast convergence and high flexibility in designing the equilibrium factor for the desirable operating state of correlated equilibria.