Active sensitivity coefficient-guided reinforcement learning for power grid real-time dispatching

Abstract

The integration of renewable energy sources into power systems and the expansion of grid scale have introduced a higher degree of operational unpredictability, thereby increasing the risk of safety hazards such as transmission line overloads. Existing reinforcement learning algorithms that incorporate expert knowledge aim to facilitate secure dispatching of the power grid. However, these algorithms, which adopt strategies akin to imitation learning during the learning process, do not effectively address systemic constraints such as line limits and power balance. In response to this issue, this study introduces a novel active sensitivity coefficient-guided reinforcement learning method for real-time power grid dispatch (SRL-Dispatching). Firstly, a sensitivity-based reinforcement learning framework is proposed using the Soft Actor-Critic (SAC) algorithm. Secondly, a sensitivity coefficient-based method for compressing the action space boundary is proposed to address the security implications associated with transmission line overloads. Subsequently, a feasible domain projection approach is introduced to ensure that dispatch operations adhere to safety constraints. By employing sensitivity factors to guide the learning process of the agent in managing line overloads and ensuring safe operation, the precision and robustness of dispatch strategies in the power system environment are significantly enhanced. Simulation results using the SG-126 power grid simulator indicate that SRL-Dispatching accelerates training by a factor of 9.8 compared to state-of-the-art RL methods, with comparable decision-making times. Across various load levels, SRL-Dispatching demonstrates superior performance in terms of renewable energy integration, line overload management, and power balance control.