Multi-timescale voltage control for distribution system based on multi-agent deep reinforcement learning

Abstract

Nowadays a large number of distributed generators and controllable elements connecting to the distribution network have resulted in higher requirements for voltage control. Existing voltage control methods require a specific physical model and are time-consuming with the complexity increasing. To overcome these challenges, this paper proposes a multi-timescale voltage control scheme using multi-agent deep reinforcement learning (MADRL). Firstly, considering the control requirements of different timescale regulators, a multi-timescale voltage control model is formulated. Then the control variables are assigned to multiple agents, and a MADRL-based voltage control scheme is proposed, which is based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm and applies Gumbel-Softmax distribution to solve discrete actions. Through centralized learning and decentralized execution, this scheme can obtain the optimal coordinated control strategy of multiple regulators adaptively and achieve the effect of distributed control. Finally, case studies are conducted on the modified IEEE-123 bus system, and the performance of the proposed method is compared with the conventional model-based optimal voltage control scheme and other DRL-based methods. The results show that the MADRL-based method possesses better performance than other methods.