Deep reinforcement learning based topology-aware voltage regulation of distribution networks with distributed energy storage

Abstract

Both the high penetration of clean energy with strong fluctuation and the complicated variable operation condition bring great challenges to the voltage regulation of the distribution network. To deal with the problem, a topology-aware voltage regulation multi-agent deep reinforcement learning (MADRL) algorithm is proposed. The distributed energy storages (DESs) are modeled as agents to regulate voltage autonomously in real-time, which could fast adapt to dynamic topological scenarios. Firstly, taking the minimization of voltage fluctuation and maximization of reserve capacity as the target, the optimal voltage regulation model is established. Secondly, a topology extraction method considering voltage sensitivity is proposed for dynamic topology clustering, and the obtained typical topology is added to the observation set of agents. Then, the optimal voltage regulation model is formulated to the decentralized partially observable Markov decision process (Dec-POMDP) framework, in which only local information is required for the agent during the test process to decision-making to realize the hierarchical and partitioned control of voltage. Finally, the multi-agent deep deterministic policy gradient (MADDPG) algorithm is used to solve the Dec-POMDP model. The feasibility and superiority of the proposed algorithm are verified and analyzed in the simulation under different scenarios.