Two-Stage Volt/Var Control in Active Distribution Networks With Multi-Agent Deep Reinforcement Learning Method

Abstract

The high penetration of intermittent renewable energy resources in active distribution networks (ADN) results in a great challenge for the conventional Volt-Var control (VVC). This article proposes a two-stage deep reinforcement learning (DRL)-based real-time VVC method to mitigate fast voltage violation while minimizing the network power loss. In the first stage, on-load tap changer (OLTC) and capacitor banks (CBs) are dispatched hourly based on the optimal power flow method. The optimization problem is formulated as a mixed-integer second-order cone programming (MISOCP) which can be effectively solved. In the second stage, the reactive power of photovoltaics (PVs) is regulated dynamically to mitigate fast voltage fluctuation based on the well-learned control strategy and local measurements. The real-time VVC problem is formulated and solved using a multi-agent deep deterministic policy gradient (MADDPG) method, which features offline centralized training and online decentralized application. Rather than using the critic network to evaluate the output of the actor-network, the gradient of the action-value function to action is derived analytically based on the voltage sensitivity method. The proposed approach is tested on the IEEE 33-bus distribution system and comparative simulation results show the enhanced control effect in mitigating voltage violations.