Online PV Smart Inverter Coordination using Deep Deterministic Policy Gradient

Abstract

Fast and frequent solar power variations present new challenges to modern power grid operation with increasing adoption of photovoltaic (PV) energy. PV smart inverters (SIs) provide a fast-response method to regulate voltage by modulating active and/or reactive power at the connection point. In this paper, a deep reinforcement learning (DRL) based algorithm is proposed to coordinate multiple SIs. A reward scheme is designed to balance voltage regulation and SI reactive power utilization. The proposed DRL agent for voltage control learns its policy through massive offline simulations and adapts to load and solar variations. The DRL agent results are compared against autonomous Volt-Var control and optimal power flow (OPF) on the IEEE 37 bus feeder and IEEE 123 bus feeder for 8760 different scenarios. The results demonstrate that a properly trained DRL agent can intelligently coordinate different SIs to satisfy grid voltage limits despite large solar and load variations. The DRL agent achieves nearly the optimal performance of OPF by mitigating all voltage violations, while reducing PV production curtailment by 88% compared to the autonomous Volt-Var scheme. Contrary to OPF, the DRL agent can provide a coordination signal in milliseconds without load and solar forecasts and without explicit knowledge of the distribution network model.