Abstract
In this paper, we introduce a new framework to address the problem of voltage regulation in unbalanced distribution grids with deep photovoltaic penetration. In this framework, both real and reactive power setpoints are explicitly controlled at each solar panel smart inverter, and the objective is to simultaneously minimize system-wide voltage deviation and maximize solar power output. We formulate the problem as a Markov decision process with continuous action spaces and use proximal policy optimization, a reinforcement learning-based approach, to solve it, without the need for any forecast or explicit knowledge of network topology or line parameters. By representing the system in a quasi-steady state manner, and by carefully formulating the Markov decision process, we reduce the complexity of the problem and allow for fully decentralized (communication-free) policies, all of which make the trained policies much more practical and interpretable. Numerical simulations on a 240-node unbalanced distribution grid, based on a real network in Midwest U.S., are used to validate the proposed framework and reinforcement learning approach.
Highlights
P HOTOVOLTAIC (PV) smart inverter technology introduced in recent years enables solar panels to act as distributed energy resources (DERs) that can provide bi-directional reactive power support to electric power grid operations [1]–[3]
Policy gradients, which we review later in this paper, are an alternative set of reinforcement learning (RL) approaches that enable continuous action spaces, and in [16]–[18], policy gradients are used to solve Volt-VAR control problems where reactive power support is selected from a continuous set using a deep neural network that maps states directly to actions
Even in the absence of a storage system, under deep enough photovoltaic penetration, we discover that the RL agent surprisingly learns by itself that it might be better to draw only parts of the available power, in order to avoid over-voltage, especially if there is an insufficient amount of reactive power resources available
Summary
P HOTOVOLTAIC (PV) smart inverter technology introduced in recent years enables solar panels to act as distributed energy resources (DERs) that can provide bi-directional reactive power support to electric power grid operations [1]–[3]. In [7]–[9], the distribution grid is modelled using the widely adopted linearized flow model, known as LinDistFlow, that assumes a tree (radial) grid structure and negligible line losses In these papers, the same voltage regulation problem is solved, and an extension to these works are [10], [11] in which limited or no communication between buses is needed and the same LinDistFlow model is adopted to provide theoretical guarantees on convergence and stability of the proposed control schemes. The key contributions of this paper are suggested as follows: 1) Joint optimization of real and reactive power injection from the PV generation is formulated, with a parameterized reward function tailored for a multi-agent RL approach Such parameterization is shown to facilitate more interpretable and easier to train deep RL policies, for a more user-friendly experience.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
