Real-Time Optimal Power Flow Using Twin Delayed Deep Deterministic Policy Gradient Algorithm

Jong-Bae Park,Jong Ha Woo,Lei Wu,Jae Hyung Roh

doi:10.1109/access.2020.3041007

Jong-Bae Park, Jong Ha Woo + Show 2 more

Open Access

https://doi.org/10.1109/access.2020.3041007

Copy DOI

Journal: IEEE Access	Publication Date: Jan 1, 2020
Citations: 44	License type: CC BY 4.0

Affiliation: Konkuk University, Stevens Institute of Technology

Abstract

The general concept of AC Optimal Power Flow (ACOPF) refers to the economic dispatch planning under electric network constraints. Moreover, each instance with the entire network must be solved in real-time (i.e., every five minutes) to ensure cost-effective power system operation while satisfying power balance equation. As the operation of power systems penetrated with intermittent renewable energy becomes more complicated, this article proposes Deep Neural Network (DNN) and Levenberg-Marquardt backpropagation-based Twin Delayed Deep Deterministic Policy Gradient (TD3) approach to improve computational performance of ACOPF. Specifically, because the ACOPF model shall consider prevailing constraints of the power system, including power balance equation, we set the appropriate reward vector in the training process to build our own policy. Furthermore, we add random Gaussian noise to individual net loads for representing uncertainty characteristics introduced by renewable energy sources. Finally, the proposed model is compared with the MAT-POWER solution on the IEEE 118-bus system to demonstrate its efficacy and robustness.

Highlights

In a deregulated power system, AC Optimal Power Flow (ACOPF) is the primary tool to offer the power system operation solution economically with high quality, which is a large-scale, multi-dimensional, nonconvex, non-linear, and constrained optimization problem [1]
To make TD3 policies explore better, we add noise to the action vector when we train the model, uncorrelated mean-zero gaussian noise, and we reduce the scale of the noise throughout the training
When we examine the model in various load conditions, it is possible to ignore power flow calculation if the error of slack generator’s output is smaller than the designated tolerance

Summary

Introduction

In a deregulated power system, ACOPF is the primary tool to offer the power system operation solution economically with high quality, which is a large-scale, multi-dimensional, nonconvex, non-linear, and constrained optimization problem [1]. It is difficult to determine the generator’s economic outputs due to the non-linear cost functions of generators and the rapid changes in load. The uncertainty of renewable energy output needs to be considered in OPF for optimum generator dispatch considering ramp-rate, increasing renewable energy integration proportion, where the forecasting error is inevitable even though advanced prediction techniques are utilized [3]. The Economic Dispatch (ED) addresses these issues to a certain extent but does not consider the loss, reactive power, and transmission line congestion [4]. To this end, Optimal Power Flow (OPF) problems are able to address these economic dispatch problems more realistically and reasonably. There has been extensive research on OPF algorithms, thanks to the recent development of optimization techniques and computational technologies [5]

Objectives

Methods

Conclusion