Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid

Abstract

Uncertainties of solar PhotoVoltaic (solar PV) generation and Electric Vehicle (EV) demand are major issues for Optimal Energy Management (OEM) tasks in MicroGrid (MG), especially regarding power system stability and increased overall demand. A Probabilistic Load Flow (PLF) with a Battery Energy Storage System (BESS) controlled by an Energy Management System (EMS) can deal with these issues. However, the PLF and optimization algorithm based on the iterative method leads to a high computation burden. Therefore, this paper proposes a model-free data-driven approach assisted Deep Reinforcement Learning (DRL) to decrease the computation burden of PLF and problem-solving. Deep Neural Networks (DNNs) are developed as a model-free data-driven to estimate the power flow parameters of MG instead of PLF. Moreover, the DRL named a Deep Deterministic Policy Gradient (DDPG) is deployed as the optimization algorithm to find the optimal solution in the OEM task. In addition, finding appropriate parameters of the DDPG is proposed in this paper. To showcase the efficacy of the proposed method, a low-voltage distribution network is developed as the MG. Objective functions including exchanged energy, carbon emission, and BESS degradation costs are considered in this work. Simulation results indicate that the proposed method is capable of reducing the computation burden by 88.47% in comparison to the Differential Evolution (DE) algorithm that uses 10 populations. Moreover, the best agent model obtained from finding suitable parameters of DDPG can provide a total cost of 115.63 USD/day, which is the lowest cost compared with the cost obtained by the DE.