Safe resource management of non-cooperative microgrids based on deep reinforcement learning

Abstract

This paper solves a bi-level optimization problem in which at the first level, the distribution system operator (DSO) as a retailer tends to extract selling energy price signals so that increases its profit and reduces the peak-to-average ratio (PAR) of the MMG transactive energy; and at the second level energy management problems of non-cooperative networked microgrids (MGs) are solved individually to minimize their cost. This game theoretically problem is known as the Stackelberg game and has been solved with classical optimization methods like Karush–Kuhn–Tucker (KKT) method previously. To solve this problem with the classical KKT method, the MGs’ information should be provided for the DSO as a game starter, which is not the MGs’ desire for their privacy concerns. With the aim of preserving the privacy of MGs, this paper has developed a decision-making mechanism based on reinforcement learning (RL) to help the DSO extract energy prices individually for each MG. Also, the deep neural network (DNN) is used as a practical tool to predict MGs’ behavior in response to signal price. Furthermore, in the model used in this study, energy storage systems (ESSs) are dedicated to MGs which makes the model non-linear, thus the proposed method unlike the classic method is able to solve such a problem. The results obtained from the comparison between the classic and proposed method show that the implementation of this machine learning-based method is not only accurate enough but also is faster than the classical method, in addition to its privilege of privacy-preserving. Finally, a sensitivity analysis has been conducted to evaluate the DSO profit and PAR under different weighting factors of the objective function to obtain an appropriate performance.