Hybrid metaheuristic multi-layer reinforcement learning approach for two-level energy management strategy framework of multi-microgrid systems

Abstract

This study builds a two-level energy management strategy framework for decentralized autonomy of microgrids and optimal coordinated operation of a multi-microgrid system. To reduce the operational cost of a combined cooling, heating and power multi-microgrid system with uncertain information and to improve the accuracy of load demand prediction, a hybrid metaheuristic multi-layer reinforcement learning algorithm is proposed for the framework of a multi-microgrid system. The proposed method is composed of a weighted delayed deep deterministic policy gradient algorithm, power adjustment network, and a genetic algorithm. At the first level, the microgrid operators utilize weighted delayed deep deterministic policy gradient algorithm with power adjustment network to optimize their operational strategies; at the second level, the distribution system operator employs a genetic algorithm to adjust its operational decision-making for minimizing the operational cost of the multi-microgrid system, reducing the peak-to-average ratios and power fluctuations at the points of common coupling. The data privacy of the parties in the multi-microgrid system is protected as each entity in the system does not have direct access to other entities’ information during the decision-making process. Numerical simulation results show that the proposed weighted delayed deep deterministic policy gradient algorithm with power adjustment network can rapidly obtain high-quality deterministic approximate optimal solution for economic dispatch of the microgrid. The framework proposed in this study achieves decentralized autonomy of microgrids, reduces the operational cost of the multi-microgrid system with incomplete or uncertain information, and indirectly improves the accuracy of load demands prediction at the points of common coupling.