A Two-Level Energy Management Strategy for Multi-Microgrid Systems With Interval Prediction and Reinforcement Learning

Abstract

Setting retail electricity prices is one of the significant strategies for energy management of multi-microgrid (MMG) systems integrated with renewable energy. Nevertheless, the need of privacy preservation, the uncertainties of renewable energy and loads, as well as the time-varying scenarios, bring challenges for pricing problems. In this paper, a two-level pricing framework is proposed based on interval predictions and model-free reinforcement learning to address these challenges. In particular, at the higher level, the distribution system operator (DSO) is viewed as an agent, which sets retail electricity prices without detailed user information for privacy protection to maximize the total revenue from selling energy with reinforcement learning. For time-varying scenarios with intermittent photovoltaic power generation and diverse loads, a differentiable trust region layer is considered in reinforcement learning to improve the robustness of the policy updating process. While at the lower level, operators in microgrids solve three-phase unbalanced optimal power flow (OPF) problems to minimize generation cost and network power loss. Additionally, to deal with the challenges from the uncertainties of renewable power generation and user loads, interval predictions are chosen to quantify prediction errors and improve the flexibility of pricing policies. Finally, a set of experiments are conducted to validate the effectiveness of the proposed method for pricing problems in MMG systems.