Energy management in residential microgrid using model predictive control-based reinforcement learning and Shapley value

Abstract

This paper presents an Energy Management (EM) strategy for residential microgrid systems using Model Predictive Control (MPC)-based Reinforcement Learning (RL) and Shapley value. We construct a typical residential microgrid system that considers fluctuating spot-market prices, highly uncertain user demand and renewable generation, and collective peak power penalties. To optimize the benefits for all residential prosumers, the EM problem is formulated as a Cooperative Coalition Game (CCG). The objective is to first find an energy trading policy that reduces the collective economic cost (including spot-market cost and peak-power cost) of the residential coalition, and then to distribute the profits obtained through cooperation to all residents. An MPC-based RL approach, which compensates for the shortcomings of MPC and RL and benefits from the advantages of both, is proposed to reduce the monthly collective cost despite the system uncertainties. To determine the amount of monthly electricity bill each resident should pay, we transfer the cost distribution problem into a profit distribution problem. Then, the Shapley value approach is applied to equitably distribute the profits (i.e., cost savings) gained through cooperation to all residents based on the weighted average of their respective marginal contributions. Finally, simulations are performed on a three-household microgrid system located in Oslo, Norway, to validate the proposed strategy, where a real-world dataset of April 2020 is used. Simulation results show that the proposed MPC-based RL approach could effectively reduce the long-term economic cost by about 17.5%, and the Shapley value method provides a solution for allocating the collective bills fairly.