Decentralized multi-agent based energy management of microgrid using reinforcement learning

Abstract

This paper proposes a multi-agent based decentralized energy management approach in a grid-connected microgrid (MG). The MG comprises of wind and photovoltaic resources, diesel generator, electrical energy storage, and combined heat and power generations to serve electrical and thermal loads at the lower-level of energy management system (EMS). All distributed energy resources (DERs) and customers are modelled as self-interested agents who adopt reinforcement learning to optimize their behaviours and operation costs. Based on this algorithm, agents have the capability to interact with each other in a distributed manner and find the best strategy in competitive environment. At the upper-level of EMS, there is an energy management agent that gathers the information of agents of lower-level and clears the MG electrical and thermal energy market in line with predetermined goals. Utilizing energy availability from different DERs and variety of customers’ consumption patterns, considering uncertainty of renewable generation and load consumption and taking into account technical constraint of DERs are the strengths of the presented framework. Performance of the proposed algorithm is investigated under different conditions of agents learning and using ε-greedy, soft-max and upper confidence bound methods. The simulation results verify efficacy of the proposed approach.