Imitation Learning-Based Energy Management Algorithm: Lille Catholic University Smart Grid Demonstrator Case Study

Abstract

This paper proposes a novel energy management approach (imitation-Q-learning) based on imitation learning (IL) and reinforcement learning (RL). The proposed approach reinforces a decision-making agent based on a modified Q-learning algorithm to mimic an expert demonstration to solve a microgrid (MG) energy management problem. Those demonstrations are derived from solving a set of linear programming (LP) problems. Consequently, the imitation-Q-learning algorithm learns by interacting with the MG simulator and imitating the LP demonstrations to make decisions in real time that minimize the MG energy costs without prior knowledge of uncertainties related to photovoltaic (PV) production, load consumption, and electricity prices. A real-scale MG at the Lille Catholic University in France was used as a case study to conduct experiments. The proposed approach was compared to the expert performances, which are the LP algorithm and the conventional Q-learning algorithm in different test scenarios. It was approximately 80 times faster than conventional Q-learning and achieved the same performance as LP. In order to test the robustness of the proposed approach, a PV inverter crush and load shedding were also simulated. Preliminary results show the effectiveness of the proposed method.