Meta-Reinforcement Learning-Based Transferable Scheduling Strategy for Energy Management

Abstract

In Home Energy Management System (HEMS), the scheduling of energy storage equipment and shiftable loads has been widely studied to reduce home energy costs. However, existing data-driven methods can hardly ensure the transferability amongst different tasks, such as customers with diverse preferences, appliances, and fluctuations of renewable energy in different seasons. This paper designs a transferable scheduling strategy for HEMS with different tasks utilizing a Meta-Reinforcement Learning (Meta-RL) framework, which can alleviate data dependence and massive training time for other data-driven methods. Specifically, a more practical and complete demand response scenario of HEMS is considered in the proposed Meta-RL framework, where customers with distinct electricity preferences, as well as fluctuating renewable energy in different seasons are taken into consideration. An inner level and an outer level are integrated in the proposed Meta-RL-based transferable scheduling strategy, where the inner and the outer level ensure the learning speed and appropriate initial model parameters, respectively. Moreover, Long Short-Term Memory (LSTM) is presented to extract the features from historical actions and rewards, which can overcome the challenges brought by the uncertainties of renewable energy and the customers’ loads, and enhance the robustness of scheduling strategies. A set of experiments conducted on practical data of Australia’s electricity network verify the performance of the transferable scheduling strategy.