Fusing domain knowledge and reinforcement learning for home integrated demand response online optimization

Abstract

Electricity–gas integrated household energy systems (HESs) expose obvious system uncertainties, requiring their integrated demand response (IDR) programs to be able to adapt automatically and quickly to system changes. Deep Reinforcement Learning (DRL) methods, though having been proven promising to tackle such problems, are typically not efficient in learning from random explorations. This paper proposes a method based on DRL with HESIDR knowledge penetration. By interpreting the domain IDR knowledge as a set of control rules, the DRL agent gains learning samples from knowledge-based exploration in addition to the traditional exploration–exploitation tradeoff. Correspondingly, we develop a cooperation scheme for action selection and replay sampling, which is based on exponential probability functions to balance the penetration of knowledge-based exploration, random exploration and policy exploitation. We conduct case studies in a typical home multi-energy system environment. After determining parameters in the exponential probability functions, the learning and cost reduction performance of the proposed algorithm was tested. The results show that the method proposed in the present study spends 48.78% less training time than the standard DQN, which enables few-minute optimization on a lightweight PC. Also, the proposed method can further reduce energy bills by 26.17% compared to the uncontrolled scenario and by 9.88% to the integrated rule-based controller.