Rough knowledge enhanced dueling deep Q-network for household integrated demand response optimization

Abstract

Implementing a household integrated demand response (HIDR) can be an effective solution to save energy and reduce carbon emissions in household multi-energy system (HMES). However, the HMES are subject to uncertainties, and HIDR optimization needs to automatically adapt to the uncertainty and make decisions within 15 min. Therefore, we propose an HIDR optimization method combining rough knowledge and a dueling deep Q-network (DDQN). Firstly, we propose an HIDR optimization framework of rough knowledge and DDQN fusion, in which knowledge is utilized to generate good DDQN samples and serves as action guidance value input to DDQN. Secondly, we formulate the household equipment models and knowledge rules. Subsequently, we design the HIDR optimization algorithm that incorporates rough knowledge into DDQN, focusing on dynamic probability of knowledge sample participation in DDQN learning, knowledge serving as DDQN action adviser, knowledge diversification and dynamic adjustment of DDQN’s random exploration probability. Simulation results show that our method saves energy costs by 5.6% and 0.9% compared to rule-based methods and DDQN, respectively. Additionally, based on an i5-10300H CPU, our method’s convergence time is no more than 13 min even for scheduling interval of 15 min. The average convergence time of our method is 29% of DDQN.