Cross temporal-spatial transferability investigation of deep reinforcement learning control strategy in the building HVAC system level

Abstract

Model free based DRL control strategies have achieved positive effects on the HVAC system optimal control. However, developing deep reinforcement learning (DRL) control strategies for different building HVAC systems is time-consuming and laborious. To address this issue, this study proposes a transfer learning and deep reinforcement learning (TL-DRL) integrated framework to achieve the DRL control strategy transfer in the building HVAC system level. Deep Q-learning (DQN) is first pre-trained in the source building until it converges to an optimal strategy. Then, the well pre-trained DQN parameters of the first few layers are transferred to the target DQN. Finally, the target DQN parameters of the last few layers are fine-tuned in the target building. An EnergyPlus-Python co-simulation testbed is developed to investigate the cross temporal-spatial transferability of DQN control strategy in the building HVAC system level. Results indicate that the proposed TL-DRL framework can effectively improve the training efficiency of control strategy by about 13.28% when transferring the first two layers compared to that of the DRL baseline models trained from scratch, while simultaneously maintaining energy consumption and indoor air temperature in an acceptable range. The proposed TL-DRL framework provides a preliminary direction for the scalability of intelligent HVAC control strategies.