Development of an HVAC system control method using weather forecasting data with deep reinforcement learning algorithms

Abstract

Heating, ventilation, and air conditioning (HVAC) systems account for a significant proportion of the energy consumption of a building. With the global energy demand increasing every year, optimal HVAC control methods that reduce energy consumption while maintaining the thermal comfort of the occupants have become more important. Weather forecasting data have thus attracted significant research attention for the optimization of building energy consumption in model-based studies. However, these HVAC control approaches have failed to consider the reduction in the predictive error of the incorporated weather model, leading to performance limitations. Accordingly, model-free reinforcement learning (RL)-based HVAC systems that can be controlled dynamically have been proposed, and these have exhibited significantly better performance than model-based methods. However, RL algorithms using weather forecasting data have not yet been assessed for use in optimal HVAC control for buildings. In this study, we propose WDQN-temPER, a hybrid HVAC control method combining a deep Q-network with a novel prioritized experience replay (PER) technique (referred to as temPER) and a gated recurrent unit (GRU) model. The GRU model predicts the future outdoor temperature and this is used as a state variable in the RL model. The proposed temPER algorithm facilitates the use of samples in the RL training process with larger changes in the outdoor temperature based on the predicted temperature. We experimentally demonstrate in EnergyPlus simulations that our proposed WDQN-temPER model outperforms a rule-based baseline model in terms of HVAC control, with energy savings of up to 58.79%.