A comparative study of DQN and D3QN for HVAC system optimization control

Abstract

Ensuring the optimal performance of Heating, Ventilation, and Air Conditioning (HVAC) systems is paramount for achieving energy efficiency. This paper investigates the application of deep reinforcement learning algorithms in HVAC system control, aiming to identify the most suitable Q-network structure for optimizing HVAC systems and comparing the performance of Deep Q-learning (DQN) and Double Dueling Deep Q-learning (D3QN) algorithms. Initially, this paper evaluates and analyses existing literature to perform a normalization treatment on the state space. Through systematic simulation and rigorous data analysis, the impact of the Q-network structure on the efficacy of the DQN and D3QN algorithms is evaluated, resulting in the proposal of specific values for the Q-network structures within these two algorithms. Subsequently, comparisons are drawn on the optimization effectiveness, stability and reliability of these algorithms across diverse engineering projects. Results highlight the superiority of the D3QN algorithm over the DQN algorithm regarding both optimization effectiveness and stability across all evaluated projects. The proposed efficient Q-network structure comprises two hidden layers, with 64 and 12 neurons respectively in each layer. The findings of this paper are crucial in providing insights for HVAC systems control optimization using reinforcement learning and pave the way for advanced research and applications in the future.