Priori knowledge-based deep reinforcement learning control for fan coil unit system

Abstract

Controlling the water pump and fan in the heating, ventilation and air-conditioning (HVAC) system is crucial for achieving optimal performance of the fan coil unit (FCU) systems. In this paper, the researchers investigate the control problem of the FCU system under the model-free assumption. Specifically, the researchers innovatively model the control problem of the FCU system as a Markov decision process (MDP) and optimize the FCU control using reinforcement learning (RL) methods. However, exploration strategies in the early stage of control using RL methods can lead to poor control performance. To address this, a novel deep deterministic policy gradient control method based on the priori knowledge (DDPG-PK) for the FCU system is proposed, which combines priori knowledge and deep reinforcement learning (DRL) algorithms, following human experience to avoid bad control behavior and achieve better policy control. To validate the performance of the proposed algorithm, the researchers build a variable operating conditions simulation model based on the actual parameters of the real FCU system and related historical data on a general thermodynamic basis. Experimental results show that, compared to other benchmark methods, the DDPG-PK method improves the FCU system's 27.5% satisfaction rate and 23.0% energy efficiency rate in the first three years, and can still keep a state-of-the-art performance in following years.