Efficient model-free control of chiller plants via cluster-based deep reinforcement learning

Abstract

Optimizing the operation of chiller plants can reduce energy consumption in heating, ventilation and air conditioning (HVAC) systems and is one of the effective ways to achieve “carbon neutrality” in buildings. Reinforcement learning (RL) demonstrates great potential in this field. However, in real applications, RL continues to face challenges in dealing with extensive state spaces and achieving stable convergence. The main challenge lies in the high coupling and complexity of chiller plant control, making it difficult for RL agents to finely optimize each state during training, resulting in suboptimal control effectiveness. Additionally, as the state space expands, the computation cost and training duration also rise greatly. To address these issues, this paper proposes a clustering-based RL control method. Firstly, the state is divided into more homogeneous subsets by the K-means algorithm. Then, different RL agents are trained based on these subsets, and a public experience pool is established between neighboring agents to enhance the experience exchange. This design makes the training process more context-oriented, enabling agents to learn from similar feature states and reducing the issue of decreased boundary state handling due to state dividing. The experimental results demonstrate that the proposed method achieves 10.1% increase in energy-saving compared to rule-based control, and is close to that of the model-based control. (1.2% difference). Through state clustering, both the learning speed and control effectiveness of the agent are significantly improved. Compared to non-clustered RL, the proposed method reduces the training time by 66.7% and improves the energy saving by 0.9%.