Intelligent control of district heating system based on RDPG

Abstract

Given the continuous expansion of heating areas in recent years, the design of a precise and dependable district heating system (DHS) has become increasingly crucial. Traditional control decisions are made based on real-time environmental temperature feedback, often leading to uneven heating on the user side and affecting residents' comfort. This paper proposes an intelligent control strategy based on the deep reinforcement learning recurrent deterministic policy gradient (RDPG) algorithm for DHSs. To explore the control performance of the RDPG algorithm on DHS, we have meticulously modeled the pivotal components of the DHSs, namely plate heat exchangers, secondary heating pipe networks, and heat users. Moreover, taking into account the periodic factors in heating regulation, the traditional recurrent neural network (RNN) in the recurrent deterministic policy gradient (RDPG) algorithm has been replaced with the long short-term memory (LSTM) network. The proposed algorithm was trained using actual data from a heat exchange station in Tianjin and compared with reinforcement learning algorithms such as TD3, DPPO, DDPG, and A3C in terms of training rewards, effectiveness, and training stability. The results of the models are evaluated and visualized. Experimental results show that the proposed control method based on the RDPG algorithm, compared to other control schemes, can achieve the highest training reward and the most stable control performance, with an indoor temperature fluctuation range of only 0.1 °C.