MBRL-MC: An HVAC Control Approach via Combining Model-Based Deep Reinforcement Learning and Model Predictive Control

Abstract

This article proposes a novel learning-based control strategy, named MBRL-MC, for the heating, ventilation, and air conditioning (HVAC) system by combining model-based deep reinforcement learning (DRL) and model predictive control (MPC). First, a thermal dynamic model of the zone is learned by a supervised learning algorithm. Based on the learned model, a neural network (NN) planning framework is designed which leverages the ideas of both reinforcement learning (RL) and MPC. The proposed planning algorithm is directly obtained without imitating the results of MPC random shooting, which avoids the compounding error during the learning procedure. In addition, the bootstrapping technique is not employed by our algorithm when constructing the update target of the critic network, which improves the learning stability. The cross-entropy method is further used to augment the RL algorithm in order to avoid potential divergence. Finally, simulation experiments in the EnergyPlus environment demonstrate the effectiveness of the proposed algorithm. The comparisons with the existing algorithms show the advantages of MBRL-MC.