Research and Application of Predictive Control Method Based on Deep Reinforcement Learning for HVAC Systems

Chenhui Fu,Yunhua Zhang

doi:10.1109/access.2021.3114161

Chenhui Fu, Yunhua Zhang

Open Access

https://doi.org/10.1109/access.2021.3114161

Copy DOI

Journal: IEEE Access	Publication Date: Jan 1, 2021
Citations: 9	License type: CC BY 4.0

Affiliation: Zhejiang Sci-Tech University

Abstract

Energy efficiency and consumption control remain a significant topic in the area of Heating, Ventilation, and Air Conditioning (HVAC) systems. Deep reinforcement learning (DRL) is an emerging technique to optimize energy consumption. Its advantage lies in the ability to tackle the time-series nature of energy data and complexity brought by environmental factors. However, most DRL algorithms have not considered both time-of-use electricity pricing and thermal comfort. This paper proposed a hybrid approach based on twin delayed deep deterministic policy gradient algorithm and model predictive control (TD3-MPC) for HVAC systems, to mitigate function approximation errors and save cost by pre-adjusting building temperatures at off-peak times. This proposed method is compared with deep deterministic policy gradient (DDPG) algorithm under simulations of five building zones. Experiment results demonstrate that TD3-MPC outperforms DDPG algorithm and potentially saves 16% of total energy consumption cost, with better stability and robustness.

Highlights

Global climate change is a concerning issue and people are actively exploring opportunities to reduce carbon emission and mitigate energy consumption
This paper proposes an energy consumption prediction control algorithm combined with relevant strategies of energy storage for HVAC systems
The algorithm is named as TD3-MPC. This approach greatly reduces the uncertainty brought by the outdoor environment and it is suitable under different indoor air capacity settings

Summary

Introduction

Global climate change is a concerning issue and people are actively exploring opportunities to reduce carbon emission and mitigate energy consumption. In China, for example, building energy consumption accounts for 21.7% of the total national energy consumption [1]. The development of lowenergy buildings is one opportunity with great potential that draws a lot of attention. The convertibility of various energy sources in buildings as well as buildings’ energy storage capacity make it an ideal choice for energy optimization [2]. HVAC systems are key contributors to the energy consumption within buildings. It is crucial to manage such systems with effective maintenance and operation strategies

Objectives

Methods

Findings

Conclusion