Reinforcement learning of room temperature set-point of thermal storage air-conditioning system with demand response

Abstract

Demand response (DR) is an effective means to reduce peak loads and enhance grid stability. Heating, ventilation, and air-conditioning (HVAC) systems have potential energy transfer characteristics and can be used as a typical flexible load for building DR. The HVAC thermostat settings are the key parameters that directly affect the elasticity of building DR and reflect the willingness of users to participate in DR. For air-conditioning DR control, the conventional method to determine thermostat settings is model-dependent, while reinforcement learning (RL) is a model-free, adaptive continuous control algorithm. Taking the proximal policy optimization RL algorithms, a neural network is used to construct a strategic framework to obtain discrete control actions, that is, thermostat settings, and a new objective function truncation method is adopted to limit the update step size and enhance the robustness of the algorithm. Thus, a TRNSYS and MATLAB joint simulation platform for the thermal storage air-conditioning system was built. This study formulated a DR strategy based on time-of-use electricity prices, which considers factors, such as environment, thermal comfort, and energy consumption; and the proposed RL algorithm is used to learn the thermostat settings in DR time. The results show that the proposed RL algorithm could realize the temperature set-point control, which saved 9.17% of the operating cost compared with a non-thermal storage air-conditioning system with a constant set-point.