Integrating few-shot personalized thermal comfort model and reinforcement learning for HVAC demand response optimization

Abstract

Heating, ventilation, and air conditioning (HVAC) is the most common equipment to maintain personalized thermal comfort (PTC), and a valuable demand response (DR) resource. The challenges to be addressed in HVAC DR optimization include uncertainties of environment, PTC varying by users and rapid decision-making. This paper proposed a fast HVAC DR optimization method that integrates deep reinforcement learning (DRL) and few-shot PTC model. Firstly, we established an HVAC DR optimization framework integrating few-shot PTC model and DRL, in which the few-shot PTC model was utilized to generate user's PTC temperature range served as the constraint and state variable of DRL. Secondly, we proposed a few-shot PTC model, which only required several field samples and had excellent engineering usability. Thirdly, we designed the HVAC DR optimization algorithm considering user's PTC based on proximal policy optimization, which automatically dealt with the uncertainties and made rapid decisions for setting HVAC setpoints of temperature dynamically. Simulation results show that the few-shot PTC model only requires 9 field samples, which is less than 1 % of existing advanced methods. Furthermore, the proposed method converges within 12.6 min under uncertainties on an i7-12700 processor, which is only about 1/2 of the standard deep deterministic policy gradient.