An adaptive control framework based on Reinforcement learning to balance energy, comfort and hygiene in heat pump water heating systems

Abstract

A major challenge in the operation of water heating systems lies in the highly stochastic nature of occupant behavior in hot water use, which varies over different buildings and can change over the time. However, the current operational strategies of water heating systems are detached from occupant behavior, and follow a conservative and energy intensive approach to ensure the availability of hot water any time it is demanded. This paper proposes a Reinforcement learning-based control framework which can learn and adapt to the occupant behavior of each specific building and make a balance between energy use, occupant comfort and water hygiene. The proposed framework is compared to the conventional approach using the real-world measurements of hot water use behavior in a single family residential building. Although the monitoring campaign has been executed during home lockdown due to COVID-19, when the occupants exhibited a very different schedule and water use related behavior, the proposed framework has learned the occupant behavior over a relatively short period of 8 weeks and provided 24.5% energy use reduction over the conventional approach, while preserving occupant comfort and water hygiene.