A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems

Abstract

Buildings are one of the main drivers of global energy consumption and CO2 emissions. Efficient energy management systems will have to integrate renewable energy sources with heating and/or cooling to mitigate climate change. In this study, we analyze the potential of deep reinforcement learning (DRL) to control a smart home with a modulating air-to-water heat pump, a photovoltaic system, a battery energy, and a thermal storage system for floor heating and hot water supply. We transform a mixed-integer linear program (MILP) into a DRL implementation. In our numerical analysis, we compare our results based on the deep deterministic policy gradient (DDPG) algorithm to the theoretical upper bound of the model predictive control (MPC) result under full information, as well as a practice-oriented rule-based benchmark. We show that our proposed DRL implementation outperforms the rule-based approach and achieves a self-sufficiency of 75% with only limited comfort violations. Analyzing different DRL formulations, we conclude that domain knowledge is key to formalizing an efficient decision problem with stable results. Our input data and models, developed using the Julia programming language, are available open source.