End-to-End Heat-Pump Control Using Continuous Time Stochastic Modelling and Uppaal Stratego

Abstract

Heatpump-based floor-heating systems for domestic heating offer flexibility in energy-consumption patterns, which can be utilized for reducing heating costs—in particular when considering hour-based electricity prices. Such flexibility is hard to exploit via classical Model Predictive Control (MPC), and in addition, MPC requires a priori calibration (i.e. model identification) which is often costly and becomes outdated as the dynamics and use of a building change. We solve these shortcomings by combining recent advancements in stochastic model identification and automatic (near-)optimal controller synthesis. Our method suggests an adaptive model-identification using the tool CTSM-R, and an efficient control synthesis based on Q-learning for Euclidean Markov Decision Processes via Uppaal Stratego. On a virtual Danish family-house from the OpSys project, we demonstrate up to 33% reduction in heating cost while retaining comparable comfort to a standard bang-bang controller. Furthermore, we show the flexibility of our method by computing the Pareto-frontier that visualizes the cost/comfort tradeoff.