Electric Water Heater Control Through Informed Fitted Q-Iteration

Abstract

The scheduling of Thermostatically Controlled Loads (TCLs) in a residential demand response (DR) context requires scalability as TCLs come in many forms. Model-free control algorithms, such as Fitted Q-iteration (FQI), can provide such scalability. They accommodate the variety in e.g. electric water heaters or heat pumps available on the market and thermal characteristics of residential buildings. However, these approaches require a significant amount of data. In the building simulation literature, Model Predictive Control (MPC) with grey-box models provides a technique that connects domain knowledge with experimental data, which results in algorithms with a higher sample efficiency. This work proposes to combine grey-box MPC and FQI in the Informed FQI method and shows how to use MPC to provide domain knowledge to the model-free FQI algorithm. The new approach is compared to FQI and MPC with a linear RC-model in terms of cost and user comfort.