CntrlDA: A building energy management control system with real-time adjustments. Application to indoor temperature

Abstract

Rule-Based Control (RBC) and Model Predictive Control (MPC) have been traditionally used to control building heating, ventilation and air conditioning (HVAC) systems. They, however, present shortcomings when faced with efficiently controlling these systems at a larger level. Reinforcement Learning (RL) has recently emerged as a viable alternative, showing promising results compared to previous methods, but still having some difficulties with untrained situations or sudden changes. CntrlDA is our proposal on improving the RL formulation by coupling it with data assimilation (DA), a technique commonly used in numerical weather prediction. Our battery of experiments, in a building simulation environment, shows that training a RL control agent with DA and external data, leads to better performance than training the agent using only the simulation data. The RL control agent with DA maintains the temperature range 15.6% more often than the RL control agent without DA. It is also shown that by including a DA stage in the control process, the agent better deals with unexpected events (which are common in real-life systems and particularly in building energy control scenarios). We show that it maintains the range 15.4% more often than the system without DA with no significant added cost of resources.