Performance, robustness, and portability of imitation-assisted reinforcement learning policies for shading and natural ventilation control

Abstract

Space heating and cooling account for approximately half of all building-related energy consumption, emitting 3Gt of CO2 annually, or nearly 10% of the global total. Operable shading, natural ventilation, and solar heating are promising strategies for reducing these emissions, leveraging minimal mechanical energy to condition space with cool night air, cold night skies, and solar radiation. However, these strategies are under-utilized because their performance depends on rigorous coordination among their operable elements. Additionally, the individuality of such systems, and the lack of physics-based models suitable for control design, have thwarted the development of widely-applicable control strategies. To address this problem, here we develop a new data-driven strategy for the design of shading and natural ventilation controls in residential buildings using policy-based reinforcement learning (RL). To limit undesirable actions and reduce training time, we first used imitation learning to initialize RL training with expert knowledge, yielding an initial policy that reduced simulated late-spring space conditioning loads by ≥40% in 24 climatically diverse cities. This policy was then trained with RL in four cities representing Mediterranean, semi-arid, humid subtropical, and continental climates. When deployed in cities with unfamiliar yet related climates, these new policies reduced space conditioning loads by ≥50% in the humid subtropics and by ≥90% in the other three climates, showing exceptional portability. Further, their performance was unexpectedly robust to variations in dwelling orientation, glazing, internal heat gain, and air leakage. These results show the extraordinary potential of imitation-assisted RL in developing high-performance policies for dynamic passive heating and cooling control that remain effective in unfamiliar situations, removing a substantial barrier to passive systems advancement in carbon-free building operation.