A reinforcement learning approach for control of window behavior to reduce indoor PM2.5 concentrations in naturally ventilated buildings

Abstract

Smart control of window behavior is a means of effectively reducing concentrations of indoor PM2.5 (particulate matter with aerodynamic diameter less than 2.5 μm) in naturally ventilated residential buildings without indoor air cleaning devices. This study aimed to develop a reinforcement learning approach to automatically control window behavior in real time for mitigation of indoor PM2.5 pollution. The proposed method trains the window controller with the use of a deep Q-network (DQN) in a specific naturally ventilated apartment in the course of a month. The trained controller can then be employed to control window behavior in order to reduce the indoor PM2.5 concentrations in that apartment. The required input data for the controller are the real-time indoor and outdoor PM2.5 concentrations with a 1-min resolution, which can easily be obtained with low-cost sensors available on the market. A series of simulations were conducted in a virtual typical apartment in Beijing and a real apartment in Tianjin. The results show that, compared with the baseline I/O ratio algorithm, the proposed reinforcement learning window-control algorithm reduced the average indoor PM2.5 concentration by 12.80% in a one-year period. Furthermore, the proposed algorithm reduced the indoor PM2.5 concentrations in the real apartment by 9.11% when compared with the I/O ratio algorithm and by 7.40% when compared with real window behavior.