Towards comfortable and cost-effective indoor temperature management in smart homes: A deep reinforcement learning method combined with future information

Abstract

The extension of in-home time has boosted people's increase demand for comfortable indoor temperatures. This paper aims to propose a smart home-based temperature control framework based on deep learning and reinforcement learning to automatically control indoor temperature. The heat pump power and the ventilation system volume could be adjusted by the proposed framework to minimize costs while maintaining indoor comfort. The uncertainty of future information and the requirement for continuous control make it challenging to build an intelligent temperature control system. Simultaneously, the human body is sensitive to different temperatures. Therefore, the selection of appropriate evaluation criteria for people’s thermal comfort is essential. To address this problem, the Predicted Mean Vote (PMV) criterion is applied to scientifically evaluate human thermal comfort. The indoor temperature control problem is transformed into a Markov decision process. A deep learning framework (Proximal Policy Optimization PPO) based on continuous control actions is proposed to solve this problem. Combined with the deep learning method Long Short-Term Memory (LSTM), future state information is applied to improve the control stability. The proposed framework performs well under different conditions in summer and winter. The proposed framework performs well under different conditions in summer and winter. It is compared by the discrete action control method Deep Q Network and the PPO algorithm, which does not contain future state information. As a result, the proposed framework achieves 24.29% and 23.63% cost savings in winter and summer, respectively.