Abstract
In recent years, home energy management systems (HEMS), which enable the automatic control of electrical equipment and home appliances, have been attracting attention as a method for saving electricity at home. HEMS achieve energy saving by visualizing energy consumption at home and controlling energy consuming equipment such as air conditioners. The optimum control law is difficult to attain, owing to uncertainties related to power demand and power supply from the electrical equipment. Deep reinforcement learning has been used to address energy optimization problems for home environments. However, in HEMS, several components such as heating, ventilation, and air conditioning (HVAC) systems, storage batteries, and electric water heaters are simultaneously controlled, and therefore, the action space becomes extremely large. Therefore, it may not be feasible to fully learn the rare experience using traditional deep reinforcement learning methods due to the large size of the state-action space and slow propagation of delayed rewards. In this study, we propose an energy management algorithm that uses the Dual Targeting Algorithm to strongly learn the experience of acquiring high returns using the quick propagation of delayed rewards via multistep returns. The proposed energy management algorithm is applied to a HEMS learning experiment to control a storage battery and an HVAC system, and its performance is compared to that of a Deep Deterministic Policy Gradient-based energy management system. As a result, it is confirmed that the proposed method can reduce the number of hours deviating from the comfort temperature range by about 17% compared to the existing method.
Highlights
In recent years, there have been many attempts to save energy by visualizing the amount of energy consumption at home and controlling energy consuming equipment such as air conditioners
The behavior can be confirmed from the behavior of recovering after the learning performance is sometimes significantly reduced in the learning curve of the exploitationoriented DDPG (ExDDPG)-based energy management algorithm, as shown in fig. 3
Since the use of eligibility trace [23] and the use of arbitrary n-step return [24] have been proposed as a deep reinforcement learning method using multistep returns, a comparison between the ExDDPG-based energy management algorithm and energy management algorithms using these methods will be required in the future works
Summary
There have been many attempts to save energy by visualizing the amount of energy consumption at home and controlling energy consuming equipment such as air conditioners. We propose an exploitationoriented DDPG (ExDDPG)-based energy management algorithm that strongly reinforces the experience of high daily returns by introducing the DTA into the DDPG-based energy management algorithm. In this algorithm, in learning for each state-action pair, when the multistep returns until the end of the day (that is 24 o’clock) is higher than the 1step returns, the multistep returns are used. We apply to the ExDDPG-based energy management algorithm to HEMS, which controls the ESS and HVAC systems, and verify its effectiveness in maintaining a comfortable room temperature and reducing electricity charges compared to the DDPG-based energy management algorithm. We detail the models used for the storage battery and the HVAC system and formulate sequential decision-making problems as Markov decision processes (MDPs)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
