A novel energy management method based on Deep Q Network algorithm for low operating cost of an integrated hybrid system

Abstract

For minimizing the operational cost of an integrated hybrid system, which provides electricity, heating, cooling and water of a residential complex, a new energy management method presented here. The proposed energy management method uses the principles of Reinforcement Learning (RL) based on Deep Q Network (DQN) algorithm to find the operational strategy for the hybrid system. Unlike the stochastic presentation of RL with Markov Decision Process (MDP) in literature, a deterministic approach proposed in this study for energy management of a hybrid system. This presented approach does not require online trial and error or historical data of system performance to determine the operational strategy. This developed RL-DQN method used for energy management of a hybrid system with photovoltaic collectors, ET collectors, wind turbines, CHP units, a water heater, a hot water storage tank and batteries. Thus for a residential complex with variable electrical and thermal loads during a year (8760 h), RL-DQN energy management method estimated two unknown variables in operational strategy, i.e., the partial load of the CHP unit and the amount of heat generation by LPG-fueled water heater. The results of RL-DQN energy management also compared with results of two rule-based methods. The results show a reduction of the operating cost to 10983 $/year by RL-DQN which is lower 6.5% and 30.4% than that for two above mentioned rule-based methods. Lower CO2 production (2.8% and 62.8%) also observed in comparison with that for two above mentioned rule based methods. With the proposed EM method, 80.1% of the required electricity supplied with renewable energies and 82.2% of the thermal load supplied without direct consumption of fossil fuels. The benefit of using predicted load and renewable energies of future hours also investigated for EM. Results show that employing the predicted information decrease the operational cost by about 3.5 percent