Data-driven stochastic energy management of multi energy system using deep reinforcement learning

Abstract

The multi energy system (MES) is promising in the process of carbon neutrality, such that multi energy resources are utilized comprehensively to reduce the operation cost. Another way is to promote carbon neutrality by increasing the penetration of renewable energy. Hence, in this paper, we study the energy management of a typical MES under the challenges of stochastic renewable supplies and energy demands. To address the challenges, a stochastic optimization problem is established as a Markov decision process (MDP). An improved deep reinforcement learning (DRL) method is then developed to achieve the dynamic optimal energy dispatch. In particular, the comfort experience of users and complex coupling are both considered in the MES. In this framework, we propose an improved soft actor critic (SAC) algorithm based on maximum entropy to improve exploration ability, together with a long short-term memory (LSTM) network to extract temporal features efficiently. Meanwhile, we add the prioritized experience replay (PER) to increase the training efficiency to speed up the convergence of the algorithm. Finally, the case study demonstrates that the proposed algorithm can converge rapidly and greatly reduce the operation cost. In addition, the effectiveness and robustness of the improved method are verified.