Dynamic energy conversion and management strategy for an integrated electricity and natural gas system with renewable energy: Deep reinforcement learning approach

Abstract

With the application of advanced information technology for the integration of electricity and natural gas systems, formulating an excellent energy conversion and management strategy has become an effective method to achieve established goals. Differing from previous works, this paper proposes a peak load shifting model to smooth the net load curve of an integrated electricity and natural gas system by coordinating the operations of the power-to-gas unit and generators. Moreover, the study aims to achieve multi-objective optimization while considering the economy of the system. A dynamic energy conversion and management strategy is proposed, which coordinates both the economic cost target and the peak load shifting target by adjusting an economic coefficient. To illustrate the complex energy conversion process, deep reinforcement learning is used to formulate the dynamic energy conversion and management problem as a discrete Markov decision process, and a deep deterministic policy gradient is adopted to solve the decision-making problem. By using the deep reinforcement learning method, the system operator can adaptively determine the conversion ratio of wind power, power-to-gas and gas turbine operations, and generator output through an online process, where the flexibility of wind power generation, wholesale gas price, and the uncertainties of energy demand are considered. Simulation results show that the proposed algorithm can increase the profit of the system operator, reduce wind power curtailment, and smooth the net load curves effectively in real time.