A deep reinforcement learning-based method for predictive management of demand response in natural gas pipeline networks

Abstract

With the increase of natural gas in the world's energy consumption, the efficient and reliable management of natural gas pipeline networks is becoming even more important than before. In recent years, Demand Response (DR) is considered an effective approach for cleaner production and economic strategy, by introducing the participation of customers (CUs).This paper proposes a novel DR method for predictive management in multi-level natural gas markets with different stakeholders. This method is able to make a better trade-off among supplier's profits, gas demand volatility and CU satisfaction. This method includes three parts: dynamic pricing model, intelligent decision making and data-driven demand forecasting. A Markov decision process-based model is developed to illustrate the process of dynamical optimizing energy prices. Then, deep learning and reinforcement learning are integrated to efficiently solve the sequential decision-making problem, based on the physics constraints of natural gas pipeline networks. Besides, to realize the function of predictive optimization, an energy demand forecasting model is developed based on the deep recurrent neural network model. The proposed dynamic pricing method is able to optimize the pricing strategies in accordance to the demand patterns, and dynamically improve the system stability and energy efficiency. Finally, we apply the developed method to a natural gas network with relatively complex topology and different CUs. The results indicate that the proposed method can achieve the targets of peak shaving and valley filling under different pricing periods. Besides, the sensitivity analysis of the critical parameters in the dynamic pricing model is analyzed in detail, which can give a solid criterion for ensuring the effectiveness of this framework.