Safe reinforcement learning method integrating process knowledge for real-time scheduling of gas supply network

Abstract

Gas supply networks play a crucial role in steel enterprises because they provide downstream customers with the gas required for production. In this paper, the real-time scheduling problem of a multi-product gas supply network is first modeled under the framework of reinforcement learning (RL). A safe RL method with prediction and safeguard modules is further developed by utilizing the process knowledge from the gas supply network. The prediction module is designed to predict state changes, and the safeguard module is developed to judge and replace dangerous actions according to current and predicted states. In order to avoid repeated dangerous actions, the safeguard module will provide a negative reward to the agent as feedback whenever a dangerous action is replaced. This active prediction-safeguard strategy is beneficial for reducing trial-and-error costs, speeding up training, and running online. Finally, case studies are implemented on an actual gas supply network to demonstrate the advantages of the proposed method.