Optimization of oxygen system scheduling in hybrid action space based on deep reinforcement learning

Abstract

The simultaneous consideration of discrete and continuous variables including equipment start-up, shutdown, oxygen output and dissipation, is required in the optimal problem of oxygen system scheduling in steel enterprises. To solve this problem involving hybrid variables, a hybrid actor-critic (HAC) algorithm is proposed in this paper. The proposed algorithm subdivides the action space into a discrete and continuous action space and evaluates the policy through the improved Q function. Besides, the correlation matrix is constructed to address the correspondence between hybrid actions. As a result, the generation of spurious gradients that lead to suboptimal action selection is prevented. To verify the effectiveness of the proposed approach, experiments with multiple groups employing the real data from steel enterprises are carried out. The results demonstrate that such a practice-based solution successfully resolves the oxygen scheduling problem and simultaneously improves the reward and algorithm convergence speed.