Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

Abstract

The flexible job shop scheduling problem (FJSP) is an important production scheduling problem in intelligent manufacturing. How to model the complex FJSP more accurately and improve the efficiency and generalization of scheduling policies is an urgent challenge to be solved. Therefore, a new end-to-end deep reinforcement learning (DRL) method combined with the meta-path-based heterogeneous graph neural network (MHGNN) is proposed to effectively solve FJSP. The task of solving FJSP is decomposed into two subtasks of operation selection and machine allocation. This dual-task FJSP is represented by introducing a heterogeneous graph and modeled as the dual Markov decision process (DMDP). A MHGNN is proposed to embed the global scheduling states of the dual-task FJSP. A heterogeneous graph neural network (GNN) framework is designed for the dual-task FJSP to efficiently encode the operation nodes and machine nodes, where the extracted embedded feature information is used to represent the operation selection policy and the machine allocation policy. Two policy networks are designed to efficiently predict the operation selection policy and the machine allocation policy, and a soft double-actors critic algorithm is proposed to train these two policy networks, where the trained policies can be used to efficiently solve FJSP instances of different scales. The experiments on three public benchmarks show that the proposed method outperforms the well-known heuristic scheduling rules and some advanced methods for solving FJSP. In particular, when solving large-scale FJSPs, the proposed method performs more prominently in terms of solution quality and solution time.