The application of heterogeneous graph neural network and deep reinforcement learning in hybrid flow shop scheduling problem

Abstract

The hybrid flow shop scheduling problem (HFSP) is a fundamental optimization problem in the process industries. HFSP needs to consider both job selection and machine allocation. The existing HFSP solution methods represented by deep reinforcement learning (DRL) are challenging to extract complex state features and lack certain adaptability in dealing with large-scale HFSP. To solve HFSP with a large scale and relatively strong flexibility, we propose an end-to-end DRL architecture based on the heterogeneous graph neural network to learn an optimal scheduling policy. First of all, by adding a set of machine nodes, the disjunctive graph of the traditional flexible scheduling problem is extended and transformed into a heterogeneous graph, which intuitively represents the allocation relationship between operations and machines, and it is regarded as the scheduling state at different times; secondly, a new HFSP heterogeneous graph neural network (HGNN) algorithm is established to embed potential relationships between different types of nodes and edges, and an embedding of the entire graph is generated; finally, we use the graph-level representation vectors of heterogeneous graphs as input to the proximal policy optimization (PPO) algorithm for learning the optimal scheduling policy. The experimental results show that compared with several well-known heuristic algorithms, the scheduling policy obtained by the architecture through learning can have good generalization performance in simple and complex problems, respectively, and has a relatively high solution efficiency.