An end-to-end deep reinforcement learning method based on graph neural network for distributed job-shop scheduling problem

Abstract

Distributed Job-shop Scheduling Problem (DJSP) is a hotspot in industrial and academic fields due to its valuable application in the real-life productions. For DJSP, the available methods aways complete the job selection first and then search for an appropriate factory to assign the selected job, which means job selection and job assignment are done independently. This paper proposes an end-to-end Deep Reinforcement Learning (DRL) method to make the two decisions simultaneously. To capture the problem characteristics and realize the objective optimization, the Markov Decision Process (MDP) of DJSP is formulated. Specialised action space made up of operation-factory pairs is designed to achieve the simultaneous decision-making. A stitched disjunctive graph representation of DJSP is specially designed, and a Graph Neural Network (GNN) based feature extraction architecture is proposed to dig the state embedding during problem solving. A Proximal Policy Optimization (PPO) method is applied to train an action-selection policy. To further lead the agent to assign jobs to the factory with smaller makespan, a probability enhancement mechanism is designed. The experimental results on 240 test instances have shown that the proposed method outperforms 8 classical Priority Dispatching Rules (PDRs), 3 closely-related RL methods and 5 metaheuristics in terms of effectiveness, stability and generalization.