Large-Scale Dynamic Scheduling for Flexible Job-Shop With Random Arrivals of New Jobs by Hierarchical Reinforcement Learning

Abstract

As the intelligent manufacturing paradigm evolves, it is urgent to design a near real-time decision-making framework for handling the uncertainty and complexity of production line control. The dynamic flexible job shop scheduling problem (DFJSP) is frequently encountered in the manufacturing industry. However, it is still challenging to obtain high-quality schedules for DFJSP with dynamic job arrivals in real-time, especially facing thousands of operations from a large-scale scene with complex contexts in an assembly plant. This paper aims to propose a novel end-to-end hierarchical reinforcement learning framework for solving the large-scale DFJSP in near real-time. In the DFJSP, the processing information of newly arrived jobs is unknown in advance. Besides, two optimization tasks, including job operation selection and job-to-machine assignment, have to be handled, which means multiple actions must be controlled simultaneously. In our framework, a higher-level layer is designed to automatically divide the DFJSP into sub-problems, i.e., static FJSPs with different scales. And two lower-level layers are constructed to solve the sub-problems. In particular, one layer based on a graph neural network is in charge of sequencing job operations, and another layer based on a multi-layer perceptron is used to assign a machine to process the job operations. Numerical experiments, including offline training and online testing, are conducted on several instances with different scales. The results verify the superior performance of the proposed framework compared with existing dynamic scheduling methods, such as well-known dispatching rules and meta-heuristics.