Dynamic scheduling for flexible job shop based on MachineRank algorithm and reinforcement learning

Abstract

This paper investigates the Dynamic Flexible Job Shop Scheduling Problem (DFJSP), which is based on new job insertion, machine breakdowns, changes in processing time, and considering the state of Automated Guided Vehicles (AGVs). The objective is to minimize the maximum completion time and improve on-time completion rates. To address the continuous production status and learn the most suitable actions (scheduling rules) at each rescheduling point, a Dueling Double Deep Q Network (D3QN) is developed to solve this problem. To improve the quality of the model solutions, a MachineRank algorithm (MR) is proposed, and based on the MR algorithm, seven composite scheduling rules are introduced. These rules aim to select and execute the optimal operation each time an operation is completed or a new disturbance occurs. Additionally, eight general state features are proposed to represent the scheduling status at the rescheduling point. By using continuous state features as the input to the D3QN, state-action values (Q-values) for each scheduling rule can be obtained. Numerical experiments were conducted on a large number of instances with different production configurations, and the results demonstrated the superiority and generality of the D3QN compared to various composite rules, other advanced scheduling rules, and standard Q-learning agents. The effectiveness and rationality of the dynamic scheduling trigger rules were also validated.