Multi-objective reinforcement learning framework for dynamic flexible job shop scheduling problem with uncertain events

Abstract

The economic benefits for manufacturing companies will be influenced by how it handles potential dynamic events and performs multi-objective real-time scheduling for existing dynamic events. Based on these, we propose a new dynamic multi-objective flexible job shop scheduling problem (DMFJSP) to simulate realistic production environment. Six dynamic events are involved in the problem including job insertion, job cancellation, job operation modification, machine addition, machine tool replacement and machine breakdown. As well as three objectives of longest job processing time (makespan), average machine utilization and average job processing delay rate with a set of constraints are also raised in the study. Then this research designs a novel dynamic multi-objective scheduling algorithm based on deep reinforcement learning. The algorithm uses two deep Q-learning networks and a real-time processing framework to process each dynamic event and generate complete scheduling scheme. In addition, an improved local search algorithm is adopted to further optimize the scheduling results and the idea of combination is used to make the scheduling rules more comprehensive. Experiments on 27 instances show the superiority and stability of our approach compared to each proposed combined rule, well-known scheduling rules and standard deep Q-learning based algorithms. Compared to the current optimal deep Q-learning method, the maximum performance improvement for our three objectives are approximately 57%, 164% and 28%.