Job Shop Scheduling Problem Research Articles

Dynamic scheduling mechanism for intelligent workshop with deep reinforcement learning method based on multi-agent system architecture

With the development and changes of industry and market demand, the personalized customization production mode with small batch and multiple batches has gradually become a new production mode. This makes production environment become more complex and dynamic. However, traditional production workshops cannot effectively adapt to this environment. Combining with new technologies, transforming traditional workshops into intelligent workshop to cope with new production mode become an urgent problem. Therefore, this paper proposes a multi-agent manufacturing system based on IoT for intelligent workshop. Meanwhile, this paper takes flexible job shop scheduling problem (FJSP) as a specific production scenario and establishes relevant mathematics model. To build the agent in intelligent workshop, this paper proposes a data-based with combination of virtual and physical agent (DB-VPA) which has information layer, software layer and physical layer. Then, based on the manufacturing system, this paper designs a dynamic scheduling mechanism with deep reinforcement learning (DRL) for intelligent workshop. This method contains three aspects: (1) Modeling production process based on Markov decision process (MDP). (2) Designing communication mechanism for DB-VPAs. (3) Designing scheduling model combining with improved genetic programming and proximal policy optimization (IGP-PPO) which is a DRL method. Finally, relevant experiments are executed in a prototype experiment platform. The experiments indicate that the proposed method has superiority and generality in solving scheduling problem with dynamic events.

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

In response to the challenges of dynamic adaptability, real-time interactivity, and dynamic optimization posed by the application of existing deep reinforcement learning algorithms in solving complex scheduling problems, this study proposes a novel approach using graph neural networks and deep reinforcement learning to complete the task of job shop scheduling. A distributed multi-agent scheduling architecture (DMASA) is constructed to maximize global rewards, modeling the intelligent manufacturing job shop scheduling problem as a sequential decision problem represented by graphs and using a Graph Embedding–Heterogeneous Graph Neural Network (GE-HetGNN) to encode state nodes and map them to the optimal scheduling strategy, including machine matching and process selection strategies. Finally, an actor–critic architecture-based multi-agent proximal policy optimization algorithm is employed to train the network and optimize the decision-making process. Experimental results demonstrate that the proposed framework exhibits generalizability, outperforms commonly used scheduling rules and RL-based scheduling methods on benchmarks, shows better stability than single-agent scheduling architectures, and breaks through the instance-size constraint, making it suitable for large-scale problems. We verified the feasibility of our proposed method in a specific experimental environment. The experimental results demonstrate that our research can achieve formal modeling and mapping with specific physical processing workshops, which aligns more closely with real-world green scheduling issues and makes it easier for subsequent researchers to integrate algorithms with actual environments.

A matheuristic-based multi-objective evolutionary algorithm for flexible assembly jobs shop scheduling problem in cellular manufacture

The flexible assembly job shop scheduling problem (FAJSP) and its extensions have received increased attention. However, multi-objective FAJSP in cellular manufacture (FAJSP-CM), a real-world extension of FAJSP with important applications in the mass customization of complex products, is barely considered in most previous studies. Hence, this study first addresses the multi-objective FAJSP-CM problem via a mixed-integer linear programming model (MILP). Then, a matheuristic-based multi-objective evolutionary algorithm (MMOEA), with a matheuristic decoding method, knowledge-guided initialization and learning-based local intensification method, is presented to efficiently tackle the FAJSP-CM problem in polynomial-time. The matheuristic decoding method, which hybrids a decomposition linear programming model (D-LP) that can quickly and optimally allocate the execution times of all operations, to search for the optimal solution in a specific space. The knowledge-guided initialization is extended to get a high-quality and great-diversity initial population for accelerated convergence. Meanwhile, the learning-based local intensification method is developed to adaptively coordinate five problem-specific local search operators to enhance the exploitation ability of the MMOEA algorithm. Finally, comprehensive experiments based on 720 instances are conducted to evaluate the proposed MILP model and MMOEA algorithm. The results of the first experiments illustrate the MILP model can obtain the Pareto solutions of small-scale instances. The remaining experiments demonstrate that the proposed MMOEA algorithm outperforms its competitors by 0.21 and 0.72 in hypervolume ratio and coverage indictors, along with approximately a 25 % improvement in success rate.

Multi-policy deep reinforcement learning for multi-objective multiplicity flexible job shop scheduling

This study considers the simultaneous minimization of makespan and total tardiness for the multi-objective multiplicity flexible job shop scheduling problem (MOMFJSP). A deep reinforcement learning framework employing a multi-policy proximal policy optimization algorithm (MPPPO) is developed to solve MOMFJSP. The MOMFJSP is treated as a Markov decision process, allowing an intelligent agent to make sequential decisions based on the current production status. This framework involves multiple policy networks with different objective weight vectors. Using MPPPO, these networks are optimized simultaneously to obtain a set of high-quality Pareto-optimal policies. Moreover, a fluid model is introduced to extract state features and devise composite dispatching rules as discrete actions. A multi-policy co-evolution mechanism (MPCEM) is proposed to facilitate collaborative evolution among policy networks, supported by a reward mechanism that considers the objective weights. A training algorithm based on MPPPO is designed for learning across multiple policy networks. The effectiveness and superiority of the proposed method are confirmed through comparisons with composite dispatching rules and other scheduling methods.

A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem With Crane Transportation and Setup Times.

Flexible job shop scheduling problem (FJSP) has attracted research interests as it can significantly improve the energy, cost, and time efficiency of production. As one type of reinforcement learning, deep Q-network (DQN) has been applied to solve numerous realistic optimization problems. In this study, a DQN model is proposed to solve a multiobjective FJSP with crane transportation and setup times (FJSP-CS). Two objectives, i.e., makespan and total energy consumption, are optimized simultaneously based on weighting approach. To better reflect the problem realities, eight different crane transportation stages and three typical machine states including processing, setup, and standby are investigated. Considering the complexity of FJSP-CS, an identification rule is designed to organize the crane transportation in solution decoding. As for the DQN model, 12 state features and seven actions are designed to describe the features in the scheduling process. A novel structure is applied in the DQN topology, saving the calculation resources and improving the performance. In DQN training, double deep Q-network technique and soft target weight update strategy are used. In addition, three reported improvement strategies are adopted to enhance the solution qualities by adjusting scheduling assignments. Extensive computational tests and comparisons demonstrate the effectiveness and advantages of the proposed method in solving FJSP-CS, where the DQN can choose appropriate dispatching rules at various scheduling situations.

Dynamic Intelligent Scheduling in Low-Carbon Heterogeneous Distributed Flexible Job Shops with Job Insertions and Transfers.

With the rapid development of economic globalization and green manufacturing, traditional flexible job shop scheduling has evolved into the low-carbon heterogeneous distributed flexible job shop scheduling problem (LHDFJSP). Additionally, modern smart manufacturing processes encounter complex and diverse contingencies, necessitating the ability to address dynamic events in real-world production activities. To date, there are limited studies that comprehensively address the intricate factors associated with the LHDFJSP, including workshop heterogeneity, job insertions and transfers, and considerations of low-carbon objectives. This paper establishes a multi-objective mathematical model with the goal of minimizing the total weighted tardiness and total energy consumption. To effectively solve this problem, diverse composite scheduling rules are formulated, alongside the application of a deep reinforcement learning (DRL) framework, i.e., Rainbow deep-Q network (Rainbow DQN), to learn the optimal scheduling strategy at each decision point in a dynamic environment. To verify the effectiveness of the proposed method, this paper extends the standard dataset to adapt to the LHDFJSP. Evaluation results confirm the generalization and robustness of the presented Rainbow DQN-based method.

An operator-inspired framework for metaheuristics and its applications on job-shop scheduling problems

The job-shop scheduling problem (JSP) is a well-known combinatorial optimization problem in manufacturing systems. For the past two decades, real-number metaheuristics have been widely used to solve the JSP using the real-number transform methods. A limitation of the real-number metaheuristics is the premature convergence due to the stochasticity of the transform methods. To eliminate this limitation, a novel operator framework has been designed, building the bridge between the discrete optimization problem (JSP) and real-number metaheuristics (also called continuous metaheuristics). Specifically, this paper captures the core operators of the real-number metaheuristics, namely, addition, subtraction, and multiplication. Firstly, three new operators are reconstructed according to several simple neighborhood structures to solve the JSP robustly and effectively. The properties of the arithmetic (symmetry) are taken into account in the reconstruction of the proposed operators to avoid excessive redundant searches. Secondly, a positional similarity-based population diversity is presented for the JSP to demonstrate the intrinsic distinctions between the proposed operators and the transform methods during the evolutionary process. Finally, the results of five widely used benchmark test suites (185 instances) show that the proposed operators can achieve a better balance between exploration and exploitation than the current real-number transform methods.

An efficient and adaptive design of reinforcement learning environment to solve job shop scheduling problem with soft actor-critic algorithm

Shop scheduling is deeply involved in manufacturing. In order to improve the efficiency of scheduling and fit dynamic scenarios, many Deep Reinforcement Learning (DRL) methods are studied to solve scheduling problems like job shop and flow shop. But most studies focus on using the latest algorithms while ignoring that the environment plays an important role in agent learning. In this paper, we design an effective, robust and size-agnostic environment for job shop scheduling. The proposed design of environment uses centralised training and decentralised execution (CTDE) to implement a multi-agent architecture. Together with the observation space we design, environmental information that is irrelevant to the current decision is eliminated as much as possible. The proposed action space enlarges the decision space of agents, which performs better than the traditional way. Finally, Soft Actor-Critic (SAC) algorithm is adapted to learning within this environment. By comparing with traditional scheduling rules, other reinforcement learning algorithms, and relevant literature, the superiority of the results obtained in this study is demonstrated.

A DQN-based memetic algorithm for energy-efficient job shop scheduling problem with integrated limited AGVs

AGVs have gained significant popularity in various industries. However, the existing literature rarely considers the integrated scheduling of production and logistics on the workshop due to the NP-hard property of both machine scheduling and AGV scheduling. The energy-efficient job shop scheduling problem with limited AGVs is investigated in this paper. A multi-objective memetic algorithm with deep Q-network (DQNMMA) is proposed to minimize the makespan and total energy consumption. In DQNMMA, ten features are selected to describe the current state of the population. This enables the DQN to dynamically adjust the crossover probability according to the population evolution. Formulas for calculating the head length and tail length of each node in the disjunctive graph model are presented for the first time to enable fast and accurate access to the critical paths. Building upon the analysis of critical paths, four problem properties are developed as the foundation for designing six neighborhood operators. Then, a property-based variable neighborhood search strategy is proposed to enhance the exploration capability of the algorithm. Numerous experimental results demonstrate that the proposed approaches can effectively enhance the performance of the algorithm, especially in solving large-scale problems. The comparative analysis with three other state-of-the-art multi-objective algorithms confirms the superiority and effectiveness of the proposed DQNMMA.

Dynamic Events in the Flexible Job-Shop Scheduling Problem: Rescheduling with a Hybrid Metaheuristic Algorithm

In the real world of manufacturing systems, production planning is crucial for organizing and optimizing various manufacturing process components. The objective of this paper is to present a methodology for both static scheduling and dynamic scheduling. In the proposed method, a hybrid algorithm is utilized to optimize the static flexible job-shop scheduling problem (FJSP) and dynamic flexible job-shop scheduling problem (DFJSP). This algorithm integrates the genetic algorithm (GA) as a global optimization technique with a simulated annealing (SA) algorithm serving as a local search optimization approach to accelerate convergence and prevent getting stuck in local minima. Additionally, variable neighborhood search (VNS) is utilized for efficient neighborhood search within this hybrid algorithm framework. For the FJSP, the proposed hybrid algorithm is simulated on a 40-benchmark dataset to evaluate its performance. Comparisons among the proposed hybrid algorithm and other algorithms are provided to show the effectiveness of the proposed algorithm, ensuring that the proposed hybrid algorithm can efficiently solve the FJSP, with 38 out of 40 instances demonstrating better results. The primary objective of this study is to perform dynamic scheduling on two datasets, including both single-purpose machine and multi-purpose machine datasets, using the proposed hybrid algorithm with a rescheduling strategy. By observing the results of the DFJSP, dynamic events such as a single machine breakdown, a single job arrival, multiple machine breakdowns, and multiple job arrivals demonstrate that the proposed hybrid algorithm with the rescheduling strategy achieves significant improvement and the proposed method obtains the best new solution, resulting in a significant decrease in makespan.

Grid Operation and Inspection Resource Scheduling Based on an Adaptive Genetic Algorithm

Grid operation and inspection a key links to ensure the safe operation of the power system, which requires efficient task allocation and resource scheduling. To address this problem, this paper proposes a resource scheduling model for grid operation and inspection based on bi-level programming. Firstly, the O&I process is analyzed and defined as a combined optimization problem of the multiple traveling salesman problem (MTSP) and the job-shop scheduling problem (JSP). Secondly, a bi-level programming model of MTSP and JSP is established according to the characteristics of the problem. Finally, an adaptive genetic algorithm is used to solve the problem. The feasibility of the model and the advancement of the algorithm are verified through the simulation of real scenarios and a large number of tests, which provide strong support for the sustainable development of the power system.

An improved memetic algorithm for multi-objective resource-constrained flexible job shop inverse scheduling problem: An application for machining workshop

Resource-constrained flexible job shop scheduling problems are commonly encountered in some manufacturing industries, and have been widely studied in recent years. However, traditional resource constrained flexible job shop scheduling problem rarely consider the uncertainties in actual manufacturing systems, which may make the original schedule become suboptimal or even unfeasible. Therefore, a resource constrained flexible job shop inverse scheduling problem (RCFJISP) is proposed in this paper, which aims to cope with uncertain events by simultaneously adjusting the machine, worker and process parameters of the original schedule. A multi-objective optimization model is constructed to minimize the makespan, worker cost, machine energy consumption and deviation index. Furthermore, an improved memetic algorithm (IMA) is developed for solving the proposed problem. In IMA, a novel double-layer encoding mechanism is designed to enhance the capacity in exploring new solution’s domains. Three initialization strategies utilizing original scheduling information are designed to improve the quality of initial solutions. An adaptive mutation strategy and a local search mechanism are designed to enhance exploration and exploitation ability of the algorithm. And a crowding operator is proposed to reflect the diversity of the population effectively. In computational experiments, 28 extended benchmarks are constructed, and the effectiveness of the proposed strategy and algorithm is verified by comparing IMA with its 4 variants and other 4 widely used algorithms. Finally, two inverse scheduling problems of a real-world hydraulic cylinder machining workshop under two uncertain situations are studied. The results demonstrate that IMA can effectively solve the actual inverse scheduling problem. With a slight adjustment to the original scheduling, it can reduce the makespan by 11.5%, the worker cost by 8.1% and the machine energy consumption by 27.9% on average.

Research on the performance of harmony search with local search algorithms for solving flexible job-shop scheduling problem

Combining the harmony search algorithm (HS) with the local search algorithm (LS) can prevent the HS from falling into a local optimum. However, how LS affects the performance of HS has not yet been studied systematically. Therefore, in this paper, it is first proposed to combine four frequently used LS with HS to obtain several search algorithms (HSLSs). Then, by taking the flexible job-shop scheduling problem (FJSP) as an example and considering decoding times, study how the parameters of HSLSs affect their performance, where the performance is evaluated by the difference rate based on the decoding times. The simulation results mainly show that (I) as the harmony memory size (HMS) gradually increases, the performance of HSLSs first increases rapidly and then tends to remain unchanged, and HMS is not the larger the better; (II) as harmony memory considering rate increases, the performance continues to improve, while the performance of pitch adjusting rate on HSLSs goes to the opposite; Finally, more benchmark instances are also used to verify the effectiveness of the proposed algorithms. The results of this paper have a certain guiding significance on how to choose LS and other parameters to improve HS for solving FJSP.

An improved genetic programming hyper-heuristic for the dynamic flexible job shop scheduling problem with reconfigurable manufacturing cells

The Dynamic Flexible Job Shop Scheduling Problem (DFJSP) is a classical and important research direction. However, current research usually considers the case where each manufacturing cell has a fixed and constant process capability. There are often situations in non-machining shops where each manufacturing cell is capable of capability reconfiguration and performs many different types of process operations, such as assembly shops and test shops. The variability of the manufacturing cell's capabilities increases the complexity of the problem compared to traditional FJSP. In this paper, we study the dynamic flexible job shop scheduling problem considering reconfigurable manufacturing cells (DFJSP-RMCs) with completion time, delay time and reconfiguration time as optimization objectives, and propose an improved Genetic Programming Hyper-Heuristic (GPHH) method to solve it. The method weighs the solution efficiency and the effectiveness of the results. In addition, an individual simplification policy (ISP) is proposed to reduce the evaluation time of the heuristic. Finally, random instances were generated under three production conditions and 10 independent runs were performed for each. Experiments show that the proposed method significantly reduces the time consumption while ensuring the quality of the results.

Enhancing constraint programming via supervised learning for job shop scheduling

Constraint programming (CP) is a powerful technique for solving constraint satisfaction and optimization problems. In CP solvers, the variable ordering strategy used to select which variable to explore first in the solving process has a significant impact on solver effectiveness. To address this issue, we propose a novel variable ordering strategy based on supervised learning, which we evaluate in the context of job shop scheduling problems. Our learning-based methods predict the optimal solution of a problem instance and use the predicted solution to order variables for CP solvers. Unlike traditional variable ordering methods, our methods can learn from the characteristics of each problem instance and customize the variable ordering strategy accordingly, leading to improved solver performance. Our experiments demonstrate that training machine learning models is highly efficient and can achieve high accuracy. Furthermore, our learned variable ordering methods perform competitively compared to four existing methods. Finally, we showcase the benefits of integrating machine learning-based variable ordering methods with conventional domain-based approaches through tie-breaking.

A Decentralized Optimization Algorithm for Multi-Agent Job Shop Scheduling with Private Information

The optimization of job shop scheduling is pivotal for improving overall production efficiency within a workshop. In demand-driven personalized production modes, achieving a balance between workshop resources and the diverse demands of customers presents a challenge in scheduling. Additionally, considering the self-interested behaviors of agents, this study focuses on tackling the problem of multi-agent job shop scheduling with private information. Multiple consumer agents and one job shop agent are considered, all of which are self-interested and have private information. To address this problem, a two-stage decentralized algorithm rooted in the genetic algorithm is developed to achieve a consensus schedule. The algorithm allows agents to evolve independently and concurrently, aiming to satisfy individual requirements. To prevent becoming trapped in a local optimum, the search space is broadened through crossover between agents and agent-based block insertion. Non-dominated sorting and grey relational analysis are applied to generate the final solution with high social welfare. The proposed algorithm is compared using a centralized approach and two state-of-the-art decentralized approaches in computational experiments involving 734 problem instances. The results validate that the proposed algorithm generates non-dominated solutions with strong convergence and uniformity. Moreover, the final solution produced by the developed algorithm outperforms those of the decentralized approaches. These advantages are more pronounced in larger-scale problem instances with more agents.

Research on Flexible Job Shop Scheduling Problem with Handling and Setup Time Based on Improved Discrete Particle Swarm Algorithm

With the gradual emergence of customized manufacturing, intelligent manufacturing systems have experienced widespread adoption, leading to a surge in research interests in the associated problem of intelligent scheduling. In this paper, we study the flexible job shop scheduling problem (FJSP) with setup time, handling time, and processing time in a multi-equipment work center production environment oriented toward smart manufacturing and make-to-order requirements. A mathematical model with the optimization objectives of minimizing the maximum completion time, the total number of machine adjustments, the total number of workpieces handled and the total load of the machine is constructed, and an improved discrete particle swarm algorithm based on Pareto optimization and a nonlinear adaptive inertia weighting strategy is proposed to solve the model. By integrating the model characteristics and algorithm features, a hybrid initialization method is designed to generate a higher-quality initialized population. Next, three cross-variance operators are used to implement particle position updates to maintain information sharing among particles. Then, the performance effectiveness of this algorithm is verified by testing and analyzing 15 FJSP test instances. Finally, the feasibility and effectiveness of the designed algorithm for solving multi-objective FJSPs are verified by designing an FJSP test example that includes processing time, setup time and handling time.

Research on flexible job-shop scheduling problem based on variation-reinforcement learning

The main focus of this paper is to solve the optimization problem of minimizing the maximum completion time in the flexible job-shop scheduling problem. In order to optimize this objective, random sampling is employed to extract a subset of states, and the mutation operator of the genetic algorithm is used to increase the diversity of sample chromosomes. Additionally, 5-tuple are defined as the state space, and a 4-tuple is designed as the action space. A suitable reward function is also developed. To solve the problem, four reinforcement learning algorithms (Double-Q-learning algorithm, Q-learning algorithm, SARS algorithm, and SARSA(λ) algorithm) are utilized. This approach effectively extracts states and avoids the curse of dimensionality problem that occurs when using reinforcement learning algorithms. Finally, experimental results using an international benchmark demonstrate the effectiveness of the proposed solution model.

Flexible job shop scheduling with stochastic machine breakdowns by an improved tuna swarm optimization algorithm

In job-shop production environments, machine breakdowns are a significant factor in reducing productivity. Existing approaches seldom consider algorithm improvement and rescheduling scheme design in an integrated manner, and lack stability considerations. This paper addresses the flexible job shop scheduling problem with random machine breakdowns, aiming to produce a stable rescheduling scheme that minimizes a combined index of maximum completion time and stability. The paper innovatively applies the tuna swarm optimization algorithm to the flexible job shop scheduling problem, proposing an efficient and superior improved version called the genetic chaos levy nonlinear tuna swarm optimization (GCLNTSO) algorithm. Three stability metrics are designed to guide the generation of efficient and stable rescheduling schemes. A rescheduling scheme is proposed that combines right-shift rescheduling with complete rescheduling. The proposed scheme is benchmarked against Brandymalter and Kacem’s problems, and compared with other algorithms from the literature. The results demonstrate that the GCLNTSO algorithm outperforms other algorithms in terms of both performance and stability.

Improved genetic algorithm for solving the total weight tardiness job shop scheduling problem

An improved genetic algorithm is proposed for the Job Shop Scheduling Problem with Minimum Total Weight Tardiness (JSSP/TWT). In the proposed improved genetic algorithm, a decoding method based on the Minimum Local Tardiness (MLT) rule of the job is proposed by using the commonly used chromosome coding method of job numbering, and a chromosome recombination operator based on the decoding of the MLT rule is added to the basic genetic algorithm flow. As a way to enhance the quality of the initialized population, a non-delay scheduling combined with heuristic rules for population initialization. and a PiMX (Precedence in Machine crossover) crossover operator based on the priority of processing on the machine is designed. Comparison experiments of simulation scheduling under different algorithm configurations are conducted for randomly generated larger scale JSSP/TWT. Statistical analysis of the experimental evidence indicates that the genetic algorithm based on the above three improvements exhibits significantly superior performance for JSSP/TWT solving: faster convergence and better scheduling solutions can be obtained.