Job Shop Environment Research Articles

Dynamic Integrated Scheduling of Production Equipment and Automated Guided Vehicles in a Flexible Job Shop Based on Deep Reinforcement Learning

The high-quality development of the manufacturing industry necessitates accelerating its transformation towards high-end, intelligent, and green development. Considering logistics resource constraints, the impact of dynamic disturbance events on production, and the need for energy-efficient production, the integrated scheduling of production equipment and automated guided vehicles (AGVs) in a flexible job shop environment is investigated in this study. Firstly, a static model for the integrated scheduling of production equipment and AGVs (ISPEA) is developed based on mixed-integer programming, which aims to optimize the maximum completion time and total production energy consumption (EC). In recent years, reinforcement learning, including deep reinforcement learning (DRL), has demonstrated significant advantages in handling workshop scheduling issues with sequential decision-making characteristics, which can fully utilize the vast quantity of historical data accumulated in the workshop and adjust production plans in a timely manner based on changes in production conditions and demand. Accordingly, a DRL-based approach is introduced to address the common production disturbances in emergency order insertions. Combined with the characteristics of the ISPEA problem and an event-driven strategy for handling dynamic events, four types of agents, namely workpiece selection, machine selection, AGV selection, and target selection agents, are set up, which refine workshop production status characteristics as observation inputs and generate rules for selecting workpieces, machines, AGVs, and targets. These agents are trained offline using the QMIX multi-agent reinforcement learning framework, and the trained agents are utilized to solve the dynamic ISPEA problem. Finally, the effectiveness of the proposed model and method is validated through a comparison of the solution performance with other typical optimization algorithms for various cases.

Probing an LSTM-PPO-Based reinforcement learning algorithm to solve dynamic job shop scheduling problem

With the growth of personalized demand and the continuous improvement in social productivity, the large-scale and few-variety centralized production model is gradually transitioning towards a personalized model of small batches and multiple varieties, which makes the manufacturing process of the job shop increasingly complex. Furthermore, disruptive events such as machinery failures and rush orders in the job shop increase the uncertainty and variability of the production environment. Traditional scheduling methods are usually based on fixed rules and heuristic algorithms, which are difficult to adapt to constantly changing production environments and demands. This may lead to inaccurate scheduling decisions and hinder the optimal allocation of job shop resources. To solve the dynamic job shop scheduling problem (JSP) more effectively, this paper proposes a Reinforcement Learning (RL) optimization algorithm integrating long short-term memory (LSTM) neural network and proximal policy optimization (PPO). It can dynamically adjust scheduling strategies according to the changing production environment, achieving comprehensive status awareness of the job shop environment to make optimal scheduling decisions. First, a state-aware network framework based on LSTM-PPO is proposed to achieve real-time perception of job shop state changes. Then, the state and action space of the job shop are described within the context of the state-aware network framework. Finally, an experimental environment is established to verify the algorithm’s effectiveness. Training the LSTM-PPO algorithm makes it feasible to achieve better performance than other scheduling methods. By comparing the initial planning time with the actual completion time of the rescheduling decision under different dynamic disturbances, the efficiency of the proposed algorithm is verified for the dynamic JSP

Dynamic flexible job shop scheduling based on deep reinforcement learning

The unpredictable dynamic events in smart factory seriously influence the scheduling schemes and production efficiency. To minimize the total tardiness of orders, this paper proposes a Deep Reinforcement Learning (DRL) method to solve the Dynamic Flexible Job Shop Scheduling Problem (DFJSP) with random job arrival. In the scheduling process, the intelligent agent can select the operations to be processed on the available machines according to the job shop state at each scheduling point by transforming DFJSP into a Markov Decision Process (MDP). For reflecting the job shop environment, eight state features are extracted, and six composite dispatching rules are designed as the action space. Moreover, a reward function is developed to enhance the learning efficiency and solution quality of the intelligent agent, the Soft-max selection strategy is applied to balance exploration and exploitation. The Dueling Double Deep Q-Network (Dueling DDQN) algorithm is introduced to train the intelligent agent, which effectively alleviates the overestimation problem. Numerical simulation results show that the proposed Dueling DDQN method effectively solves the DFJSP. Compared with existing scheduling strategies, the proposed approach not only exhibits superior performance but also maintains its advantages among different scale instances.

A two-level evolutionary algorithm for dynamic scheduling in flexible job shop environment

A two-level evolutionary algorithm for dynamic scheduling in flexible job shop environment

A machine learning-based simulation metamodeling method for dynamic scheduling in smart manufacturing systems

Conventional Digital Twins (DTs) in smart manufacturing rely on complex and time-intensive simulation models, hindering real-time DT-based decision-making. However, the availability of big data in Manufacturing Execution Systems (MES) enables training different Machine Learning (ML) models for fast and accurate predictions and decision assessments. Accordingly, this paper proposes an ML-Based Simulation Metamodeling Method (MLBSM) to facilitate DT-based decision-making for dynamic production scheduling in complex Stochastic Flexible Job Shop (SFJS) environments. The proposed MLBSM integrates three modules: a novel data vectorizing method (SPBM), multi-output Adaptive Boosting Regressor (ABR) models, and a new statistical risk evaluation method. SPBM converts unstructured production log data into numerical vectors for ABR training by calculating numeric penalty scores for each job based on the position of operations in the schedule queue. Each trained ABR predicts mean job completion times for various dynamic scenarios based on shift schedules. The risk evaluation method estimates the standard deviation of job completion times and calculates the delay probability scores for each job, aiding DT in promptly evaluating production schedules. Working seamlessly together, MLBSM modules present a novel way of using production log data for ML training and ultimately bypassing several computationally intensive simulation replications. In this research, a simulation model generates the synthetic MES data, focusing on the machining process at a photolithography workstation in the semiconductor manufacturing. Experiments demonstrate the MLBSM’s accuracy and efficiency, predicting high-risk jobs with over 80% recall and being at least 70 times faster than conventional simulation runs. Sensitivity analyses also confirm the MLBSM’s consistency under different workstation conditions.

Investigation of Flexible Job Shop with Machine Availability Constraints

In real-world scheduling applications, machines may be unavailable during certain time periods for deterministic and stochastic reasons. This situation does not pose any problems if the jobs always have more than one machine available for processing. However, it becomes an issue if the only available machine is the one which more than one job needs for processing. Thus, the investigation of limited machine availability, along with the practical requirement to handle this feature of scheduling problems, are of huge significance. This paper examined a flexible job shop environment of a manufacturing firm to optimize different performance criteria related to makespan, due dates, priorities and penalties by using a metaheuristic approach, taking into account the precedence constraints. The work investigated the case in which all machines are available for processing and another case in which some of the machines are known in advance to be unavailable. From the results, the best schedules are analysed and the perspectives of the findings to decision-makers are discussed with the purpose of achieving high machine utilization, cost reduction and customer satisfaction.

Combination of genetic algorithm and shifting bottleneck heuristics to solve a job shop scheduling problem in wood processing company

The scheduling problem has always been the concern of most companies in the flexible Job shop or Job shop environments because of its complexity. It is really a challenge for researchers to solve or find out the optimum solution. The lead time production at Teak Woods line in a wood processing company, a case study, faced with about 80% of orders which could not be done on time, although its capacity meet requirements. Therefore, it is very useful and necessary to set up a production scheduling model in order to reduce production time and synchronize the product’s parts through efficient modulation in an optimum sequence of jobs on the shopfloor. A combination of Genetic Algorithm and Shifting Bottleneck Heuristics was used to find out high quality solutions. It shows that the productivity increased from 2.32 m3/day to 4.3 m3/day (up 98.27%); Processing time was reduced from 3248.6 minutes to 1352.1 minutes (down 58.38%); The sync time decreased from 7554.8 minutes to 350.8 minutes (down 95.36%).

A cooperative coevolutionary hyper-heuristic approach to solve lot-sizing and job shop scheduling problems using genetic programming

Lot-sizing and scheduling in a job shop environment is a fundamental problem that appears in many industrial settings. The problem is very complex, and solutions are often needed fast. Although many solution methods have been proposed, with increasingly better results, their computational times are not suitable for decision-makers who want solutions instantly. Therefore, we propose a novel greedy heuristic to efficiently generate production plans and schedules of good quality. The main innovation of our approach represents the incorporation of a simulation-based technique, which directly generates schedules while simultaneously determining lot sizes. By utilising priority rules, this unique feature enables us to address the complexity of job shop scheduling environments and ensures the feasibility of the resulting schedules. Using a selection of well-known rules from the literature, experiments on a variety of shop configurations and complexities showed that the proposed heuristic is able to obtain solutions with an average gap to Cplex of 4.12%. To further improve the proposed heuristic, a cooperative coevolutionary genetic programming-based hyper-heuristic has been developed. The average gap to Cplex was reduced up to 1.92%. These solutions are generated in a small fraction of a second, regardless of the size of the instance.

Dynamic scheduling for flexible job shop with insufficient transportation resources via graph neural network and deep reinforcement learning

The smart workshop is a powerful tool for manufacturing companies to reduce waste and improve production efficiency through real-time data analysis for self-organized production. Automated Guided Vehicles (AGVs) have been widely used for material handling in smart workshop due to their high degree of autonomy, flexibility and powerful end-to-end capability to cope with logistics tasks in production modes such as multiple species and small batch, and mass customization. However, the highly dynamic, complex and uncertain nature of the smart job shop environment makes production scheduling with insufficient transportation resources in mind a challenge. To this end, this paper addresses the dynamic flexible job shop scheduling problem with insufficient transportation resources (DFJSP-ITR), and learn high-quality priority dispatching rule (PDR) end-to-end to minimize makespan by the proposed deep reinforcement learning (DRL) method. To achieve integrated decision making for operation, machine and AGV, an architecture based on heterogeneous graph neural network (GNN) and DRL is proposed. Considering the impact of different AGV distribution methods on the scheduling objective, this paper compares two different AGV distribution methods. Experiments show that the proposed method has superiority and good generalization ability compared with the current PDRs-based methods regardless of the AGV distribution strategy used.

Research on Terminal Distance Index-Based Multi-Step Ant Colony Optimization for Mobile Robot Path Planning

Aiming at the path planning problem of mobile robot, when the traditional ant colony optimization (ACO) is simulated in grid model, it is assumed that ants can only move to adjacent nodes, that is, the step size is 1 and there are 8 movable directions. However, in practice, the moving direction of the ant is completely free. Therefore, an improved terminal distance index-based multi-step ant colony optimization (TDI-MSACO) for mobile robot path planning is proposed in this paper. A multi-step ant colony optimization (MSACO) is firstly used to improve the flexibility of ant’s movement and the path obtained by using MSACO is shorter and more in line with the actual situation, through the simulation of a large number of cases, it is determined that the optimal step size is 2 or 3. In addition, aiming at the problems of pheromone updating mechanism in MSACO, a concept of terminal distance index (TDI) is proposed to replace pheromone concentration and accelerate the convergence speed of the MSACO. In order to verify the effectiveness of the improved TDI-MSACO, the simulations are tested on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$20\times 20$</tex-math> </inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$30\times 30$</tex-math> </inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$50\times 50$</tex-math> </inline-formula> grid models and the results show that the improved TDI-MSACO has faster convergence speed and shorter path. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —the motivation of this paper comes from the need to develop a fast and efficient path planning method for practical applications such as material transport in shop floor, cleaning, monitoring of dangerous radioactive sites and military applications, and so on. The ant colony optimization (ACO), inspired from the foraging behavior of ant species, is a swarm intelligence algorithm for solving hard combinatorial optimization problems. The ACO is widely used in robot path planning areas because of its characteristics of positive feedback, distributed computation. However, it has the weaknesses of premature convergence and low search speed, which greatly hinder its application. To improve the performance of the ACO, an improved terminal distance index-based multi-step ant colony optimization (TDI-MSACO) for mobile robot path planning is proposed in this paper. Multi-step and terminal distance index are used to improve the flexibility of ant movement and accelerate the convergence speed, respectively. The effectiveness of the TDI-MSACO method for solving mobile path planning problems is proved by comparing the simulation results with those previously presented in the literature. Furthermore, the proposed method can be expanded to the dynamic environment containing multiple static and dynamic obstacles of many sizes and forms, it also can be used to solve the energy-efficient path planning problems. Meanwhile, it can be further modified and applied to the simultaneous scheduling of machines and transport robots in an actual flexible job shop environment.

A data-driven simulation-optimization framework for generating priority dispatching rules in dynamic job shop scheduling with uncertainties

Modeling and optimizing dynamic job shop scheduling problems (DJSSP) without ample assumptions is inherently challenging due to the increasing complexity and uncertainty arising from various disruptive events. Moreover, establishing job due dates in dynamic job shop environments has become increasing difficult due to unforeseen events, while it remains a pivotal factor in determining tardiness-related objectives. A research gap exists in exploring the potential synergy between the two aforementioned aspects and its resulting implications. In this study, we propose a novel framework that combines data-driven simulation, machine learning, and evolutionary algorithm to generate new Priority Dispatching Rules (PDRs) for optimizing makespan and total tardiness. The framework incorporates a data-exchange mechanism for real-time updates of the simulation model based on the shop floor. Moreover, an online due date prediction model using XGBoost is developed, replacing the previous rough estimation of the Total Work Content (TWK) rule. Simulation experiments of various scales demonstrate that XGBoost-assisted due date prediction leads to an average reduction of 1.6% in makespan and 65.0% in total tardiness compared to the TWK rule. The proposed method outperforms existing well-known dispatching rules, yielding considerable improvements with average reductions of 90.8%, 40.3%, and 44.6% in total tardiness for vary-sized experiments. Finally, the approach is validated through a real industrial case study.

An optimization algorithm for the multi-objective flexible fuzzy job shop environment with partial flexibility based on adaptive teaching–learning considering fuzzy processing times

An optimization algorithm for the multi-objective flexible fuzzy job shop environment with partial flexibility based on adaptive teaching–learning considering fuzzy processing times

Combining Reinforcement Learning Algorithms with Graph Neural Networks to Solve Dynamic Job Shop Scheduling Problems

Smart factories have attracted a lot of attention from scholars for intelligent scheduling problems due to the complexity and dynamics of their production processes. The dynamic job shop scheduling problem (DJSP), as one of the intelligent scheduling problems, aims to make an optimized scheduling decision sequence based on the real-time dynamic job shop environment. The traditional reinforcement learning (RL) method converts the scheduling problem with a Markov process and combines its own reward method to obtain scheduling sequences in different real-time shop states. However, the definition of shop states often relies on the scheduling experience of the model constructor, which undoubtedly affects the optimization capability of the reinforcement learning model. In this paper, we combine graph neural network (GNN) and deep reinforcement learning (DRL) algorithm to solve DJSP. An agent model from job shop state analysis graph to scheduling rules is constructed, thus avoiding the problem that traditional reinforcement learning methods rely on scheduling experience to artificially set the state feature vectors. In addition, a new reward function is defined, and the experimental results prove that our proposed reward method is more effective. The effectiveness and feasibility of our model is demonstrated by comparing with general deep reinforcement learning algorithms on minimizing the earlier and later completion time, which also lays the foundation for solving the DJSP later.

An Approach to Integrated Scheduling of Flexible Job-Shop Considering Conflict-Free Routing Problems.

This study proposes an approach to minimize the maximum makespan of the integrated scheduling problem in flexible job-shop environments, taking into account conflict-free routing problems. A hybrid genetic algorithm is developed for production scheduling, and the optimal ranges of crossover and mutation probabilities are also discussed. The study applies the proposed algorithm to 82 test problems and demonstrates its superior performance over the Sliding Time Window (STW) heuristic proposed by Bilge and the Genetic Algorithm proposed by Ulusoy (UGA). For conflict-free routing problems of Automated Guided Vehicles (AGVs), the genetic algorithm based on AGV coding is used to study the AGV scheduling problem, and specific solutions are proposed to solve different conflicts. In addition, sensors on the AGVs provide real-time data to ensure that the AGVs can navigate through the environment safely and efficiently without causing any conflicts or collisions with other AGVs or objects in the environment. The Dijkstra algorithm based on a time window is used to calculate the shortest paths for all AGVs. Empirical evidence on the feasibility of the proposed approach is presented in a study of a real flexible job-shop. This approach can provide a highly efficient and accurate scheduling method for manufacturing enterprises.

Automatically evolving preference-based dispatching rules for multi-objective job shop scheduling

Dispatching rules represent a simple heuristic for finding good solutions for job shop scheduling problems. Due to their fast applicability and easy handling, they are often used in manufacturing companies to create appropriate production schedules. It has been shown that dispatching rules that are specifically designed for the requirements of a particular environment improve the performance of schedules. Hyper-heuristics based on genetic programming can be used for the automated generation of such dispatching rules. Evolutionary algorithms search the space of dispatching rule components for the most effective priority function to optimize the performance of the resulting schedule. Various studies have highlighted the advantages in the single-objective case, which made it possible to derive a large number of new dispatching rules that exceeded previous benchmark rules. Because it is usually necessary to consider more than one objective simultaneously to ensure effective creation of schedules, the need for a multi-objective optimization method arises. In this paper, we propose an interactive multi-objective optimization method, namely the reference point method, implemented in a hyper-heuristic genetic programming framework. A decision support system has also been developed and implemented in a web-based application to facilitate interaction with the user. Incorporating preferences into the solution process aims to efficiently evolve a dispatching rule that meets the expectations of a decision-maker. A fictitious experiment was carried out in a benchmark job shop environment. The results show that the final solution selected by the decision-maker can produce schedules achieving a desired compromise between the makespan, total tardiness, and total waiting time. Testing the evolved dispatching rule on an independent set of instances and comparing its performance with other benchmark dispatching rules revealed that the proposed method successfully finds dispatching rules that meet the decision-maker’s expectations and are capable of reproducing similar compromise schedules for unseen problems in the same environment.

An Interactive Two-Stage Framework for Simultaneous Machine Selection and Buffer Allocation

In manufacturing, machine selection and buffer allocation are extremely important and widely studied problems, as both can significantly affect the system performance. Solving either problem alone in a deterministic setting is already challenging. In this study, however, we simultaneously solve a high dimensional machine selection and buffer allocation problem in a highly stochastic and complex production line environment. The objective is to simultaneously select the optimal combination of machine type and buffer size allocation to maximize production throughput under limited cost and total buffer size constraints. We develop an efficient, interactive two-stage optimization method, called Machine Selection and Buffer Allocation algorithm (MSBA), that consists of a rapid machine selection (RMS) procedure and an adaptive global particle and local hyperbox search (AGPLHS) algorithm to solve the problem via simulation optimization. MSBA achieves seamless integration of RMS and AGPLHS so as to enable the simultaneous machine selection and buffer allocation to be solved efficiently. Furthermore, we compare our overall framework with three common existing methods (Genetic Algorithm, Nelder-Mead Algorithm and Adaptive Tabu Search). It is shown that our interactive two-stage framework outperforms the competing algorithms both in terms of effectiveness and efficiency. Note to Practitioners—Two important problems to solve in many production processes such as semiconductor manufacturing are machine selection and buffer allocation. Various methodologies have been utilized to solve each of the two problems individually. This work proposes an interactive two-stage simulation optimization algorithm which simultaneously solves both problems in complex, high dimensional and profoundly stochastic environments. While the most basic problem setting is a serial multi-workstation environment, complexities such as re-work stations, merging, splitting, parallel machine, and multi-product processor can be incorporated into the simulation model and solved using our proposed methodology. The methodology is user-friendly and intuitive yet adaptable enough to handle a wide range of production line environments or solve other stochastic optimization problems which require the simultaneous optimization of a mixture of binary and integer-based solutions. The numerical study, consisting of two different job-shop environments and including three benchmark algorithms, provides evidence for the efficiency of the proposed algorithm. Sensitivity analysis on user-defined parameters is also provided for the benefit of practitioners.

Research on a Real-time Job-shop Scheduling Method Based on Reinforcement Learning

At present, manufacturing models are characterized by multi-variety, small batch, and diversification. It is insufficient to use traditional scheduling methods for production management with high performance. A real-time production scheduling system based on reinforcement learning (RL) is suggested in an effort to address the aforementioned issues. A brand-new manufacturing neural network is created to learn the state-action values for production scheduling in real time using high-dimensional data as the input. The detailed setup of network inputs, neural network, action, and reward are also designed. Then, a policy-based reinforcement learning algorithm is proposed to achieve the optimum objective. Finally, By contrasting the proposed scheduling strategy with rule-based approaches in a smart manufacturing environment, its efficacy is demonstrated. according to experimental data, the suggested algorithm can successfully improve performance in the dynamic job-shop environment.

Integrated Scheduling of the Production and Maintenance of Parallel Machine Job-shop Considering Stochastic Machine Breakdowns

The integrated scheduling of production and maintenance can make equipment maintenance in line with the production pace, so as to effectively prevent anormal interruptions of the production process due to equipment failure, and ensure the smooth implementation of the production scheduling plan. Aiming at the parallel machine job-shop environment, and considering stochastic machine failures and different degradation speeds of parallel machines, this paper introduced the minimal maintenance and preventive maintenance strategies to establish an integrated scheduling model for production and maintenance, designed a genetic algorithm based on process coding and binary hybrid coding to solve the model, and verified the correctness of the proposed model and the effectiveness of the algorithm through an instance. This study provided an effective decision-making method for parallel machine job-shop scheduling problems.

An Improved Elephant Herding Optimization for Energy-Saving Assembly Job Shop Scheduling Problem with Transportation Times

The energy-saving scheduling problem (ESSP) has gained increasing attention of researchers in the manufacturing field. However, there is a lack of studies on ESSPs in the assembly job shop environment. In contrast with traditional scheduling problems, the assembly job shop scheduling problem (AJSP) adds the additional consideration of hierarchical precedence constraints between different jobs of each final product. This paper focuses on developing a methodology for an energy-saving assembly job shop scheduling problem with job transportation times. Firstly, a mathematical model is constructed with the objective of minimizing total energy consumption. Secondly, an improved elephant herding optimization (IEHO) is proposed by considering the problem’s characteristics. Finally, thirty-two different instances are designed to verify the performance of the proposed algorithm. Computational results and statistical data demonstrate that the IEHO has advantages over other algorithms in terms of the solving accuracy for the considered problem.

Effective dispatching rules mining based on near-optimal schedules in intelligent job shop environment

In many intelligent job shop, a scheduling system has collected and stored huge amounts of production data. Meanwhile, managers still believe that more scheduling knowledge should be further acquired from historical production data. To this end, this paper focuses on exploring newly dispatching rules via near-optimal schedules generation, data transformation, and dispatching rules mining. Without loss of the optimality, we propose a swap operation-based method to improve the quality of feasible schedules, generated by existing dispatching rules or meta-heuristic algorithms. Then, data transformation converts them into training data set with four types of constructed attributes. Thirdly, we present a hybrid genetic algorithm and random forest to extract high-quality training data set and evolve dispatching rules. The experimental results indicate that the proposed approach is effective and robust, and the newly dispatching rules are superior to the existing dispatching rules and some learning algorithms. Finally, a case study with two scenarios in a real environment also shows that the proposed approach outperforms classical rules and meta-heuristic algorithms.