Dynamic Job Shop Research Articles

Gated‐Attention Model with Reinforcement Learning for Solving Dynamic Job Shop Scheduling Problem

Job shop scheduling problem (JSSP) is one of the well‐known NP‐hard combinatorial optimization problems (COPs) that aims to optimize the sequential assignment of finite machines to a set of jobs while adhering to specified problem constraints. Conventional solution approaches which include heuristic dispatching rules and evolutionary algorithms has been largely in use to solve JSSPs. Recently, the use of reinforcement learning (RL) has gained popularity for delivering better solution quality for JSSPs. In this research, we propose an end‐to‐end deep reinforcement learning (DRL) based scheduling model for solving the standard JSSP. Our DRL model uses attention‐based encoder of Transformer network to embed the JSSP environment represented as a disjunctive graph. We introduced Gate mechanism to modulate the flow of learnt features by preventing noise features from propagating across the network to enrich the representations of nodes of the disjunctive graph. In addition, we designed a novel Gate‐based graph pooling mechanism that preferentially constructs the graph embedding. A simple multi‐layer perceptron (MLP) based action selection network is used for sequentially generating optimal schedules. The model is trained using proximal policy optimization (PPO) algorithm which is built on actor critic (AC) framework. Experimental results show that our model outperforms existing heuristics and state of the art DRL based baselines on generated instances and well‐known public test benchmarks. © 2023 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

A self-learning discrete salp swarm algorithm based on deep reinforcement learning for dynamic job shop scheduling problem

A self-learning discrete salp swarm algorithm based on deep reinforcement learning for dynamic job shop scheduling problem

Scheduling AIV transporter using simulation-based supervised learning: A case study on a dynamic job-shop with three workstations

Dynamic job shop scheduling consists of scheduling jobs dynamically with different routing on a set of machines. A feasible and quick solution can be computed using heuristics. One well-known heuristic in job shop problems is selecting the priority dispatching rule (such as giving priority to the job with the Shortest Processing Time called SPT). Nowadays, with the application of industry 4.0 technologies such as sensors, Intelligent robots, etc., workshop data are more accessible and can be exploited to find the appropriate dispatching rule depending on the state of the shop. This work proposes a data-driven methodology for scheduling an AIV (Autonomous Intelligent Vehicle) that supplies three workstations. Our approach is based on data collected from an Arena simulation model fed to a supervised learning algorithm. This one helps identify one among five dispatching rules for each scheduling decision.

Responsiveness to sudden disturbances in manufacturing through dynamic job shop scheduling using Quantum Annealing

An efficient production planning and control is inevitable for the economic operation of a manufacturing system. As an essential part of production planning and control, process scheduling aims to assign processes to the available resources of a manufacturing system under consideration of its objectives. This forms an optimization problem also known as job shop scheduling which can be solved with computer aided techniques. Exact solution methods are only practical up to a certain number of functional units and processes, therefore, approximation methods are used in industry. However, as the problem size increases, the computational solving time increases significantly and the solution quality decreases in equal measure. In order to react to the current effects of global crises, such as disruptions in supply chains, approaches for fast and efficient rescheduling are needed. A dynamic shop job scheduling approach using Quantum Annealing bears the potential to close this research gap. Previous work has shown that Quantum Annealing is able to solve static job shop scheduling problems within seconds while finding good solutions. However, in a flexible environment such as a manufacturing system, the static approach is not suitable for process scheduling. Therefore, a dynamic Quantum Annealing based approach for job shop scheduling with consideration of machine breakdowns and new job arrivals is proposed. The approach monitors a manufacturing system and reacts to changes in the job pool or availabilities in functional units with rescheduling. The method is tested with several use cases involving small and large-scale problems and is compared with a simulated annealing approach. Thereby, the Quantum Annealing based computations show better results regarding solution quality and computing time. Moreover, the dynamic approach bears the potential for industrial application, especially as a supplement to a conventional advanced planning system.

Designing Dispatching Rules via Novel Genetic Programming with Feature Selection in Dynamic Job-Shop Scheduling

Genetic Programming (GP) has been widely employed to create dispatching rules intelligently for production scheduling. The success of GP depends on a suitable terminal set of selected features. Specifically, techniques that consider feature selection in GP to enhance rule understandability for dynamic job shop scheduling (DJSS) have been successful. However, existing feature selection algorithms in GP focus more emphasis on obtaining more compact rules with fewer features than on improving effectiveness. This paper is an attempt at combining a novel GP method, GP via dynamic diversity management, with feature selection to design effective and interpretable dispatching rules for DJSS. The idea of the novel GP method is to achieve a progressive transition from exploration to exploitation by relating the level of population diversity to the stopping criteria and elapsed duration. We hypothesize that diverse and promising individuals obtained from the novel GP method can guide the feature selection to design competitive rules. The proposed approach is compared with three GP-based algorithms and 20 benchmark rules in the different job shop conditions and scheduling objectives. Experiments show that the proposed approach greatly outperforms the compared methods in generating more interpretable and effective rules for the three objective functions. Overall, the average improvement over the best-evolved rules by the other three GP-based algorithms is 13.28%, 12.57%, and 15.62% in the mean tardiness (MT), mean flow time (MFT), and mean weighted tardiness (MWT) objective, respectively.

Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems

• The manufacturing equipment in the workshop is constructed as an equipment agent with the support of edge computing node, and a multi-agent manufacturing system with the ability of online scheduling and scheduling strategy optimization is presented. • A self-organizing negotiation mechanism based on contract network protocol (CNP) is proposed to guide the cooperation and competition among multiple agents , and to provide support for intelligent decision-making. • The decision-making module of equipment agent called artificial intelligence (AI) scheduler is established based on the multi-layer perceptron, and enables the manufacturing equipment to intelligently generate an optimal production strategy according to the perceived workshop state. • An adaptive learning strategy of AI scheduler based on proximal policy optimization (PPO) algorithm is proposed to improve its decision-making performance under the disturbance of orders and resources. • Experimental results show that the proposed methodology achieves better performance in terms of production strategy learning and disturbance handling compared with other optimization-based approaches and dispatching rules . Personalized orders bring challenges to the production paradigm, and there is an urgent need for the dynamic responsiveness and self-adjustment ability of the workshop. Traditional dispatching rules and heuristic algorithms solve the production planning and control problems by making schedules. However, the previous methods cannot work well in a changeable workshop environment when encountering a large number of stochastic disturbances of orders and resources. Recently, the potential of artificial intelligence (AI) algorithms in solving the dynamic scheduling problem has attracted researchers' attention. Therefore, this paper presents a multi-agent manufacturing system based on deep reinforcement learning (DRL), which integrates the self-organization mechanism and self-learning strategy. Firstly, the manufacturing equipment in the workshop is constructed as an equipment agent with the support of edge computing node, and an improved contract network protocol (CNP) is applied to guide the cooperation and competition among multiple agents, so as to complete personalized orders efficiently. Secondly, a multi-layer perceptron is employed to establish the decision-making module called AI scheduler inside the equipment agent. According to the perceived workshop state information, AI scheduler intelligently generates an optimal production strategy to perform task allocation. Then, based on the collected sample trajectories of scheduling process, AI scheduler is periodically trained and updated through the proximal policy optimization (PPO) algorithm to improve its decision-making performance. Finally, in the multi-agent manufacturing system testbed, dynamic events such as stochastic job insertions and unpredictable machine failures are considered in the verification experiments. The experimental results show that the proposed method is capable of obtaining the scheduling solutions that meet various performance metrics, as well as dealing with resource or task disturbances efficiently and autonomously.

Job Shop 제조라인의 설비 가공지연 최소화를 위한 시계열 예측 기반의 자율주행 로봇 할당 제어

제조 운영 환경의 빠른 변화에 유연하게 대응하기 위해 기존의 rail 기반의 AGV, OHT 제조물류에서 벗어나 rail이 필요 없는 AMR 기반의 물류로 전환하는 새로운 시도들이 나타나고 있다. AMR은 주행용 rail을 설치할 필요가 없으므로 설비 증설이나 이동과 같은 레이아웃 변화에 즉시 대응 가능한 장점이 있다. 그러나 AMR은 이동 경로가 다양하며 상대적으로 속도가 느려서 기존 rail 기반 물류 시스템 대비 운반 요청 대응력이 떨어진다는 단점이 있다. 따라서 발생하는 작업요청을 적절한 AMR에 배정하는 작업 할당이 전체 제조물류 운반 성능에 미치는 영향도가 커졌다. 본 연구에서는 동적인 job shop 제조 환경에서 예측기반의 AMR 작업할당 방법을 제안하고 성능을 평가한다. 구체적으로, 운반을 요청할 설비를 예측하고 해당 설비까지 이동거리를 최소화하는 위치로 AMR을 미리 이동시켜서 실제 운반 요청 발생 시 재빠르게 대응하려는 것이다. 실험 결과 기존의 방법에 비해 AMR 대기시간이 감소하는 성능을 보였다.

Research on an Adaptive Real-Time Scheduling Method of Dynamic Job-Shop Based on Reinforcement Learning

With the rapid development of modern industrialization in our country and the continuous improvement of people’s living standards, the changing market has put forward new requirements for traditional manufacturing enterprises. The order product demand of many manufacturing enterprises is changing from a single variety, large batch to multiple varieties, small batch. In view of this change, the traditional job-shop scheduling method is far from enough, which greatly affects the efficiency of the production job-shop. In order to solve the above problems, this paper proposes a real-time scheduling method based on reinforcement learning applied in the dynamic job-shop and a new type of neural network is designed at the same time. The neural network is designed with the high-dimensional data in the above problem as input, and a policy-based reinforcement learning algorithm is proposed based on this network. In the process of research, it was found that the reinforcement learning method not only enables the agent to use historical data for learning, but also enables it to explore and learn other possible high-reward actions within a certain range, so as to realize the optimization of production goals under real-time scheduling. The effectiveness of the proposed real-time scheduling method is verified by comparing with other common rule-based scheduling methods in the manufacturing environment.

Numerical Simulation and Assessment of Meta Heuristic Optimization Based Multi Objective Dynamic Job Shop Scheduling System

In today's world of manufacturing, cost reduction becomes one of the most important issues. A successful business needs to reduce its cost to be competitive. The programming of the machine is playing an important role in production planning and control as a tool to help manufacturers reduce their costs maximizing the use of their resources. The programming problem is not only limited to the programming of the machine, but also covers many other areas such such as computer and information technology and communication. From the definition, programming is an art that involves allocating, planning the allocation and utilization of resources to achieve a goal. The aim of the program is complete tasks in a reasonable amount of time. This reasonableness is a performance measure of how well the resources are allocated to tasks. Time or time-dependent functions are always it used as performance measures. The objectives of this research are to develop Intelligent Search Heuristic algorithms (ISHA) for equal and variable size sub lot for m machine flow shop problems, to Implement Particle Swarm Optimization algorithm (PSO) in matlab, to develop PSO based Optimization program for efficient job shop scheduling problem. The work also address solution to observe and verify results of PSO based Job Shop Scheduling with help of graft chart.

Assessing the Effect of Setup Times and Shop Utilization Levels on Performance of ORR Policies in a Stochastic Dynamic Job Shop with Sequence-Dependent Setup Times: A Simulation Approach

Assessing the Effect of Setup Times and Shop Utilization Levels on Performance of ORR Policies in a Stochastic Dynamic Job Shop with Sequence-Dependent Setup Times: A Simulation Approach

Surrogate-assisted automatic evolving of dispatching rules for multi-objective dynamic job shop scheduling using genetic programming

Dispatching rules are simple but efficient heuristics to solve multi-objective job shop scheduling problems, particularly useful to face the challenges of dynamic shop environments. A promising method to automatically evolve non-dominated rules represents multi-objective genetic programming based hyper-heuristic (MO-GP-HH). The aim of such methods is to approximate the Pareto front of non-dominated dispatching rules as good as possible in order to provide a sufficient set of efficient solutions from which the decision maker can select the most preferred one. However, one of the main drawbacks of existing approaches is the computational demanding simulation-based fitness evaluation of the evolving rules. To efficiently allocate the computational budget, surrogate models can be employed to approximate the fitness. Two possible ways, that estimate the fitness either based on a simplified problem or based on samples of fully evaluated individuals making use of machine learning techniques are investigated in this paper. Several representatives of both categories are first examined with regard to their selection accuracy and execution time. Furthermore, we developed a surrogate-assisted MO-GP-HH framework, incorporating a pre-selection task in the NSGA-II algorithm. The most promising candidates are consequently implemented in the framework. Using a dynamic job shop scenario, the two proposed algorithms are compared to the original one without using surrogates. With the aim to minimize the mean flowtime and maximum tardiness, experimental results demonstrate that the proposed algorithms outperform the former. Making use of surrogates leads to a reduction in computational costs of up to 70%. Another interesting finding shows that the enhanced ability to identify duplicates based on the phenotypic characterization of individuals is particularly helpful in increasing diversity within a population. This study illustrates the positive effect of this mechanism on the exploration of the entire Pareto front.

An End-to-End Deep Learning Method for Dynamic Job Shop Scheduling Problem

Job shop scheduling problem (JSSP) is essential in the production, which can significantly improve production efficiency. Dynamic events such as machine breakdown and job rework frequently occur in smart manufacturing, making the dynamic job shop scheduling problem (DJSSP) methods urgently needed. Existing rule-based and meta-heuristic methods cannot cope with dynamic events in DJSSPs of different sizes in real time. This paper proposes an end-to-end transformer-based deep learning method named spatial pyramid pooling-based transformer (SPP-Transformer), which shows strong generalizability and can be applied to different-sized DJSSPs. The feature extraction module extracts the production environment features that are further compressed into fixed-length vectors by the feature compression module. Then, the action selection module selects the simple priority rule in real time. The experimental results show that the makespan of SPP-Transformer is 11.67% smaller than the average makespan of dispatching rules, meta-heuristic methods, and RL methods, proving that SPP-Transformer realizes effective dynamic scheduling without training different models for different DJSSPs. To the best of our knowledge, SPP-Transformer is the first application of an end-to-end transformer in DJSSP, which not only improves the productivity of industrial scheduling but also provides a paradigm for future research on deep learning in DJSSP.

Feature selection approach for evolving reactive scheduling policies for dynamic job shop scheduling problem using gene expression programming

Dispatching rules are one of the most widely applied methods for solving Dynamic Job Shop Scheduling problems (DJSSP) in real-world manufacturing systems. Hence, the automated design of effective rules has been an important subject in the scheduling literature for the past several years. High computational requirements and difficulty in interpreting generated rules are limitations of literature methods. Also, feature selection approaches in the field of automated design of scheduling policies have been developed for the tree-based GP approach only. Therefore, the aim of this study is to propose a feature selection approach for the Gene Expression Programming (GEP) algorithm to evolve high-quality rules in simple structures with an affordable computational budget. This integration speeds up the search process by restricting the GP search space using the linear representation of the GEP algorithm and creates concise rules with only meaningful features using the feature selection approach. The proposed algorithm is compared with five algorithms and 30 rules from the literature under different processing conditions. Three performance measures are considered including total weighted tardiness, mean tardiness, and mean flow time. The results show that the proposed algorithm can generate smaller rules with high interpretability in a much shorter training time.

Effective and interpretable dispatching rules for dynamic job shops via guided empirical learning

The emergence of Industry 4.0 is making production systems more flexible and also more dynamic. In these settings, schedules often need to be adapted in real-time by dispatching rules. Although substantial progress was made until the ’90s, the performance of these rules is still rather limited. The machine learning literature is developing a variety of methods to improve them. However, the resulting rules are difficult to interpret and do not generalise well for a wide range of settings. This paper is the first major attempt at combining machine learning with domain problem reasoning for scheduling. The idea consists of using the insights obtained with the latter to guide the empirical search of the former. We hypothesise that this guided empirical learning process should result in effective and interpretable dispatching rules that generalise well to different scenarios. We test our approach in the classical dynamic job shop scheduling problem minimising tardiness, one of the most well-studied scheduling problems. The simulation experiments include a wide spectrum of scenarios for the first time, from highly loose to tight due dates and from low utilisation conditions to severely congested shops. Nonetheless, results show that our approach can find new state-of-the-art rules, which significantly outperform the existing literature in the vast majority of settings. Overall, the average improvement over the best combination of benchmark rules is 19%. Moreover, the rules are compact, interpretable, and generalise well to extreme, unseen scenarios. Therefore, we believe that this methodology could be a new paradigm for applying machine learning to dynamic optimisation problems.

A novel feature selection for evolving compact dispatching rules using genetic programming for dynamic job shop scheduling

Because of advances in computational power and machine learning algorithms, the automated design of scheduling rules using Genetic Programming (GP) is successfully applied to solve dynamic job shop scheduling problems. Although GP-evolved rules usually outperform dispatching rules reported in the literature, intensive computational costs and rule interpretability persist as important limitations. Furthermore, the importance of features in the terminal set varies greatly among scenarios. The inclusion of irrelevant features broadens the search space. Therefore, proper selection of features is necessary to increase the convergence speed and to improve rule understandability using fewer features. In this paper, we propose a new representation of the GP rules that abstracts the importance of each terminal. Moreover, an adaptive feature selection mechanism is developed to estimate terminals’ weights from earlier generations in restricting the search space of the current generation. The proposed approach is compared with three GP algorithms from the literature and 30 human-made rules from the literature under different job shop configurations and scheduling objectives, including total weighted tardiness, mean tardiness, and mean flow time. Experimentally obtained results demonstrate that the proposed approach outperforms methods from the literature in generating more interpretable rules in a shorter computational time without sacrificing solution quality.

Scheduling with Maintenance Technique in Dynamic Job Shop System including Machine Breakdowns

Scheduling with Maintenance Technique in Dynamic Job Shop System including Machine Breakdowns

Real-time scheduling simulation optimisation of job shop in a production-logistics collaborative environment

ABSTRACT In a complex and dynamic job shop containing logistics factor, schedule needs to be generated rapidly, so the real-time scheduling method is more suitable for such scenario. Such method takes advantage of local information within a short time due to the rapid changes of information under uncertain environment. Therefore, how to make use of the future information by prediction while ensuring the robustness of schedule is a valuable problem. To solve it, firstly, a new real-time scheduling model and algorithm is proposed. There is a new kind of release moment of task information which can give AGVs the longest time to prepare for the task than existing research. Secondly, a real-time information update mechanism is designed to increase schedule’s robustness. Finally, a large-scale and dynamic job shop simulation experimental platform is developed. Dynamic factors include the random insertion of orders and failures of equipment. Results show that the method proposed outperforms existing research in terms of customer satisfaction, equipment utilisation and energy consumption. The robustness of schedule can also be acceptable. This paper also finds a rule that in job shop with the large proportion of logistics transportation time, the above method can achieve more competitive results.

Towards a priority rule to integrate maintenance operations into production schedules

Industry 4.0 techniques provide additional data on the current state of production systems which can be used to adapt and improve the scheduling. In previous work, a data-driven approach for production control of dynamic job shops has been developed, which reacted in real-time on occurring events but handled maintenance only reactive. This approach will be extended towards preventive maintenance by developing an integrated method for scheduling production and maintenance tasks simultaneously. This paper presents an approach of a priority rule which will be used to ensure the execution of maintenance operations but still offers the flexibility of the concrete time of execution in a small maintenance window. The proposed rule is tested in a simulation and shows the benefit against benchmarks.

Surrogate-Assisted Automatic Evolving of Dispatching Rules for Multi-Objective Dynamic Job Shop Scheduling Using Genetic Programming

Surrogate-Assisted Automatic Evolving of Dispatching Rules for Multi-Objective Dynamic Job Shop Scheduling Using Genetic Programming

Genetic programming-based hyper-heuristic approach for solving dynamic job shop scheduling problem with extended technical precedence constraints

Extended technical precedence constraints (ETPC) in dynamic job shop scheduling problem (DJSP) are the precedence constraints existing between different jobs instead of the conventional technical precedence constraints existing in the operations of the same job. This paper presents the mathematical programming model of the DJSP with ETPC to minimize the mean weighted tardiness of the jobs. The mathematical model contributes to the solution and modelling of the DJSP with ETPC and it is used to solve small-sized problems to optimality. To solve industry-sized problems, a constructive heuristic called the dispatching rule (DR) is employed. This paper investigates the use of genetic programming (GP) as a hyper-heuristic in the automated generation of the problem-specific DRs for solving the problem under consideration. The genetic programming-based hyper heuristic (GPHH) approach constructs the DRs which are learned from the training instances and then verified on the test instances by the simulation experiments. To enhance the efficiency of the approach when evolving effective DRs to solve the problem, the approach is improved with strategies which consist of a problem-specific attribute selection for GP and a threshold condition mechanism for fitness evaluation. The simulation results verify the effectiveness and efficiency of the evolved DRs to the problem under consideration by comparing against the existing classical DRs. The statistical analysis of the simulation results shows that the evolved DRs outperform the selected benchmark DRs on the problem under study. The sensitivity analysis also shows that the DRs generated by the GPHH approach are robust under different scheduling performance measures. Moreover, the effects of the model parameters, including the percentage of jobs with ETPC and the machine utilization, on the performance of the DRs are investigated.