Disjunctive Graph Research Articles

Evolution strategies-based optimized graph reinforcement learning for solving dynamic job shop scheduling problem

The job shop scheduling problem (JSSP) with dynamic events and uncertainty is a strongly NP-hard combinatorial optimization problem (COP) with extensive applications in the manufacturing system. Recently, growing interest has been aroused in utilizing machine learning techniques to solve the JSSP. However, most prior arts cannot handle dynamic events and barely consider uncertainties. To close this gap, this paper proposes a framework to solve a dynamic JSSP (DJSP) with machine breakdown and stochastic processing time based on Graph Neural Network (GNN) and deep reinforcement learning (DRL). To this end, we first formulate the DJSP as a Markov Decision Process (MDP), where disjunctive graph represent the states. Secondly, we propose a GNN-based model to effectively extract the embeddings of the state by considering the features of the dynamic events and the stochasticity of the problem, e.g., the machine breakdown and stochastic processing time. Then, the model constructs solutions by dispatching optimal operations to machines based on the learned embeddings. Notably, we propose to use the evolution strategies (ES) to find optimal policies that are more stable and robust than conventional DRL algorithms. The extensive experiments show that our method substantially outperforms existing reinforcement learning-based and traditional methods on multiple classic benchmarks.

A novel priority dispatch rule generation method based on graph neural network and reinforcement learning for distributed job-shop scheduling

With the development of a global economy, distributed manufacturing becomes common in the industrial field. The Distributed Job-shop Scheduling Problem (DJSP), which is widespread in real-life production, is a hotspot in the academic field. The existing Priority Dispatch Rules (PDRs), which are used to assign a value to each waiting job according to some method and select the job with minimum or maximum “value” for next processing, are all relatively simple but lack self-learning ability, while the metaheuristics are all complex and with fixed evolutionary trajectory and cannot change with the manufacturing environment. This paper proposes a novel PDR generation method based on Graph Neural Network (GNN) and Reinforcement Learning (RL), which can self-learn and self-evolute by interacting with the scheduling environment. To combine DJSP with GNN closely, a new solution representation based on disjunctive graph is designed. DJSP is formulated as a Markov decision process, and the problem features and inner connections among the vertices of the disjunctive graph are fully explored by the GNN. An Actor-Critic RL method is applied to automatically train the network parameters to optimize the policy, so that it can be used to schedule the best action at each step. Comprehensive experiments on 240 test instances are conducted to evaluate the performance of the proposed method, and the results indicate that the proposed method shows greater effectiveness, generalizability and stability than other 8 classical PDRs, 5 metaheuristics and 3 RL-based methods.

A novel MILP model for job shop scheduling problem with mobile robots

The mobile robot is the essential equipment for automated logistics in the intelligent workshop, but the literature on shop scheduling rarely considers transport resources. This paper studies the integrated scheduling of machines and mobile robots, which can facilitate the efficiency of production systems. For the job shop scheduling problem with mobile robots (JSPMR), the existing mathematical models are too complex to obtain the optimal solution in an efficient time. Therefore, a novel mixed integer linear programming (MILP) model is proposed to minimize the makespan. Firstly, in view of the property of the problem, a disjunctive graph model is modified to describe the relationship between transport and processing tasks. Secondly, a more accurate and simplified MILP is proposed based on the modified disjunctive graph model. Two related proofs are given to prove the proposed model satisfies all special situations. Thirdly, the proposed MILP is tested on the well-known benchmark, including 82 instances. The proposed model is the first MILP model to obtain optimal solutions for all instances. Finally, 40 larger-scale instances are presented based on a real-world engineering case and used to validate the performance of models further. The comparison results verify the effectiveness and superior computational performance of the proposed model.

Dynamic scheduling for dual-objective job shop with machine breakdown by reinforcement learning

In modern complicated and changing manufacturing environments, unforeseen dynamic events such as machine breakdown or unexpected job arrival make required production resources unpredictable. The scheduling scheme is desired to maintain high stability in dynamic manufacturing environments. To cope with the classic disturbance of machine breakdown, a robust pro-active scheduling scheme is proposed by inserting the repair time into a disjunctive graph for reinforcement learning (IRDRL) in this paper. Firstly, a new mathematical model is developed to predict the machine fault which is assumed to be determined by service time and bearing load. Secondly, a disjunctive graph with breakdown information is designed to express the dynamic scheduling status. Then, an online scheduling framework is built based on the well-trained model through the proximal policy optimization (PPO) algorithm. Finally, compared with the classical methods such as the right-shift strategy and static model of reinforcement learning (RL), the proposed robust pro-active scheduling scheme is verified with high robustness, stability, and short running time.

Gated‐Attention Model with Reinforcement Learning for Solving Dynamic Job Shop Scheduling Problem

Job shop scheduling problem (JSSP) is one of the well‐known NP‐hard combinatorial optimization problems (COPs) that aims to optimize the sequential assignment of finite machines to a set of jobs while adhering to specified problem constraints. Conventional solution approaches which include heuristic dispatching rules and evolutionary algorithms has been largely in use to solve JSSPs. Recently, the use of reinforcement learning (RL) has gained popularity for delivering better solution quality for JSSPs. In this research, we propose an end‐to‐end deep reinforcement learning (DRL) based scheduling model for solving the standard JSSP. Our DRL model uses attention‐based encoder of Transformer network to embed the JSSP environment represented as a disjunctive graph. We introduced Gate mechanism to modulate the flow of learnt features by preventing noise features from propagating across the network to enrich the representations of nodes of the disjunctive graph. In addition, we designed a novel Gate‐based graph pooling mechanism that preferentially constructs the graph embedding. A simple multi‐layer perceptron (MLP) based action selection network is used for sequentially generating optimal schedules. The model is trained using proximal policy optimization (PPO) algorithm which is built on actor critic (AC) framework. Experimental results show that our model outperforms existing heuristics and state of the art DRL based baselines on generated instances and well‐known public test benchmarks. © 2023 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

A Hybrid Evolutionary Algorithm Using Two Solution Representations for Hybrid Flow-Shop Scheduling Problem.

As an extension of the classical flow-shop scheduling problem, the hybrid flow-shop scheduling problem (HFSP) widely exists in large-scale industrial production systems and has been considered to be challenging for its complexity and flexibility. Evolutionary algorithms based on encoding and heuristic decoding approaches are shown effective in solving the HFSP. However, frequently used encoding and decoding strategies can only search a limited area of the solution space, thus leading to unsatisfactory performance during the later period. In this article, a hybrid evolutionary algorithm (HEA) using two solution representations is proposed to solve the HFSP for makespan minimization. First, the proposed HEA searches the solution space by a permutation-based encoding representation and two heuristic decoding methods to find some promising areas. Afterward, a Tabu search (TS) procedure based on a disjunctive graph representation is introduced to expand the searching space for further optimization. Two classical neighborhood structures focusing on critical paths are extended to the problem-specific backward schedules to generate candidate solutions for the TS. The proposed HEA is tested on three public HFSP benchmark sets from the existing literature, including 567 instances in total, and is compared with some state-of-the-art algorithms. Extensive experimental results indicate that the proposed HEA performs much better than the other algorithms. Moreover, the proposed method finds new best solutions for 285 hard instances.

Exact and metaheuristic approaches to solve the integrated production scheduling, berth allocation and storage yard allocation problem

This work studies an industrial complex specialized in producing and exporting fertilizers. The integration of the production scheduling, berth allocation, and storage yard allocation decisions is necessary to improve the overall performance of the complex as these problems are interrelated. Exact and metaheuristic approaches are proposed to solve the problem. First, a mixed-integer linear formulation along with a branch-and-cut procedure are proposed. The numerical experiments show that these exact methods solve small-sized instances optimally but fail to provide good quality solutions for industrial-sized instances. Therefore, a multi-start GRASP-ILS is proposed where initial solutions are constructed using a greedy randomized adaptive search procedure (GRASP) and improved using an iterated local search algorithm (ILS). The metaheuristic approach is based on a disjunctive graph representation and additional conditions to respect the storage yard constraints and other operational constraints. The results show that the multi-start GRASP-ILS can be effectively used to solve larger instances of the problem.

A Deep Reinforcement Learning Framework Based on an Attention Mechanism and Disjunctive Graph Embedding for the Job-Shop Scheduling Problem

The job-shop scheduling problem (JSSP) is a classical NP-hard combinatorial optimization problem, and the operating efficiency of manufacturing system is affected directly by the quality of its scheduling scheme. In this article, a novel deep reinforcement learning framework is proposed for solving the classical JSSP, where each machine has to process each job exactly once. This method based on an attention mechanism and disjunctive graph embedding, and a sequence-to-sequence pattern is used to model the JSSP in the framework. A disjunctive graph embedding process based on node2vec is used to learn the disjunctive graph representations containing JSSP characteristics, thereby generalizing the model considerably. An improved transformer architecture based on a multihead attention mechanism is used to generate solutions. Containing a parallel-computing encoder and a recurrent-computing decoder, it is adept at learning long-range dependencies and effective at solving large-scale scheduling problems. Experimental results verified the effectiveness of the proposed method.

Optimisation framework and method for solving the serial dual-shop collaborative scheduling problem

Due to the production process or environment, most products need to go through multiple production workshops, from raw materials to finished products. For example, a large structural part must go through a blanking workshop and a machining workshop. However, the research on the serial multi-shop cooperative scheduling problem is insufficient. Different from the single shop scheduling problem, the multi-shop scheduling problem can be regarded as a scheduling problem composed of multiple sub-problems. Therefore, optimisation processes between subproblems may be the key to solving this kind of problem. This paper focuses on the optimisation framework for the serial multi-shop cooperative scheduling problem. A specific serial dual-shop collaborative scheduling problem is abstracted from the production, whose first and second shop is the hybrid flow shop and the job shop, respectively. A disjunctive graph model of this specific problem is constructed. After that, four different optimisation frameworks are proposed, and testing instances are designed to compare these frameworks. Under different optimisation frameworks, the same tabu search algorithm with an efficient neighbourhood structure is used to optimize this problem. Finally, the optimisation results of different frameworks are compared and the results of each optimisation framework are analyzed.

A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem

This paper presents an end-to-end deep reinforcement framework to automatically learn a policy for solving a flexible Job-shop scheduling problem (FJSP) using a graph neural network. In the FJSP environment, the reinforcement agent needs to schedule an operation belonging to a job on an eligible machine among a set of compatible machines at each timestep. This means that an agent needs to control multiple actions simultaneously. Such a problem with multi-actions is formulated as a multiple Markov decision process (MMDP). For solving the MMDPs, we propose a multi-pointer graph networks (MPGN) architecture and a training algorithm called multi-Proximal Policy Optimization (multi-PPO) to learn two sub-policies, including a job operation action policy and a machine action policy to assign a job operation to a machine. The MPGN architecture consists of two encoder-decoder components, which define the job operation action policy and the machine action policy for predicting probability distributions over different operations and machines, respectively. We introduce a disjunctive graph representation of FJSP and use a graph neural network to embed the local state encountered during scheduling. The computational experiment results show that the agent can learn a high-quality dispatching policy and outperforms handcrafted heuristic dispatching rules in solution quality and meta-heuristic algorithm in running time. Moreover, the results achieved on random and benchmark instances demonstrate that the learned policies have a good generalization performance on real-world instances and significantly larger scale instances with up to 2000 operations.

Auction-based approach with improved disjunctive graph model for job shop scheduling problem with parallel batch processing

The job-shop scheduling problem (JSSP) is encountered in several industries, including the military where heat treatment is applied prior to the machining process in production. This study aims to minimize the overall make-span of a JSSP with parallel batch processing. The problem is formulated as a mixed-integer linear programming model. Feasible solutions are derived from an auction-based approach for forming batches, allocating operation machines, and scheduling. An improved disjunctive graph model is further developed to search for better solutions. We conduct numerical experiments to test a set of benchmark instances. A comparison of the results with those obtained applying other existing algorithms and CPLEX demonstrates the effectiveness and stability of the proposed auction-based approach and improved graph model. Furthermore, a statistical analysis using IBM SPSS shows that the proposed auction-based approach has an absolute advantage in solving medium-scale and large-scale instances of JSSP with batch processing.

A Disjunctive Graph Solution Representation for the Continuous and Dynamic Berth Allocation Problem

A Disjunctive Graph Solution Representation for the Continuous and Dynamic Berth Allocation Problem

Job Shop Scheduling: A Novel DRL approach for continuous schedule-generation facing real-time job arrivals

We present a DRL-based novel approach to solve the Job Shop Scheduling Problem (JSSP) in real-time while facing unpredictable job arrival disruptions. Our proposed approach consists of continuously generating improved schedules based on a rescheduling technique: It leads to continuous generation of schedules in triggered rescheduling points, and thus gives immediate response to random job arrivals. To implement the proposed technique, we use Proximal Policy Optimization Actor and Critic (PPO-AC), a combination of two RL algorithms. PPO-AC is used to assign job operations to available machines based on the job shop state that is represented by dynamic disjunctive graphs, modeling precedence constraints between job operations, and resource sharing constraints. Graph Embedding modeling is also applied for dynamic graph representation in PPO-AC algorithm. Preliminary numerical experiments of our innovative solution are discussed in this paper.

A multi-population, multi-objective memetic algorithm for energy-efficient job-shop scheduling with deteriorating machines

This paper focuses on an energy-efficient job-shop scheduling problem within a machine speed scaling framework, where productivity is affected by deterioration. To alleviate the deterioration effect, necessary maintenance activities must be put in place during the scheduling process. In addition to sequencing operations on machines, the problem at hand aims to determine the appropriate speeds of machines and positions of maintenance activities for the schedule, in order to minimise the total weighted tardiness and total energy consumption simultaneously. To deal with this problem, a multi-population, multi-objective memetic algorithm is proposed, in which the solutions are distributed into sub-populations. Besides a general local search, an advanced objective-oriented local search is also executed periodically on a portion of the population. These local search methods are designed based on a new disjunctive graph introduced to cover the solution space. Furthermore, an efficient non-dominated sorting method for bi-objective optimisation is developed. The performance of the memetic algorithm is evaluated via a series of comprehensive computational experiments, comparing it with state-of-the-art algorithms presented for job-shop scheduling problems with/without considering energy efficiency. Experimental results confirm that the proposed algorithm can outperform other algorithms being compared across a range of performance metrics.

Cooperative-Evolution-Based WPT Resource Allocation for Large-Scale Cognitive Industrial IoT

The recently developed technique of wireless power transfer (WPT) provides a promising way to charge the wireless sensor networks (WSNs) of cognitive industrial Internet of Things (IoT) deployed in areas that are difficult for humans to access. Previous work has focused on the power allocation strategy at the wireless node level. However, the priority among different modes in an identical wireless node has not been taken into consideration, and different modes equipped with different types of batteries accomplish different tasks in an identical wireless node. One challenging scenario is rechargeable WSNs with a large number of wireless nodes. In this article, we aim to optimize the power allocation strategy in priority constraint WPT systems with a large number of wireless nodes. Traditional WPT systems consist of a rechargeable WSN and a mobile charger, which are deployed for charging wireless nodes in a wireless manner. However, the constructed WPT system consists of a rechargeable WSN and multiple mobile chargers with adequate power, which can charge wireless nodes simultaneously. Each solution of the power allocation strategy can be represented as one disjunctive graph, and the critical path (CP) in the disjunctive graph is the core factor in determining the final maximum cost. Thus, we propose a decomposition strategy that can identify the interacting variables based on the CP by exploiting the perturbation technique. Then, the decomposed subcomponents are cooperatively evolved by adopting a cooperative evolutionary algorithm (CEA). The proposed CP-based grouping strategy combined with CEA is named CPCEA. Three state-of-the-art methods are tested and compared with CPCEA, and three scales of datasets are considered. The experimental results demonstrate the validity of CPCEA.

Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN

Traditional approaches for job shop scheduling problems are ill-suited to deal with complex and changeable production environments due to their limited real-time responsiveness. Based on disjunctive graph dispatching, this work proposes a deep reinforcement learning (DRL) framework, that combines the advantages of real-time response and flexibility of a deep convolutional neural network (CNN) and reinforcement learning (RL), and learns behavior strategies directly according to the input manufacturing states, thus is more appropriate for practical order-oriented manufacturing problems. In this framework, a scheduling process using a disjunction graph is viewed as a multi-stage sequential decision-making problem and a deep CNN is used to approximate the state-action value. The manufacturing states are expressed as multi-channel images and input into the network. Various heuristic rules are used as available actions. By adopting the dueling double Deep $Q$ -network with prioritized replay (DDDQNPR), the RL agent continually interacts with the scheduling environment through trial and error to obtain the best policy of combined actions for each decision step. Static computational experiments are performed on 85 JSSP instances from the well-known OR-Library. The results indicate that the proposed algorithm can obtain optimal solutions for small scale problems, and performs better than any single heuristic rule for large scale problems, with performances comparable to genetic algorithms. To prove the generalization and robustness of our algorithm, the instances with random initial states are used as validation sets during training to select the model with the best generalization ability, and then the performance of the trained policy on scheduling instances with different initial states is tested. The results show that the agent is able to get better solutions adaptively. Meanwhile, some studies on dynamic instances with random processing time are performed and experiment results indicate that out method can achieve comparable performances in dynamic environment in the short run.

Simulation Optimization on Complex Job Shop Scheduling with Non-Identical Job Sizes

This paper addresses the complex job shop scheduling problem with the consideration of non-identical job sizes. By simultaneously considering practical constraints of sequence dependent setup times, incompatible job families and job dependent batch processing time, we formulate this problem into a simulation optimization problem based on the disjunctive graph representation. In order to find scheduling policies that minimise the expectation of mean weighted tardiness, we propose a genetic programming based hyper heuristic to generate efficient dispatching rules. And then, based on the nested partition framework together with the optimal computing budget allocation technique, a hybrid rule selection algorithm is proposed for searching machine group specified rule combinations. Numerical results show that the proposed algorithms outperform benchmark algorithms in both solution quality and robustness.

An RFID Data Matrices-Based Evaluation Method for Process Logistics State

At present, radio frequency identification (RFID) technology has been widely applied in manufacturing industry. How to use collected RFID data to effectively evaluate process logistics state is an urgent problem. Firstly, process logistics state model based on extended disjunctive graph was established. Secondly, configuration scheme of RFID readers/tags and production elements was proposed according to the time and space characteristics of process logistics. Then, process logistics state matrices including jobs, warehouses, buffers, machine tools, and vehicles were constructed. The real-time process logistics states can be deduced by operation of above data matrices. Finally, a case study was proposed to verify the feasibility of the proposed methods.

Hybrid algorithm based on improved extended shifting bottleneck procedure and GA for assembly job shop scheduling problem

With the makespan as the optimisation goal, we propose a hybrid solving method that combines improved extended shifting bottleneck procedure (i-ESB) and genetic algorithm (GA) for the assembly job shop scheduling problem (AJSSP). Hybrid genetic algorithm (HGA) uses a GA based on operation constraint chain coding to achieve global search and a local search based on an i-ESB. In the design of i-ESB, an extended disjunctive graph model (EDG) corresponding to AJSSP is presented. The calculation method of the operation head and tail length based on EDG is studied, as well as the searching method of key operations. The Schrage algorithm with disturbance is used to solve the single-machine scheduling subproblem. The selection criterion for bottleneck machines is increased. A greedy bottleneck machine re-optimisation process is designed. The effectiveness and superiority of the proposed algorithm are verified by testing and analysing the relevant examples in the literature.

An efficient Pareto approach for solving the multi-objective flexible job-shop scheduling problem with regular criteria

In this paper, a general local search approach for the Multi-Objective Flexible Job-shop Scheduling Problem (MOFJSP) is proposed to determine a Pareto front for any combination of regular criteria. The approach is based on a disjunctive graph, a fast estimation function to evaluate moves and a hierarchical test to efficiently control the set of non-dominated solutions. Four search strategies using two neighborhood structures are developed. Numerical experiments are conducted on test instances of the literature with three sets of criteria to minimize and using metrics to evaluate and compare Pareto fronts. The results show that our approach provides sets of non-dominated solutions of good quality.