Mean Tardiness Research Articles

Collaborative dynamic scheduling in a self-organizing manufacturing system using multi-agent reinforcement learning

Personalized product demands have made the production mode of many varieties and small batches mainstream. Self-organizing manufacturing systems represented by multi-agent-based manufacturing systems (MAMS) are pursued by manufacturing enterprises to support collaborative dynamic scheduling among various pieces of manufacturing equipment. In contrast to traditional scheduling methods including meta-heuristic algorithms and dispatching rules, reinforcement learning (RL), whether in the form of single-agent RL (SARL) or multi-agent RL (MARL), combines high response and robust performance in dynamic environments. However, the SARL-based methods suffer from the curse of dimensionality in the action space, while current MARL-based methods fail to achieve the optimization of collaborative scheduling decisions among heterogeneous manufacturing equipment. These factors directly hinder the effective resolution of dynamic scheduling in the MAMS. Therefore, we present a novel MARL-based approach that solves the dynamic flexible job-shop scheduling problem in the MAMS (DFJSP-MAMS), with the objective of minimizing the mean tardiness. Here, various items of heterogeneous manufacturing equipment in the MAMS, including the warehouse, machines, buffers, and vehicles, are encapsulated as agents. Moreover, a partially observable Markov game is constructed to specify the collaborative scheduling process among these agents and transfer the DFJSP-MAMS into a MARL task. Furthermore, an action space consisting of several weight variables is designed for each agent to offer the weight selection of dispatching rules. Subsequently, a policy network is built for each agent to determine the weights and scheduling decisions. Finally, the multi-agent deep deterministic policy gradient algorithm is employed to train these agents for the optimization of collaborative scheduling decisions among various pieces of heterogeneous equipment. Experiment results demonstrate that the proposed approach outperforms single dispatching rules, the SARL-based method, and other MARL-based methods under 36 distinct manufacturing environment conditions.

Minimizing Conditional Mean Tardiness and Mean Earliness on a Single Machine with the due Windows Approach

A Multicriteria scheduling problem that involves four due date-based performance measures; the total tardiness, proportion of tardy jobs, total earliness as well as the proportion of early jobs was reduced to an equivalent bicriteria problem of minimizing conditional mean Tardiness and conditional mean earliness. A schedule that optimizes these measures tend towards an ideal Just-In-Time (JIT) schedule. Two solution methods named GOA 1 and GOA 2 were proposed to solve the equivalent reduced problem. A single machine and due window approach was explored. The ideal JIT schedule with zero tardiness and zero earliness was set as the optimal and used for benchmarking the proposed solution as well as other methods found in the literature. The results show that the proposed heuristics yielded results that are not significantly different from the optimal in some instances.

Corrected Aggregate Workload approach on order release by considering job’s routing position induced variable indirect load

Purpose Workload contribution calculation approaches in the existing literature overestimate or underestimate indirect workload, which increases both workload fluctuation and shop floor throughput performance. This study optimizes a Corrected Aggregate Workload (CAW) approach to control the workload contribution of workstations and Work In Progress (WIP) levels, thereby improving the shop floor throughput performance.Design/methodology/approach This study adopts simulation experiment by SimPy, and experimental factors are: (1) two workload contribution methods (CAW method and considering Position Corrected Aggregate Workload [PCAW] method); (2) two release methods (LUMS COR release and immediate release); (3) eleven workload norms for LUMS COR release (from 7- to 15-time units), and infinite workload norm for immediate release; and (4) two dispatching rules (First Come First Served, FCFS and Operation Due Date, ODD). Each scenario is replicated 100 times, and for each replication data are collected for 10,000 time units, being the warm-up period set to 3,000-time units.Findings The results of this study confirm that the PCAW calculation method outperforms the CAW method, especially during higher workload norm levels. The PCAW method is considered the better solution in practice due to its excellent performance in terms of percentage tardiness and mean tardiness time. The efficient workload contribution approach, as discussed in this study, has the potential to offset delivery performance loss that results from throughput performance loss.Originality/valueThis study proposes a novel approach that considers the workstations’ position in the routing of the job and the position of jobs CAW method. The results demonstrated that it allows shop floor throughput time to be short and feasible. It controls WIP by workload contribution of workstations, resulting in a lean shop floor. Therefore, workload contribution calculation is of particular significance for high-variety Make-To-Order (MTO) companies.

Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers

Dynamic events and transportation constraints would significantly affect the full utilization of resources and the reduction of production costs in distributed job shops. Therefore, in this paper, a deep reinforcement learning algorithm (DRL)-based real-time scheduling method is developed to minimize the mean tardiness of the dynamic distributed job shop scheduling problem with transfers (DDJSPT) considering random job arrivals. Firstly, the proposed DDJSPT is modeled as a Markov decision process (MDP). Then, ten problem-oriented state features covering four aspects of factories, machines, jobs, and operations are elaborately extracted from the dynamic distributed job shop. After that, eleven composite rules considering the uniqueness of DDJSPT are constructed as a pool of actions to intelligently prioritize unfinished jobs and allocate the selected job to an appropriate factory. Moreover, a justified reward function adapted from the objective is designed for better convergence of DRLs. Subsequently, five DRLs are employed to address the DDJSPT, encompassing deep Q-network (DQN), double DQN (DDQN), dueling DQN (DlDQN), trust region policy optimization (TRPO), and proximal policy optimization (PPO). Finally, grounded in numerical comparison experiments under 243 production configurations of the DDJSPT, the effectiveness and generalization of DRL-based scheduling methods are credibly verified and confirmed.

Managing premature idleness in high-variety manufacturing

AbstractThis paper shows the effectiveness of labour transfers in addressing premature idleness caused by controlled order release. Controlled order release restricts order entry to the shop floor and is commonly employed in high-variety manufacturing where it results in benefits such as stable work-in-progress. However, it can increase waiting times when orders are blocked from release, while capacities are idling. This issue, known as premature idleness, negatively impacts delivery performance. Previous studies have primarily focused on addressing premature idleness through input control by releasing new orders to idling workstations. This approach overlooks the potential of output control during premature idleness, transferring labour to assist at other workstations in a dual resource constrained setting. Using simulation, this study demonstrates that output control significantly improves delivery performance—in terms of mean tardiness and percentage tardy—and reduces total and shop floor throughput times. Importantly, this result proves robust, even when the efficiency of the assisting worker is severely limited. Shop-level performance improves despite the efficiency loss of the worker. The impact of the where-rule is minimal, while the efficacy of the priority dispatching rule depends on the joint efficiency of collaborating workers. Finally, we show that combining input control and output control enhances performance, providing opportunities for further research on the role of both control approaches in high-variety manufacturing.

A Multi-Agent Reinforcement Learning Approach to the Dynamic Job Shop Scheduling Problem

In a production environment, scheduling decides job and machine allocations and the operation sequence. In a job shop production system, the wide variety of jobs, complex routes, and real-life events becomes challenging for scheduling activities. New, unexpected events disrupt the production schedule and require dynamic scheduling updates to the production schedule on an event-based basis. To solve the dynamic scheduling problem, we propose a multi-agent system with reinforcement learning aimed at the minimization of tardiness and flow time to improve the dynamic scheduling techniques. The performance of the proposed multi-agent system is compared with the first-in–first-out, shortest processing time, and earliest due date dispatching rules in terms of the minimization of tardy jobs, mean tardiness, maximum tardiness, mean earliness, maximum earliness, mean flow time, maximum flow time, work in process, and makespan. Five scenarios are generated with different arrival intervals of the jobs to the job shop production system. The results of the experiments, performed for the 3 × 3, 5 × 5, and 10 × 10 problem sizes, show that our multi-agent system overperforms compared to the dispatching rules as the workload of the job shop increases. Under a heavy workload, the proposed multi-agent system gives the best results for five performance criteria, which are the proportion of tardy jobs, mean tardiness, maximum tardiness, mean flow time, and maximum flow time.

Dynamic scheduling for flexible job shop using a deep reinforcement learning approach

Due to the influence of dynamic changes in the manufacturing environment, a single dispatching rule (SDR) cannot consistently attain better results than other rules for dynamic scheduling problems. Dynamic selection of the most appropriate rule from several SDRs based on the Deep Q-Network (DQN) offers better scheduling performance than using an individual SDR. However, the discreteness of action space caused by the DQN and the simplicity of the action as an SDR limit the selection range and restrict performance improvement. Thus, in this paper, we propose a scheduling method based on deep reinforcement learning for the dynamic flexible job-shop scheduling problem (DFJSP), aiming to minimize the mean tardiness. Firstly, a Markov decision process with composite scheduling action is provided to elaborate the flexible job-shop dynamic scheduling process and transform the DFJSP into an RL task. Subsequently, a composite scheduling action aggregated by SDRs and continuous weight variables is designed to provide a continuous rule space and SDR weight selection. Moreover, a reward function related to mean tardiness performance criteria is designed such that maximizing the cumulative reward is equivalent to minimizing the mean tardiness. Finally, a policy network with states as inputs and weights as outputs is constructed to generate the scheduling decision at each decision point. Also, the deep deterministic policy gradient (DDPG) algorithm is used to train the policy network to select the most appropriate weights at each decision point, thereby aggregating the SDRs into a better rule. Results from numerical experiments reveal that the proposed scheduling method achieves significantly better scheduling results than an SDR and the DQN-based method in dynamically changeable manufacturing environments.

Modified multi-objective simulated annealing algorithm for scheduling a flow shop production system with setup times

The presence of setup times and multiple objectives makes the scheduling problem complex. This paper presents two algorithms namely: multi-objective simulated annealing (MOSA) and modified MOSA for the simultaneous minimisation of makespan and mean tardiness in scheduling a flow shop production system wherein setup times depend on processing sequence. The modified MOSA algorithm is developed by hybridising MOSA with variable neighbourhood search. Benchmark problems are used to conduct computational experiments. The analysis of the algorithms reveals that modified MOSA outperforms MOSA for the measures considered. Hybridisation improves the performance of MOSA. The proposed algorithms are applied on scheduling a real-world SDST flow shop and the results indicate superior performance of modified MOSA.

Designing Dispatching Rules via Novel Genetic Programming with Feature Selection in Dynamic Job-Shop Scheduling

Genetic Programming (GP) has been widely employed to create dispatching rules intelligently for production scheduling. The success of GP depends on a suitable terminal set of selected features. Specifically, techniques that consider feature selection in GP to enhance rule understandability for dynamic job shop scheduling (DJSS) have been successful. However, existing feature selection algorithms in GP focus more emphasis on obtaining more compact rules with fewer features than on improving effectiveness. This paper is an attempt at combining a novel GP method, GP via dynamic diversity management, with feature selection to design effective and interpretable dispatching rules for DJSS. The idea of the novel GP method is to achieve a progressive transition from exploration to exploitation by relating the level of population diversity to the stopping criteria and elapsed duration. We hypothesize that diverse and promising individuals obtained from the novel GP method can guide the feature selection to design competitive rules. The proposed approach is compared with three GP-based algorithms and 20 benchmark rules in the different job shop conditions and scheduling objectives. Experiments show that the proposed approach greatly outperforms the compared methods in generating more interpretable and effective rules for the three objective functions. Overall, the average improvement over the best-evolved rules by the other three GP-based algorithms is 13.28%, 12.57%, and 15.62% in the mean tardiness (MT), mean flow time (MFT), and mean weighted tardiness (MWT) objective, respectively.

A multi-objective discrete differential evolution algorithm for energy-efficient two-stage flow shop scheduling under time-of-use electricity tariffs

In response to the appeal for environmental protection and the efficient utilization of energy, local governments in China actively promote time-of-use (TOU) electricity tariffs for manufacturing enterprises. Motivated by a real-life industrial scenario of large special-purpose pressure vessel production, this paper addresses an energy-efficient two-stage flow shop scheduling problem under TOU tariffs to minimize the total electricity cost and the mean tardiness. Since the problem is computationally intractable, we focus on developing a multi-objective discrete differential evolution (MDDE) algorithm. Specifically, based on the optimal properties of the problem, we tailor an encoding scheme that consists of two job sequences and an idle time vector. The novel mutation and crossover operators are designed to generate the trail individuals, and the hypervolume contribution indicator is incorporated into the bi-criteria selection operator to measure the quality of the solution. Furthermore, two neighborhood structures are designed to iteratively improve the non-dominated solutions in the external archive set. We evaluate the performance of the MDDE algorithm via extensive computational experiments. The experimental results indicate that the MDDE algorithm can obtain the good approximate Pareto front, and it outperforms the well-known NSGA-II, SPEA2 and MOEA/D algorithms in solution quality.

Towards System State Dispatching in High‐Variety Manufacturing

This study proposes a shift towards system state dispatching in the production control literature on high-variety manufacturing. System state dispatching lets the decision on what order to produce next be driven by system-wide implications while trading of an array of control objectives. This contrasts the current literature that uses hierarchical order review and release methods that control the system at release, whilst myopic priority rules control order dispatching based on local queue information. We develop such a system state dispatching method, called FOCUS, and test it using simulation. The results show that FOCUS enables a big leap forward in production control performance. Specifically, FOCUS reduces the number of orders delivered late by a factor of one to eight and mean tardiness by a factor of two to ten compared to state-of-the-art production control methods. These results are consistent over a wide variety of conditions related to routing direction, routing length, process time variability and due date tightness.

Feature selection approach for evolving reactive scheduling policies for dynamic job shop scheduling problem using gene expression programming

Dispatching rules are one of the most widely applied methods for solving Dynamic Job Shop Scheduling problems (DJSSP) in real-world manufacturing systems. Hence, the automated design of effective rules has been an important subject in the scheduling literature for the past several years. High computational requirements and difficulty in interpreting generated rules are limitations of literature methods. Also, feature selection approaches in the field of automated design of scheduling policies have been developed for the tree-based GP approach only. Therefore, the aim of this study is to propose a feature selection approach for the Gene Expression Programming (GEP) algorithm to evolve high-quality rules in simple structures with an affordable computational budget. This integration speeds up the search process by restricting the GP search space using the linear representation of the GEP algorithm and creates concise rules with only meaningful features using the feature selection approach. The proposed algorithm is compared with five algorithms and 30 rules from the literature under different processing conditions. Three performance measures are considered including total weighted tardiness, mean tardiness, and mean flow time. The results show that the proposed algorithm can generate smaller rules with high interpretability in a much shorter training time.

Multi-Objective Approach with a Distance Metric in Genetic Programming for Job Shop Scheduling

The goal of the Fourth Industrial Revolution is to develop smart factories that ensure flexibility and adaptability in complex production environments, without human intervention. Smart factories are based on three main pillars: integration through digitalization, employment of flexible structures, and the use of artificial intelligence (AI) methods. Genetic programming (GP) is one of the most promising AI approaches used in the automated design of production-scheduling rules. However, promoting diversity and controlling the bloating effect are major challenges to the success of GP algorithms in developing production-scheduling rules that deliver high-quality solutions. Therefore, we introduced a multi-objective technique to increase the diversity among GP individuals while considering the program length as an objective to avoid the bloating effect. The proposed approach employs a new diversity metric to measure the distance between GP individuals and the best rule in the current generation. Subsequently, the non-dominated sorting genetic algorithm II (NSGA-II) was used to select individuals based on three objectives: solution quality, similarity value, and program length. To assess the effectiveness of the proposed approach, we compare the two versions with three GP methods in the literature in terms of automatically generating dispatching rules on 10 benchmark instances of the job-shop scheduling problem. The experimental results show that the proposed distance measure enhances the phenotypic diversity of individuals, resulting in improved fitness values without the need for additional fitness assessments. In addition, the integration of NSGA-II with the GP algorithm facilitates the evolution of superior job shop dispatching rules with high diversity and shorter lengths under the makespan and mean tardiness objectives.

A novel feature selection for evolving compact dispatching rules using genetic programming for dynamic job shop scheduling

Because of advances in computational power and machine learning algorithms, the automated design of scheduling rules using Genetic Programming (GP) is successfully applied to solve dynamic job shop scheduling problems. Although GP-evolved rules usually outperform dispatching rules reported in the literature, intensive computational costs and rule interpretability persist as important limitations. Furthermore, the importance of features in the terminal set varies greatly among scenarios. The inclusion of irrelevant features broadens the search space. Therefore, proper selection of features is necessary to increase the convergence speed and to improve rule understandability using fewer features. In this paper, we propose a new representation of the GP rules that abstracts the importance of each terminal. Moreover, an adaptive feature selection mechanism is developed to estimate terminals’ weights from earlier generations in restricting the search space of the current generation. The proposed approach is compared with three GP algorithms from the literature and 30 human-made rules from the literature under different job shop configurations and scheduling objectives, including total weighted tardiness, mean tardiness, and mean flow time. Experimentally obtained results demonstrate that the proposed approach outperforms methods from the literature in generating more interpretable rules in a shorter computational time without sacrificing solution quality.

Effect of Routing Flexibility on the Performance of Order Release Policies in a Flexible Job Shop with Sequence-Dependent Setup Time: A Simulation Study

ABSTRACT Order review and release (ORR) in a flexible job shop considering sequence-dependent setup time (SDST) is a complicated job shop problem. Routing flexibility helps in increasing the flexibility of the system by providing alternative routes to jobs. This research work assesses the effect of routing flexibility on order release policies in a flexible job shop considering SDST within a stochastic and dynamic (SCDM) manufacturing environment. In this study, five order release policies, viz., Corrected Aggregate Load Approach (CALA), Aggregate Workload Trigger (AGGWLT), Corrected Workload Trigger (CorrWLT), Work Center Workload Trigger (WCWLT), and Lancaster University Management School Corrected Order Release (LUMSCOR), are considered with planned release date sequencing rule. Four performance measures are considered in the present research work, i.e., mean throughput time, mean lead time, mean tardiness, and total setups. For experimental purposes, a simulation model is developed with six routing flexibility levels with the help of Promodel® software. Results indicate that the performance of all ORR policies can be improved by considering routing flexibility along with it. For a given ORR policy, as routing flexibility increases, there is a decrease in system performance measures up to a certain level, and after that, it starts increasing. Thus, routing flexibility has an optimum level. The least value of workload trigger level for all ORR policies provides the best results for mean throughput time performance measure. Further, as the workload trigger level increases, the best results are obtained for the other three performance measures except the mean throughput time.

Simulation Study of Scheduling Heuristics for Parallel Machines with Sequence-Independent Setups

In this paper we examine a dynamic job scheduling problem on identical, parallel machines. Jobs that belong to different families arrive to the system at random time intervals. Stochastic processing times of jobs are considered and each family has different due date allowances. The manufacturing system consists of a single input queue and several machines. When switching from one job type to another on some machine, a sequence-independent setup is incurred. The scheduling of jobs is performed by some priority rule and nine alternative rules are considered in this study. The discrete-event simulation model of the system is developed and the performance of the scheduling heuristics is assessed in terms of the metrics of mean cycle time, mean earliness and mean tardiness. A total of thirteen simulation cases is considered, where the differentiating factors of the cases are the average job interarrival times, the due date and the setup parameters of jobs.

On the integration of card-allocation and dispatching decisions in POLCA systems: an assessment by simulation

POLCA is an important card-based control system for low volume, high variety production contexts. A job can only be produced at an upstream station if it has acquired a POLCA card that has returned from its downstream station. A common assumption in the POLCA literature is that cards are allocated to jobs as soon as they return to the upstream station. This dissects the queue in front of a station into jobs that have a card (and can be produced) and those that do not have a card (and cannot be produced). This artificially and prematurely constrains the dispatching decision, i.e. the decision concerning which job to produce next at a station. In response, this paper proposes integrating the card-allocation and dispatching decisions such that the allocation of POLCA cards to jobs is postponed until the dispatching decision is made. Simulation results demonstrate that this integrated approach does not improve performance under simple ERD dispatching, as is commonly applied in the POLCA literature. But when a more powerful rule is applied, percentage tardy and mean tardiness performance improve by more than 75% and 50%, respectively, for an integrated decision. Most importantly, results suggest that in production environments like the one considered in this study, the integrated approach dispenses with the use of POLCA altogether if a suitable priority rule is used.

Analysis of Integrated Preventive Maintenance and Machine Failure in Stochastic Flexible Job Shop Scheduling with Sequence-dependent Setup Time

ABSTRACT When we consider the real-time situation in scheduling problems, it always helps to enhance the manufacturing system and increases the system performance. In this study, the effect of five input parameters, i.e., reliability-centered preventive maintenance, percentage of machine failure (PMF), mean time to repair for random machine breakdown, due date tightness factor, and routing flexibility (RF) on stochastic flexible job shop scheduling problem (SFJSSP) under simultaneously reliability-centered preventive maintenance and random machine breakdown environment with sequence-dependent setup time is evaluated. The effects of input parameters are measured using four different performance measures, i.e., mean flow time (MFT), makespan (Cmax), mean tardiness (MT), and total setups (TS). A statistical response surface methodology is used to assess the performance measures. ANOVA analysis is used to determine the model’s suitability. The results show that PMF and RF are found as the most common significant input factors for all the performance measures. Multi-objective optimization is performed using the desirability function approach to optimize the system performance measures. It is found that the minimum value of MFT, Cmax, MT, and TS performance measures for optimum performance of the SFJSSP are predicted as 123.432, 220,561, 103.399, and 102,171, respectively, with composite desirability, D of 0.916. The confirmatory results show that the error between the predicted and experimental results is less than 5%. Moreover, considering both uncertainties with dynamic jobs arrival environment shows the study’s real-time scheduling scenario and novelty.

Development and analysis of priority decision rules using MCDM approach for a flexible job shop scheduling: A simulation study

Flexible job shop scheduling problem (FJSSP) is a critical operational planning step in manufacturing that ensures on-time delivery of goods leading to customer satisfaction. The process of assigning jobs to machines has a larger solution space and consumes a huge amount of time and effort. This research focuses on solving FJSSP using the discrete event simulation (DES) model and integrates it with Composite Dispatching Rules (CDRs) and multi-criteria decision making (MCDMs) based priority rules. Performance evaluation of the flexible job shop is carried out to minimize the Makespan, mean Flow-Time, mean Tardiness, and maximum Tardiness. Best performing CDRs and hybrid MCDM techniques are employed for minimizing the objectives considered in this study. The proposed solution framework integrates Fuzzy Analytic Hierarchy Process (FAHP) for assigning weights to the criteria and MCDM approaches namely technique for order of preference by similarity to ideal solution (TOPSIS), The Evaluation based on Distance from Average Solution (EDAS), Compromise Programming (CP), and Weighted Average Method (WAM) separately to prioritize the jobs. Benchmark problems of different sizes are tested using the best performing CDRs available in the literature and proposed MCDM approaches. The proposed approach is implemented in a real-world large-scale flexible job shop (FJS) that produces auto components handling 114 job variants with 245 operations using 28 machines. The performance analysis indicates that no single rule outperforms all the objective measures considered in this study. The MCDMs based rules perform better compared to CDRs for large-scale problems by considering realistic criteria such as demand, due date, setup time, process time, customer priority, and number of operations. The methodology proposed in this study can be tailored in terms of the criteria used for ranking and production parameters to be executable for any real-time instance.

Dynamically adjusting the k-values of the ATCS rule in a flexible flow shop scenario with reinforcement learning

ABSTRACT Given the fact that finding the optimal sequence in a flexible flow shop is usually an NP-hard problem, priority-based sequencing rules are applied in many real-world scenarios. In this contribution, an innovative reinforcement learning approach is used as a hyper-heuristic to dynamically adjust the k-values of the ATCS sequencing rule in a complex manufacturing scenario. For different product mixes as well as different utilisation levels, the reinforcement learning approach is trained and compared to the k-values found with an extensive simulation study. This contribution presents a human comprehensible hyper-heuristic, which is able to adjust the k-values to internal and external stimuli and can reduce the mean tardiness up to 5%.