Dynamic Scheduling Problem Research Articles

A Rescheduling Strategy for Multipurpose Batch Processes with Processing Time Variation and Demand Uncertainty

In this paper, we address the problem of dynamic scheduling of a multipurpose batch process subject to two types of disturbances, namely, processing time variation and demand uncertainty. We propose a rescheduling strategy that combines several ideas. First, when we generate a new schedule, we simultaneously construct a Directed Acyclic Graph (DAG) to represent this new schedule. While each node in the DAG represents an operation, each arc represents the dependency of an operation on another. Based on this DAG, we then use a simple procedure to determine how long an operation is allowed to be delayed without affecting the current makespan. After that, when the new schedule is used for online execution, we trigger a rescheduling procedure only when (1) we infer from the predetermined delayable time information that the current makespan will be extended, or (2) we observe new demands, or (3) the current schedule is not guaranteed to be feasible. In the rescheduling procedure, only the affected operations are allowed to be revised, while those unaffected operations are fixed. By doing this, we can reduce system nervousness and improve computational efficiency. The computational results demonstrate that our method can achieve an order of magnitude of reduction in both the number of operation changes and the computational time with a slightly better long-term makespan, compared to the widely used periodically–completely rescheduling strategy.

Multi-objective dynamic distributed flexible job shop scheduling problem considering uncertain processing time

Multi-objective dynamic distributed flexible job shop scheduling problem considering uncertain processing time

Piecewise linear value function approximations in nonlinear dynamic scheduling problems with VTOLs

Modern vertical take-off and landing vehicles (VTOLs) could significantly affect the future of mobility. The broad range of their application fields encompasses urban air mobility, transportation and logistics as well as reconnaissance and observation missions in a military context. This article presents an efficient numerical framework for dynamic scheduling of multiple VTOLs. It combines a scheduling problem with optimal trajectory planning. The whole setting is formulated as a mixed-integer bilevel optimization problem. At the upper level, VTOLs are scheduled, and their starting times are computed. The solution of the lower level problem involves the computation of a value function and yields optimal trajectories for every aerial vehicle. In order to solve the bilevel problem, it is recast into a single-level one. The resulting mixed-integer nonlinear program is then piecewise linearized and solved numerically by a mixed-integer linear solver based on the Branch-and-Bound algorithm. Numerical results prove the feasibility of the approach proposed herein.

A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process

Unpredictable sudden disturbances such as machine failure, processing time lag, and order changes increase the deviation between actual production and the planned schedule, seriously affecting production efficiency. This phenomenon is particularly severe in flexible manufacturing. In this paper, a dynamic scheduling method combining iterative optimization and deep reinforcement learning (DRL) is proposed to address the impact of uncertain disturbances. A real-time DRL production environment model is established for the flexible job scheduling problem. Based on the DRL model, an agent training strategy and an autonomous decision-making method are proposed. An event-driven and period-driven hybrid dynamic rescheduling trigger strategy (HDRS) with four judgment mechanisms has been developed. The decision-making method and rescheduling trigger strategy solve the problem of how and when to reschedule for the dynamic scheduling problem. The data experiment results show that the trained DRL decision-making model can provide timely feedback on the adjusted scheduling arrangements for different-scale order problems. The proposed dynamic-scheduling decision-making method and rescheduling trigger strategy can achieve high responsiveness, quick feedback, high quality, and high stability for flexible manufacturing process scheduling decision making under sudden disturbance.

An effective two-stage memetic algorithm for the dynamic flexible job-shop scheduling problem with job inspection

An effective two-stage memetic algorithm for the dynamic flexible job-shop scheduling problem with job inspection

A reinforcement learning approach for the online dynamic home health care scheduling problem.

Over recent years, home health care has gained significant attention as an efficient solution to the increasing demand for healthcare services. Home health care scheduling is a challenging problem involving multiple complicated assignments and routing decisions subject to various constraints. The problem becomes even more challenging when considered on a rolling horizon with stochastic patient requests. This paper discusses the Online Dynamic Home Health Care Scheduling Problem (ODHHCSP), in which a home health care agency has to decide whether to accept or reject a patient request and determine the visit schedule and routes in case of acceptance. The objective of the problem is to maximize the number of patients served, given the limited resources. When the agency receives a patient's request, a decision must be made on the spot, which poses many challenges, such as stochastic future requests or a limited time budget for decision-making. In this paper, we model the problem as a Markov decision process and propose a reinforcement learning (RL) approach. The experimental results show that the proposed approach outperforms other algorithms in the literature in terms of solution quality. In addition, a constant runtime of less than 0.001 seconds for each decision makes the approach especially suitable for an online setting like our problem.

A novel method for solving dynamic flexible job-shop scheduling problem via DIFFormer and deep reinforcement learning

Due to the dynamic changes of manufacturing environments, heuristic scheduling rules are unstable in dynamic scheduling. Although meta-heuristic methods provide the best scheduling quality, their solution efficiency is limited by the scale of the problem. Therefore, a novel method for solving the dynamic flexible job-shop scheduling problem (DFJSP) via diffusion-based transformer (DIFFormer) and deep reinforcement learning (D-DRL) is proposed. Firstly, the DFJSP is modeled as a Markov decision process, where the state space is constructed in the form of the heterogeneous graph and the reward function is designed to minimize the makespan and maximize the machine utilization rate. Secondly, DIFFormer is used to encode the operation and machine nodes to better capture the complex dependencies between nodes, which can effectively improve the representation ability of the model. Thirdly, a selective rescheduling strategy is designed for dynamic events to enhance the solution quality of DFJSP. Fourthly, the twin delayed deep deterministic policy gradient (TD3) algorithm is adopted for training an efficient scheduling model. Finally, the effectiveness of the proposed D-DRL is validated through a series of experiments. The results indicate that D-DRL achieves better solution quality and higher solution efficiency when solving DFJSP instances.

Fuzzy flexible job shop dynamic scheduling considering machine remaining processing capacity

Dynamic fuzzy flexible job shop scheduling (DFFJSP) is significant in the medical equipment industry. Generally the dynamic flexible job shop scheduling problem considers a single dynamic factor or ignores resources such as machine itself also plays a huge role in dealing dynamic issues. Thus fuzzy processing time and machine breakdowns are considered in this paper. In order to copy with two problems mentioned above, the model of DFFJSP containing two objectives of makespan and energy consumption is established. To solve this problem, the remaining processing capacity of broken machine is applied to weaken the impacts of machine breakdowns and a multi-objective evolutionary algorithm (NG-WOA) is proposed, which integrating NSGA-II and whale optimal algorithm (WOA). Furthermore, the joining of variable neighborhood search makes NG-WOA skip the local optimum solution in time. Through experiments, the validity of remaining processing capacity is verified in rescheduling scheme. Then NG-WOA algorithm is proven can find better solutions when compared with other two algorithms according to the distribution of pareto forefront solutions in benchmark examples. The results of this paper show that the remaining processing capacity has greater advantages in guaranteeing the stability of dynamic scheduling problems, because the machine’s processing capacity can be utilized fully compared to previous strategies for fault disturbances.

A multi-level rescheduling approach for a dynamic remote operations scheduling problem

ABSTRACT In this paper, we tackle a dynamic scheduling problem faced by a large international company. The problem involves assigning installation projects arriving over time to specialised technicians who execute them remotely. Each project consists of several tasks having processing times, release dates, and execution deadlines. The company needs to assign projects to technicians and schedule tasks complying with technicians' skills, precedence constraints between tasks, and tasks requiring multiple technicians simultaneously. The problem is dynamic as new projects and tasks become available over time, requiring their allocation to technicians. We formulate the offline problem as a mixed integer linear program that minimises the makespan, and we address the dynamic version solving restricted problems within a rolling horizon framework. The approach systematically implements different levels of schedule adjustment to incorporate new information. To study scalability, we validate our algorithm by using both real-world and synthetic simulations demonstrating its efficiency and effectiveness. Additionally, we provide interesting managerial insights for the company.

Probing an LSTM-PPO-Based reinforcement learning algorithm to solve dynamic job shop scheduling problem

With the growth of personalized demand and the continuous improvement in social productivity, the large-scale and few-variety centralized production model is gradually transitioning towards a personalized model of small batches and multiple varieties, which makes the manufacturing process of the job shop increasingly complex. Furthermore, disruptive events such as machinery failures and rush orders in the job shop increase the uncertainty and variability of the production environment. Traditional scheduling methods are usually based on fixed rules and heuristic algorithms, which are difficult to adapt to constantly changing production environments and demands. This may lead to inaccurate scheduling decisions and hinder the optimal allocation of job shop resources. To solve the dynamic job shop scheduling problem (JSP) more effectively, this paper proposes a Reinforcement Learning (RL) optimization algorithm integrating long short-term memory (LSTM) neural network and proximal policy optimization (PPO). It can dynamically adjust scheduling strategies according to the changing production environment, achieving comprehensive status awareness of the job shop environment to make optimal scheduling decisions. First, a state-aware network framework based on LSTM-PPO is proposed to achieve real-time perception of job shop state changes. Then, the state and action space of the job shop are described within the context of the state-aware network framework. Finally, an experimental environment is established to verify the algorithm’s effectiveness. Training the LSTM-PPO algorithm makes it feasible to achieve better performance than other scheduling methods. By comparing the initial planning time with the actual completion time of the rescheduling decision under different dynamic disturbances, the efficiency of the proposed algorithm is verified for the dynamic JSP.

Flight Arrival Scheduling via Large Language Model

The flight arrival scheduling problem is one of the critical tasks in air traffic operations, aiming to ensure that the flight arrive in the correct sequence safely. Existing methods primarily focus on the terminal area and often overlook the presence of training flight at the airport. Due to the limited generalization of traditional methods and varying control practices at different airports, training flight at airports still rely on manual control for arrival sorting. To effectively address these issues, we propose a novel method for slot allocation that leverages the strong reasoning capabilities and generalization potential of large language models (LLMs). Our method conceptualizes the dynamic scheduling problem for training flight as a language modeling problem, a perspective not previously explored. Specifically, we represent the allocator’s inputs and outputs as language tokens, utilizing LLMs to generate conflict-free results based on a language description of requested landing information and assigned training flight information. Additionally, we employ a reset strategy to create a small dataset for scenario-specific samples, enabling LLMs to quickly learn allocation schemes from the dataset. We demonstrated the capability of LLMs in addressing time conflicts by evaluating metrics such as answer accuracy, conflict rate, and total delay time (without the wrong answer). These findings underscore the feasibility of employing LLMs in the field of air traffic control.

Whittle index approach to the multi-class queueing systems with convex holding costs and IHR service times

AbstractWe consider the dynamic scheduling problem in a single-server multi-class queueing system, where the accumulated holding costs of a customer is a convexly increasing function of the time that the customer spends in the system. This problem was already addressed by van Mieghem in his seminal paper (Ann Appl Probab 5:808–833, 1995). He launched the so-called generalized $$c\mu $$ c μ rule and proved its asymptotic optimality in the heavy traffic limit. The optimal scheduling problem with convex holding costs have also been studied in many other papers. However, the holding costs in these papers are typically count-convex, i.e., the aggregate holding cost rate related to a class of customers increases convexly as a function of the current number of customers in this class, which is clearly different from the time-convex holding costs introduced in van Mieghem (Ann Appl Probab 5:808–833, 1995) and studied also in the present paper. We utilize the Whittle index approach to develop a novel heuristic policy for the dynamic scheduling problem with time-convex holding costs. Instead of the original open version of the continuous-time scheduling problem, we apply the Whittle index approach to the corresponding discrete-time closed version of the problem. As the main theoretical contribution, we prove that, for IHR service times, the discounted discrete-time problem is indexable and also give an explicit expression for the Whittle index itself. A special challenge in our model is the two-dimensional state space with an infinite number of possible states. By utilizing the discrete-time results, we develop a novel index policy for the original continuous-time problem and demonstrate, by numerical simulations, that this Whittle index policy outperforms the generalized $$c\mu $$ c μ rule. Thus, while the generalized $$c\mu $$ c μ rule is asymptotically optimal in the heavy traffic limit, it can still be improved in normal traffic conditions.

Enhanced Monte‐Carlo tree search for dynamic flexible job shop scheduling with transportation time constraints

AbstractWith the increasing uncertainty and complexity of industrial production, dynamic events are inevitably happened during the production process. This paper deals with the transportation time constrained dynamic flexible job shop scheduling problem with machine breakdown, new jobs arrival, jobs cancellation, and processing time change of operations. Considering the actual production situation, a makespan rescheduling model based on transportation time is established. A Monte‐Carlo tree search based algorithm is proposed to solve the dynamic flexible job shop scheduling problem such that the makespan is minimized. In order to improve the accuracy of the developed algorithm and the quality of the obtained scheduling schemes, subtree keeping policy, multi‐branching simulation, reinforcement learning, greedy strategy, and expectation‐best evaluation method are employed. Moreover, a subtree pruning policy is embedded to improve the search efficiency of the developed algorithm. Simulation experiments are conducted on a series of instances of different sizes to compare the developed approach with the GA‐based scheduling approach and the Monte‐Carlo tree search based approach. The simulation results indicate that the developed approach is superior to the existing scheduling approaches in terms of the qualities of the obtained solutions.

Dynamic scheduling of hybrid flow shop problem with uncertain process time and flexible maintenance using NeuroEvolution of Augmenting Topologies

AbstractA hybrid flow shop is pivotal in modern manufacturing systems, where various emergencies and disturbances occur within the smart manufacturing context. Efficiently solving the dynamic hybrid flow shop scheduling problem (HFSP), characterised by dynamic release times, uncertain job processing times, and flexible machine maintenance has become a significant research focus. A NeuroEvolution of Augmenting Topologies (NEAT) algorithm is proposed to minimise the maximum completion time. To improve the NEAT algorithm's efficiency and effectiveness, several features were integrated: a multi‐agent system with autonomous interaction and centralised training to develop the parallel machine scheduling policy, a maintenance‐related scheduling action for optimal maintenance decision learning, and a proactive scheduling action to avoid waiting for jobs at decision moments, thereby exploring a broader solution space. The performance of the trained NEAT model was experimentally compared with the Deep Q‐Network (DQN) and five classical priority dispatching rules (PDRs) across various problem scales. The results show that the NEAT algorithm achieves better solutions and responds more quickly to dynamic changes than DQN and PDRs. Furthermore, generalisation test results demonstrate NEAT's rapid problem‐solving ability on test instances different from the training set.

AI Services-Oriented Dynamic Computing Resource Scheduling Algorithm Based on Distributed Data Parallelism in Edge Computing Network of Smart Grid

Massive computational resources are required by a booming number of artificial intelligence (AI) services in the communication network of the smart grid. To alleviate the computational pressure on data centers, edge computing first network (ECFN) can serve as an effective solution to realize distributed model training based on data parallelism for AI services in smart grid. Due to AI services with diversified types, an edge data center has a changing workload in different time periods. Selfish edge data centers from different edge suppliers are reluctant to share their computing resources without a rule for fair competition. AI services-oriented dynamic computational resource scheduling of edge data centers affects both the economic profit of AI service providers and computational resource utilization. This letter mainly discusses the partition and distribution of AI data based on distributed model training and dynamic computational resource scheduling problems among multiple edge data centers for AI services. To this end, a mixed integer linear programming (MILP) model and a Deep Reinforcement Learning (DRL)-based algorithm are proposed. Simulation results show that the proposed DRL-based algorithm outperforms the benchmark in terms of profit of AI service provider, backlog of distributed model training tasks, running time and multi-objective optimization.

Tri-objective lot-streaming scheduling optimization for hybrid flow shops with uncertainties in machine breakdowns and job arrivals using an enhanced genetic programming hyper-heuristic

Lot-streaming scheduling has been widely recognized as a means to improve shop productivity, but there is few research on lot-streaming scheduling problems under dynamic disturbances. To fill the gap, lot-streaming scheduling optimization approach for hybrid flow shops with uncertainties in machine breakdowns and job arrivals is proposed. A mathematical model is formulated with objectives of minimizing maximum tardiness, total idle energy consumption of the machine, and maximum makespan. Since the Genetic Programming Hyper Heuristic algorithm has better results in solving dynamic scheduling problems, a Collaborative Harmony Search-based Genetic Programming Hyper Heuristic (CHS-GPHH) is presented to solve the dynamic lot-streaming hybrid flow shop scheduling problem (DLS-HFSSP) with the two dynamic events occurring simultaneously. In the improved algorithm, a neighborhood structure based on harmony search is developed for lot splitting. To verify the effectiveness of the proposed approach, the various comparative studies c are conducted on the lot-streaming dynamic hybrid flow shop scheduling. The results demonstrate the effectiveness of each improvement component of the CHS-GPHH, and verify that CHS-GPHH is an effective approach to deal with DLS-HFSSP with the in all the scenarios.

Digital Twin-Driven Dynamic Scheduling of Flexible Manufacturing System in the Context of Smart Factory Producing Brass Accessories

This study aims to deal with a dynamic scheduling problem of a real Flow Shop follow-up of an assembly process considering the specific constraints and requirements of a brass accessories manufacturing (BAM) company. We basically set up a Digital Twin-driven dynamic scheduling approach for Industry 4.0 by addressing uncertainties of machine availability. In fact, the setup of this Digital Twin (DT) is the outcome of the combination of both optimization and simulation. For the first step, we have elaborated a Mixed Integer Linear Programming (MILP) scheduling model by taking into account the dedicated requisites of our case study. Concerning simulation, a 3D simulation platform of a workshop producing brass accessories controlled by a Cyber-Physical Production System (CPPS) has been developed, in our previous work, including constraints and stochastic aspects. The simulation constraints are difficult or impossible to be modeled in the MILP model. These designs are integrated with the real workshop to construct the Digital Twin. The proposed tool enables the rescheduling of production orders based on machine disturbances and unpredictability. Validation scenarios have been designed and conducted to highlight the efficacy of the Digital Twin approach utilizing the brass accessories case study. We are confident that this represents the initial endeavor addressing the optimization of production rescheduling in a flow shop follow-up of a mixed assembly system.

A transformer-based deep reinforcement learning approach for dynamic parallel machine scheduling problem with family setups

A transformer-based deep reinforcement learning approach for dynamic parallel machine scheduling problem with family setups

Study on dynamic scheduling method of airport refueling vehicles based on DQN

Aiming at the low utilization rate of airport refueling vehicles and long solution time of exact algorithm caused by the uncertainty of actual flight time, a deep Q network dynamic scheduling method for refueling vehicles combining with the multi-objective deep reinforcement learning framework was proposed. Firstly, an optimization model is established to maximize the on-time rate of refueling tasks and the average proportion of idle vehicles. Then, the five state features that measure the current state of the vehicle are designed as inputs to the network. According to the two objectives, the two scheduling strategies are proposed as the action space so that the algorithm can generate the dynamic scheduling scheme based on the dynamic flight data in real time. Finally, the dynamic scheduling model for airport refueling vehicles is solved, and the effectiveness and real-time performance of the algorithm are verified by different scale examples. The results show that the average number of on-time refueling tasks per day is 9.43 more than that via manual scheduling, and the average working time of vehicles is reduced by 57.6 minutes, which shows the excellent ability of the present method in solving the dynamic scheduling problem of refueling vehicles.

Out-of-order execution enabled deep reinforcement learning for dynamic additive manufacturing scheduling

Additive Manufacturing (AM) has revolutionized the production landscape by enabling on-demand customized manufacturing. However, the efficient management of dynamic AM orders poses significant challenges for production planning and scheduling. This paper addresses the dynamic scheduling problem considering batch processing, random order arrival and machine eligibility constraints, aiming to minimize total tardiness in a parallel non-identical AM machine environment. To tackle this problem, we propose the out-of-order enabled dueling deep Q network (O3-DDQN) approach. In the proposed approach, the problem is formulated as a Markov decision process (MDP). Three-dimensional features, encompassing dynamic orders, AM machines, and delays, are extracted using a ‘look around’ method to represent the production status at a rescheduling point. Additionally, five novel composite scheduling rules based on the out-of-order principle are introduced for selection when an AM machine completes processing or a new order arrives. Moreover, we design a reward function that is strongly correlated with the objective to evaluate the agent’s chosen action. Experimental results demonstrate the superiority of the O3-DDQN approach over single scheduling rules, randomly selected rules, and the classic DQN method. The average improvement rate of performance reaches 13.09% compared to composite scheduling rules and random rules. Additionally, the O3-DDQN outperforms the classic DQN agent with a 6.54% improvement rate. The O3-DDQN algorithm improves scheduling in dynamic AM environments, enhancing productivity and on-time delivery. This research contributes to advancing AM production and offers insights into efficient resource allocation.