Online Rescheduling Research Articles

Deep Reinforcement Learning for Integration of Train Trajectory Optimization and Timetable Rescheduling Under Disturbances.

High-speed trains are susceptible to unexpected events such as strong winds and equipment failures, which can result in deviations from the scheduled timetable. As the density of traffic increases, these delays can quickly spread to other trains, eventually leading to conflicts in the timetable. To ensure the efficiency of high-speed railways, quickly resolving potential conflicts and generating appropriate rescheduling schemes are essential. The existing hierarchical structure of train control and online rescheduling tends to be inefficient in terms of information communication and can even lead to unfeasible rescheduled timetables and trajectories. To address these issues, an integrated structure of timetable rescheduling and train trajectory optimization is proposed by introducing the train minimum running time into the process of timetable rescheduling and using the adjusted running time as the objective of trajectory optimization. The integration model is formulated by considering the constraints of timetable rescheduling such as the maximum number of trains overtaking trains, platforms at stations, and the priority of the train, as well as the constraints of trajectory optimization. A deep reinforcement learning (DRL)-based approach is proposed to solve the problem. Numerical experiments are conducted on a segment of the Beijing-Shanghai high-speed railway line, using adapted data to demonstrate the effectiveness of the proposed method in rescheduling timetables and optimizing train trajectories. The results show that the integrated rescheduled timetable and the optimized train trajectory can be generated simultaneously and the computation time exhibits a linear increase with respect to the size of the problem.

Integration of Train Control and Online Rescheduling for High-Speed Railways in Case of Emergencies

The high-speed train control system is essential to the safety and efficiency of train operation. With the rapid increase of high-speed railway (HSR) operating mileage and development of information technology, the disposal flow and methods in emergency response are still based on dispatchers and drivers’ experience within the “layered” architecture of current system. There is a certain gap between current processing methods and effective resolution, which may even cause the spread of delay along with the railway networks. Therefore, we propose an integration system of operation control and online rescheduling to improve the recovery ability of HSR carrying capacity. We first describe the framework, information flow, and disposal process of the current system and analyze the shortcomings in handling emergencies. Then, the basic concept, system structure, and framework of the integration system are introduced. Finally, taking temporary speed restriction caused by strong wind as an example, we also analyze the principle of why and how the integration system can promote the recovery ability of HSR carrying capacity.

Real-time online rescheduling for multiple agile satellites with emergent tasks

The emergent task is a kind of uncertain event that satellite systems often encounter in the application process. In this paper, the multi-satellite distributed coordinating and scheduling problem considering emergent tasks is studied. Due to the limitation of onboard computational resources and time, common online onboard rescheduling methods for such problems usually adopt simple greedy methods, sacrificing the solution quality to deliver timely solutions. To better solve the problem, a new multi-satellite onboard scheduling and coordinating framework based on multi-solution integration is proposed. This method uses high computational power on the ground and generates multiple solutions, changing the complex onboard rescheduling problem to a solution selection problem. With this method, it is possible that little time is used to generate a solution that is as good as the solutions on the ground. We further propose several multi-satellite coordination methods based on the multi-agent Markov decision process (MMDP) and mixed-integer programming (MIP). These methods enable the satellite to make independent decisions and produce high-quality solutions. Compared with the traditional centralized scheduling method, the proposed distributed method reduces the cost of satellite communication and increases the response speed for emergent tasks. Extensive experiments show that the proposed multi-solution integration framework and the distributed coordinating strategies are efficient and effective for onboard scheduling considering emergent tasks.

End-to-end on-line rescheduling from Gantt chart images using deep reinforcement learning

With the advent of the socio-technical manufacturing paradigm, the way in which rescheduling decisions are taken at the shop floor has radically changed in order to guarantee highly efficient production under increasingly dynamic conditions. To cope with uncertain production environments, a drastic increase in the type and degree of automation used at the shop floor for handling unforeseen events and unplanned disturbances is required. In this work, the on-line rescheduling task is modelled as a closed-loop control problem in which an artificial autonomous agent implements a control policy generated off-line using a schedule simulator to learn schedule repair policies directly from high-dimensional sensory inputs. The rescheduling control policy is stored in a deep neural network, which is used to select repair actions in order to achieve a small set of repaired goal states. The rescheduling agent is trained using Proximal Policy Optimisation based on a wide variety of simulated transitions between schedule states using colour-rich Gantt chart images and negligible prior knowledge as inputs. An industrial example is discussed to highlight that the proposed approach enables end-to-end deep learning of successful rescheduling policies to encode task-specific control knowledge that can be understood by human experts.

Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning

In modern volatile and complex manufacturing environment, dynamic events such as new job insertions and machine breakdowns may randomly occur at any time and different objectives in conflict with each other should be optimized simultaneously, leading to an urgent requirement of real-time multi-objective rescheduling methods that can achieve both time efficiency and solution quality. In this regard, this paper proposes an on-line rescheduling framework named as two-hierarchy deep Q network (THDQN) for the dynamic multi-objective flexible job shop scheduling problem (DMOFJSP) with new job insertions. Two practical objectives including total weighted tardiness and average machine utilization rate are optimized. The THDQN model contains two deep Q network (DQN) based agents. The higher-level DQN is a controller determining the temporary optimization goal for the lower DQN. At each rescheduling point, it takes the current state features as input and chooses a feasible goal to guide the behaviour of the lower DQN. Four different goals corresponding to four different forms of reward functions are suggested, each of which optimizes an indicator of tardiness or machine utilization rate. The lower-level DQN acts as an actuator. It takes the current state features together with the higher optimization goal as input and chooses a proper dispatching rule to achieve the given goal. Six composite dispatching rules are developed to select an available operation and assign it on a feasible machine, which serve as the candidate action set for the lower DQN. A novel training framework based on double DQN (DDQN) is designed. The trained THDQN is compared with each proposed composite dispatching rule, existing well-known dispatching rules as well as other reinforcement learning based scheduling methods on a wide range of test instances. Results of numerical experiments have confirmed both the effectiveness and generality of the proposed THDQN.

Reinforcement learning of adaptive online rescheduling timing and computing time allocation

Mathematical optimization methods have been developed to a vast variety of complex problems in the field of process systems engineering (e.g., the scheduling of chemical batch processes). However, the use of these methods in online scheduling is hindered by the stochastic nature of the processes and prohibitively long solution times when optimized over long time horizons. The following questions are raised: When to trigger a rescheduling, how much computing resources to allocate, what optimization strategy to use, and how far ahead to schedule? We propose an approach where a reinforcement learning agent is trained to make the first two decisions (i.e., rescheduling timing and computing time allocation). Using neuroevolution of augmenting topologies (NEAT) as the reinforcement learning algorithm, the approach yields, on average, better closed-loop solutions than conventional rescheduling methods on three out of four studied routing problems. We also reflect on expanding the agent’s decision-making to all four decisions.

Online Scheduling for Dynamic VM Migration in Multicast Time-Sensitive Networks

With the development of hardware virtualization and cloud computing, modern industry has a tendency to upgrade from the traditional industrial networks to virtual machine (VM) based networks. To provide firm latency guarantees for control messages in these networks, the time-sensitive network (TSN) is a promising technology due to its determinacy for real-time applications. However, TSN faces the challenge of providing a rapid response to dynamic transmission requirement changes incurred by VM migrations. In this paper, we proposed an online scheduling approach to deal with dynamic VM migrations in multicast TSN. In this approach, we devise a novel online scheduling framework [minimal distance tree (MDT) construction - heuristic breadth first search] containing an offline scheduling phase and an online rescheduling phase. While the offline phase introduces a MDT to increase reusable scheduling results, the online phase proposes a heuristic scheduling approach to reuse the results of the offline phase as much as possible to accelerate the rescheduling process. Experiments show that our framework can provide a rapid response to dynamic VM migrations compared with the existing approaches where the amount of control data does not exceed 50% of the bandwidth.

Incorporating automation logic in online chemical production scheduling

In this paper, we address the problem of incorporating knowledge of the automation system in a chemical plant into the online scheduling problem. Optimization models for online scheduling necessarily omit some of the plant dynamics to ensure sufficiently fast solution times for use in online rescheduling. This can result in the computed schedules not being feasible when executed in the plant. To overcome this difficulty, we propose to use online analysis of a formal model of the automation logic to detect these infeasible schedules and avoid them when rescheduling. Model checking is applied to an abstraction of the automation system’s dynamics to detect infeasible schedules, and a state-space resource task network scheduling model is used to account for the associated delay information. We demonstrate the techniques using an illustrative running example, and show the utility of the integrated approach using a case study involving multiple batch reactions.

Scheduling of Energy-Integrated Batch Process Systems Using a Pattern-Based Framework

In this paper, a novel pattern-based method is developed for the generation of optimal schedules for energy-integrated batch process systems. The proposed methodology is based on the analysis of available schedules for the identification of repetitive patterns. It is shown that optimal schedules of energy-integrated batch processes are composed of several repeating sections (or building blocks), and their sizes and relative positions are dependent on the scheduling horizon and constraints. Based on such a decomposition, the proposed pattern-based algorithm generates an optimal schedule by computing the number and sequence of these blocks. The framework is then integrated with rigorous optimization-based approach wherein it is shown that the learning from the pattern-based solution significantly improves the performance of rigorous optimization. The main advantage of the pattern-based method is the significant reduction in computational time required to solve large scheduling problems, thus enabling the possibility of on-line rescheduling. Three literature examples were considered to demonstrate the presence of repeating patterns in optimal schedules of energy-integrated batch systems. The effectiveness of the proposed methodology was illustrated using an integrated reactor-separator system.

Reliable, Distributed Scheduling and Rescheduling for Time-Critical, Multiagent Systems

This paper addresses two main problems with many heuristic task allocation approaches—solution trapping in local minima and static structure. The existing distributed task allocation algorithm known as performance impact (PI) is used as the vehicle for developing solutions to these problems as it has been shown to outperform the state-of-the-art consensus-based bundle algorithm for time-critical problems with tight deadlines, but is both static and suboptimal with a tendency toward trapping in local minima. This paper describes two additional modules that are easily integrated with PI. The first extends the algorithm to permit dynamic online rescheduling in real time, and the second boosts performance by introducing an additional soft-max action-selection procedure that increases the algorithm’s exploratory properties. This paper demonstrates the effectiveness of the dynamic rescheduling module and shows that the average time taken to perform tasks can be reduced by up to 9% when the soft-max module is used. In addition, the solution of some problems that baseline PI cannot handle is enabled by the second module. These developments represent a significant advance in the state of the art for multiagent, time-critical task assignment. Note to Practitioners —This work was motivated by the limitations of current agent-to-task allocation algorithms that do not use a central server for communication. In previously published work, the current state-of-the-art consensus-based bundle algorithm has demonstrated poor performance when applied to model task allocation problems with critical time limits, often failing to assign all of the tasks, especially when the deadlines are tight. The performance impact (PI) algorithm has a much better success rate with these model problems but would be flawed when applied to real missions because it has no mechanism for online replanning when new information becomes available. In addition, it is somewhat restricted in the way it searches for a problem solution, meaning that more efficient plans are often available but are not discovered. This paper tackles both of these shortcomings. The PI algorithm is extended to include a module that permits rescheduling when necessary, and a further module is introduced that widens the scope of the solution search. A third module that is able to offer robust plans, even for large-scaled missions involving many agents and tasks, has also been developed, although it is not discussed here. Implementation and testing of a version of PI that incorporates all three of these modules are the final goal of this research.

On a shared human-robot task scheduling and online re-scheduling

Human-Robot (HR) teams are expected to work more efficiently and interdependently compared to the manual non-added value manufacturing processes. This paper discusses a method for enabling the shared HR task scheduling and dynamic re-scheduling. Humans, robots, and HR teams are involved in the resources model, while the task sequence and the task precedence constraints consist of the workload model. An intelligent multi-criteria decision-making framework is proposed for evaluating alternative solutions for the shared HR task scheduling. Dynamic re-scheduling is possible in the cases of unexpected events during the execution of the task. The dynamic rescheduling is triggered by IEC61499 events generated in running time. The method has been implemented as a software tool in Java and has been demonstrated in the hybrid assembly of a vehicle turbocharger.

Модели, методы и алгоритмы планирования в грид и облачных вычислениях

Certain intricate problems, e.g., processing the results of physical experiments in the large hadron collider at the CERN, utilize significant distributed computing resources, part of which is shared with their owners. This factor, even within the framework of virtual organizations, causes competition for utilization of resources between both independent users and global (users’) and local job flows of computing resource owners. This circumstance adds much difficulty to the problem of providing the required quality of service in scalable computing. The presently known scheduling algorithms, their combinations and heuristics do not offer tools for producing efficient schedules under the conditions of heterogeneous distributed environments with a dynamically changing composition of computing resources. Under such conditions, so called economic models for allocating non-dedicated resources and for scheduling distributed computing turn out to be highly efficient in fields like grid and cloud computing, and multiagent systems. The article analyzes the current state of investigations in the field of methods and algorithms for efficiently scheduling competing flows of structured and parallel jobs in distributed heterogeneous computing environments, namely, in grid infrastructures and in cloud services. Primary attention is paid to methods and tools for scheduling intricate jobs and their flows with due regard to the preferences of stakeholders (users, resource owners, and administrators), and specific features of applications. With these factors duly taken into account, more efficient utilization of the resources in distributed computing environments may be achieved. The article presents an approach for comprehensively solving the problem of scheduling intricate jobs and their flows. Execution of a user job involves the need to allocate a set of simultaneously accessible computing resources (slot windows). The problem of selecting suitable resources is complicated by such factors as the availability of local high-priority jobs, the dynamics of resources utilization, and advanced reservations. The computing node load schedules are kept in line with the actual demands by means of a scheduling horizon. Online rescheduling implies the need to minimize the computational complexity of the relevant algorithms. Competition among the multitude of different job flows is a factor that degrades the level of resource accessibility and decreases the number of alternative scheduling options. One possible approach for solving this problem is to use anticipation scheduling, a method that makes it possible to increase the number of possible alternative schedules by means of special slot selection mechanisms. The anticipation scheduling procedure opens the possibility to evaluate the efficiency of different resource combinations. The obtained data serve as a basis for producing a feasible job flow execution schedule close to the optimal solution.

Cloud Avoidance Scheduling Algorithm for Agile Optical Satellites

The existence of cloud seriously influences the imaging quality and efficiency of traditional optical satellites, which can be overcome thanks to the enhancement of the mobility of the new generation of agile satellites. The problem of agile optical satellite scheduling considering real-time cloud information is therefore investigated. A two-phased scheduling framework, containing off-line scheduling on the ground and on-line rescheduling onboard, is proposed. An algorithm based on the ant colony algorithm is designed to solve this problem. An on-line re-scheduling algorithm based on confliction sliding strategy is proposed. A series of experiments are carried out to testify the effectiveness of the algorithm. The results show that almost 50 % of the satellite observation capacity is saved with consideration of real-time cloud information.

A Pattern-based Method for Scheduling of Energy-integrated Batch Process Networks

This paper describes a pattern-based method for the generation of flexible-horizon production schedules for energy-integrated batch systems using a benchmark reactor-distillation system. The proposed methodology consists of structural decomposition of an already existing schedule for a fixed time horizon to identify repetitive patterns. It is found that such a schedule can be divided into three sections - initial, final and an intermediate section - of manipulable sizes. Based on such a decomposition, it is possible to generate an extended (or shorter) horizon schedule by increasing (or decreasing) the sizes of these sections. The proposed method allows for the generation of new schedules without solving mathematically rigorous mixed integer programming problems. The key advantage of the proposed method is the significant reduction in the time required for solving large scheduling problems, especially for online rescheduling applications. The effectiveness of the proposed methodology is illustrated through the same benchmark scheduling problem.

Online Rescheduling in Semiconductor Manufacturing

Semiconductor manufacturing rescheduling problems are difficult to solve in real time due to numerous disturbances. We focus on maintaining the existing schedule and propose online semiconductor manufacturing rescheduling. To handle temporary disturbances and obtain a new feasible schedule, our proposal revises the given schedule based on affected operation rescheduling with operation sorting (sorting-AOR) based on schedule stability. To reduce deadline violations, we propose improving schedules two or more days ahead based on a fast local search algorithm for semicritical paths. In problems with 200,000 processes, this handles disturbances and gets new feasible schedules and better revised schedules in less than 1 minute. This is more effective than conventional dispatching-rule-based approaches, some of which have been applied in different facilities because it provides higher schedule stability and fewer deadline violations.

An online approach to dynamic rescheduling for production planning applications

Manufacturing firms must often consider how to plan the production of a new order while containing the impact on the existing plan. Two approaches are typically used to solve the problem: online scheduling and rescheduling. State-of-the-art for both strategies is analyzed and methods available in the literature are proved to be inefficient for real world cases coming from production planning problems in a manufacturing firm. We propose an alternative approach combining the most effective aspects of both traditional approaches. The already available production plan and the characteristics of the new order to be planned are studied and used in order to generate a new production plan that meets two requirements: containing the number of changes to the existing plan and minimizing the delays due to the newly planned order. Constraint programming is used to implement the proposed approach. Results on case studies are also provided to evaluate the effectiveness of the proposed approach.

Workload-ahead-driven online energy minimization techniques for battery-powered embedded systems with time-constraints

This article proposes a new online voltage scaling (VS) technique for battery-powered embedded systems with real-time constraints. The VS technique takes into account the execution times and discharge currents of tasks to further reduce the battery charge consumption when compared to the recently reported slack forwarding technique [Ahmed and Chakrabarti 2004], while maintaining low online complexity of O(1). Furthermore, we investigate the impact of online rescheduling and remapping on the battery charge consumption for tasks with data dependency which has not been explicitly addressed in the literature and propose a novel rescheduling/remapping technique. Finally, we take leakage power into consideration and extend the proposed online techniques to include adaptive body biasing (ABB) which is used to reduce the leakage power. We demonstrate and compare the efficiency of the presented techniques using seven real-life benchmarks and numerous automatically generated examples.

Workload-ahead-driven online energy minimization techniques for battery-powered embedded systems with time-constraints

This article proposes a new online voltage scaling (VS) technique for battery-powered embedded systems with real-time constraints. The VS technique takes into account the execution times and discharge currents of tasks to further reduce the battery charge consumption when compared to the recently reported slack forwarding technique [Ahmed and Chakrabarti 2004], while maintaining low online complexity of O (1). Furthermore, we investigate the impact of online rescheduling and remapping on the battery charge consumption for tasks with data dependency which has not been explicitly addressed in the literature and propose a novel rescheduling/remapping technique. Finally, we take leakage power into consideration and extend the proposed online techniques to include adaptive body biasing (ABB) which is used to reduce the leakage power. We demonstrate and compare the efficiency of the presented techniques using seven real-life benchmarks and numerous automatically generated examples.

On-Line Simulation and Control in Manufacturing Systems

In this paper a novel on-line simulation framework for multi-resource (i.e., machines-workers-auxiliary resources) constrained Flexible Manufacturing Systems (FMSs) is presented. This framework is based on combined optimization and simulation techniques. Optimization procedures are used for initial off-line scheduling and on-line rescheduling (in case of disturbance), using Petri nets, genetic algorithms and dispatching rules. Various performance measures such as minimizing the makespan and the mean flow time were considered. This framework is Implemented using Arena, Microsoft Access, and Microsoft Visual C/C++. Random events are introduced on-line using a user interface provided by Arena Real-time template. The proposed framework will be implemented for the control of an industrial flexible printed circuit board assembly system.

Online rescheduling of mass rapid transit systems: fuzzy expert system approach

The paper is concerned with the online rescheduling of mass rapid transit (MRT) trains after sudden increases of passenger flow. In particular, the use of predictive fuzzy control (PFC) for carrying out the train dwell-time control in an event-driven framework is considered. Two fuzzy expert systems for improving the performance of MRT systems and for enhancing functions of the typical automatic train control (ATC) systems are proposed. Using the event-driven approach, the only decisions to be made in each of the two fuzzy expert systems will be whether or not to dispatch the train at a proposed reschedule time. Due to its simplicity, the proposed methodology is fast enough for online implementation. An object-oriented environment has been developed for implementing the control and simulation algorithms that reduces the efforts required for understanding the interactive operation of railway subsystems and for preparing study data. Therefore, the efforts for future development will be reduced. The proposed approach is used for adjusting the train dwell time and for studying the effects on the MRT performance. Results show that the train dwell time adjustment is an effective means of maintaining the quality of train service after sudden load disturbances. Several simulated case studies are included to illustrate the effectiveness of the proposed approach.