Heuristic Scheduling Research Articles

Optimization of High-Performance Computing Job Scheduling Based on Offline Reinforcement Learning

In large-scale, distributed high-performance computing systems, the increasing complexity of job scheduling has expanded along with the growth of computational resources and job diversity. While heuristic scheduling strategies with various optimization objectives have shown promising results, their effectiveness is often limited in real-world applications due to the dynamic nature of workloads and system configurations. Deep reinforcement learning (DRL) methods offer the potential to address scheduling challenges. However, their trial-and-error learning approach can lead to suboptimal performance or resource wastage in the early stages. To mitigate these risks, this paper introduces an offline reinforcement learning-based job scheduling method. By training on historical data, the method avoids the pitfalls of deploying immature strategies in live environments. We constructed an offline dataset by combining expert scheduling trajectories with early-stage trial data from online reinforcement learning. This enables the development of more robust scheduling policies. Experimental results demonstrate that, compared to heuristic and online DRL algorithms, the proposed approach achieves more efficient scheduling performance across various workloads and optimization goals, showcasing its practicality and broad applicability.

A novel method for solving dynamic flexible job-shop scheduling problem via DIFFormer and deep reinforcement learning

Due to the dynamic changes of manufacturing environments, heuristic scheduling rules are unstable in dynamic scheduling. Although meta-heuristic methods provide the best scheduling quality, their solution efficiency is limited by the scale of the problem. Therefore, a novel method for solving the dynamic flexible job-shop scheduling problem (DFJSP) via diffusion-based transformer (DIFFormer) and deep reinforcement learning (D-DRL) is proposed. Firstly, the DFJSP is modeled as a Markov decision process, where the state space is constructed in the form of the heterogeneous graph and the reward function is designed to minimize the makespan and maximize the machine utilization rate. Secondly, DIFFormer is used to encode the operation and machine nodes to better capture the complex dependencies between nodes, which can effectively improve the representation ability of the model. Thirdly, a selective rescheduling strategy is designed for dynamic events to enhance the solution quality of DFJSP. Fourthly, the twin delayed deep deterministic policy gradient (TD3) algorithm is adopted for training an efficient scheduling model. Finally, the effectiveness of the proposed D-DRL is validated through a series of experiments. The results indicate that D-DRL achieves better solution quality and higher solution efficiency when solving DFJSP instances.

Proactive Scheduling for mmWave Wireless LANs

To cope with growing wireless bandwidth demand, millimeter wave (mmWave) communication has been identified as a promising technology to deliver Gbps throughput. However, due to the susceptibility of mmWave signals to blockage, applications can experience significant performance variability as users move around due to rapid and significant variation in channel conditions. In this context, proactive schedulers that make use of future data rate prediction have potential to bring a significant performance improvement as compared to traditional schedulers. In this work, we explore the possibility of proactive scheduling that uses mobility prediction and some knowledge of the environment to predict future channel conditions. We present both an optimal proactive scheduler, which is based on an integer linear programming formulation and provides an upper bound on proactive scheduling performance, and a greedy heuristic proactive scheduler that is suitable for practical implementation. Extensive simulation results show that proactive scheduling has the potential to increase average user data rate by up to 35% over the classic proportional fair scheduler without any loss of fairness and incurring only a small increase in jitter. The results also show that the efficient proactive heuristic scheduler achieves from 60% to 75% of the performance gains of the optimal proactive scheduler. Finally, the results show that proactive scheduling performance is sensitive to the quality of mobility prediction and, thus, use of state-of-the-art mobility prediction techniques will be necessary to realize its full potential.

Genetic Programming With Lexicase Selection for Large-Scale Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling is a prominent combinatorial optimisation problem with many real-world applications. Genetic programming has been widely used to automatically evolve effective scheduling heuristics for dynamic flexible job shop scheduling. A limitation of genetic programming is the premature convergence due to the loss of population diversity. To overcome this limitation, this work considers using lexicase selection to improve population diversity, which has achieved success on regression and program synthesis problems. However, it is not trivial to apply lexicase selection to genetic programming for dynamic flexible job shop scheduling, since a fitness case (training scheduling simulation) is often large-scale, making the fitness evaluation very time-consuming. To address this issue, we propose a new multi-case fitness scheme, which creates multiple cases from a single scheduling simulation. Based on the multi-case fitness, we develop a new genetic programming algorithm with lexicase selection, which uses a single simulation for fitness evaluation, thus achieving a better balance between the number of cases for lexicase selection and evaluation efficiency. The experiments on a wide range of dynamic scheduling scenarios show that the proposed algorithm can achieve better population diversity and final performance than the current genetic programming parent selection methods and a state-of-the-art deep reinforcement learning method.

Robust Qubit Mapping Algorithm via Double-Source Optimal Routing on Large Quantum Circuits

Qubit mapping is a critical aspect of implementing quantum circuits on real hardware devices. Currently, the existing algorithms for qubit mapping encounter difficulties when dealing with larger circuit sizes involving hundreds of qubits. In this article, we introduce an innovative qubit mapping algorithm, Duostra, tailored to address the challenge of implementing large-scale quantum circuits on real hardware devices with limited connectivity. Duostra operates by efficiently determining optimal paths for double-qubit gates and inserting SWAP gates accordingly to implement the double-qubit operations on real devices. Together with two heuristic scheduling algorithms, Limitedly Exhaustive Search and Shortest-Path Estimation, it yields results of good quality within a reasonable runtime, thereby striving toward achieving quantum advantage. Experimental results showcase our algorithm’s superiority, especially for large circuits beyond the NISQ era. For example, on large circuits with more than 50 qubits, we can reduce the mapping cost on an average 21.75% over the virtual best results among QMAP, t \(|ket\rangle\) , Qiskit, and SABRE. Besides, for mid-size circuits such as the SABRE-large benchmark, we improve the mapping costs by 4.5%, 5.2%, 16.3%, 20.7%, and 25.7%, when compared to QMAP, TOQM, t \(|ket\rangle\) , Qiskit, and SABRE, respectively.

Simulation Model of a Steelmaking-Continuous Casting Process Based on Dynamic-Operation Rules.

The steelmaking-continuous casting process (SCCP) is a complex manufacturing process which exhibits the distinct features of process manufacturing. The SCCP involves a variety of production elements, such as multiple process routes, a wide array of smelting and auxiliary devices, and a variety of raw and auxiliary materials. The production-simulation of SCCP holds a natural advantage in being able to accurately depict the intricate production behavior involved, and this serves as a crucial tool for optimizing the production operation of the SCCP. This paper thoroughly considers the various production elements involved in the SCCP, such as the fluctuation of the converter smelting cycle, fluctuation of heat weight, and ladle operation. Based on the Plant Simulation software platform, a dynamic simulation model of the SCCP is established and detailed descriptions are provided regarding the design of an SCCP using dynamic-operation rules. Additionally, a dynamic operational control program for the SCCP is developed using the SimTalk language, one which ensures the continuous operation of the caster in the SCCP, using the discrete simulation platform. The effectiveness of the proposed dynamic simulation model is verified by the total completion time of the production plan, the transfer time of the heat among the different processes, and the frequency of ladle turnover. The simulation's results indicate that the dynamic simulation model has a satisfactory effect in simulating the actual production process. On this basis, the application effects of different schedules are compared and analyzed. Compared with a heuristic schedule, the optimized schedule based on the "furnace-machine coordinating" mode reduces the weighted value of total completion time by 8.7 min, reduces the weighted value of transfer waiting time by 45.5 min, and the number of rescheduling times is also reduced, demonstrating a better application effect and verifying the optimizing effect of the "furnace-machine coordinating" mode on the schedule.

AG-SpTRSV: An Automatic Framework to Optimize Sparse Triangular Solve on GPUs

Sparse Triangular Solve (SpTRSV) has long been an essential kernel in the field of scientific computing. Due to its low computational intensity and internal data dependencies, SpTRSV is hard to implement and optimize on GPUs. Based on our experimental observations, existing implementations on GPUs fail to achieve the optimal performance due to their sub-optimal parallelism setups and code implementations, and lack of consideration of the irregular data distribution. Moreover, their algorithm design lacks the adaptability to different input matrices, which may involve substantial manual efforts of algorithm redesigning and parameter tuning for performance consistency. In this work, we propose AG-SpTRSV, an automatic framework to optimize SpTRSV on GPUs, which provides high performance on various matrices while eliminating the costs of manual design. AG-SpTRSV abstracts the procedures of optimizing an SpTRSV kernel as a scheme and constructs a comprehensive optimization space based on it. By defining a unified code template and preparing code variants, AG-SpTRSV enables fine-grained dynamic parallelism and adaptive code optimizations to handle various tasks. Through computation graph transformation and multi-hierarchy heuristic scheduling, AG-SpTRSV generates schemes for task partitioning and mapping, which effectively address the issues of irregular data distribution and internal data dependencies. AG-SpTRSV searches for the best scheme to optimize the target kernel for the specific matrix. A learned lightweight performance model is also introduced to reduce search costs and provide an efficient end-to-end solution. Experimental results with SuiteSparse Matrix Collection on NVIDIA Tesla A100 and RTX 3080 Ti show that AG-SpTRSV outperforms state-of-the-art implementations with geometric average speedups of 2.12x ∼ 3.99x. With the performance model enabled, AG-SpTRSV can provide an efficient end-to-end solution, with preprocessing times ranging from 3.4 to 245 times of the execution time.

Performance of Priority Rules for Finance-Based and Resource-Constrained Project Scheduling Heuristics

Performance of Priority Rules for Finance-Based and Resource-Constrained Project Scheduling Heuristics

Mitigating Co-Activity Conflicts and Resource Overallocation in Construction Projects: A Modular Heuristic Scheduling Approach with Primavera P6 EPPM Integration

This paper proposes a heuristic approach for managing complex construction projects. The tool incorporates Primavera P6 EPPM and Synchro 4D, enabling proactive clash detection and resolution of spatial conflicts during concurrent tasks. Additionally, it performs resource verification for sufficient allocation before task initiation. This integrated approach facilitates the generation of conflict-free and feasible construction schedules. By adhering to project constraints and seamlessly integrating with existing industry tools, the proposed solution offers a comprehensive and robust approach to construction project management. This constitutes, to our knowledge, the first dynamic digital twin for the delivery of a complex project.

Local-search based heuristics for advertisement scheduling

In the MAXSPACE problem, given a set of ads A, one wants to place a subset A' of A into K slots B_1, ..., B_K of size L. Each ad A_i in A has size s_i and frequency w_i. A schedule is feasible if the total size of ads in any slot is at most L, and each ad A_i in A' appears in exactly w_i slots. The goal is to find a feasible schedule that maximizes the space occupied in all slots. We introduce MAXSPACE-RDWV, a MAXSPACE generalization with release dates, deadlines, variable frequency, and generalized profit. In MAXSPACE-RDWV each ad A_i has a release date r_i >= 1, a deadline d_i >= r_i, a profit v_i that may not be related with s_i and lower and upper bounds w^min_i and w^max_i for frequency. In this problem, an ad may only appear in a slot B_j with r_i <= j <= d_i, and the goal is to find a feasible schedule that maximizes the sum of values of scheduled ads. This paper presents some algorithms based on meta-heuristics GRASP, VNS, and Tabu Search for MAX\-SPACE and MAXSPACE-RDWV. We compare our proposed algorithms with Hybrid-GA proposed by Kumar et al (2006). We also create a version of Hybrid-GA for MAXSPACE-RDWV and compare it with our meta-heuristics. Some meta-heuristics like VNS and GRASP+VNS have better results than Hybrid-GA for both problems. In our heuristics, we apply a technique that alternates between maximizing and minimizing the fullness of slots to obtain better solutions. We also applied a data structure called BIT to the neighborhood computation in MAXSPACE-RDWV and showed that this enabled our algorithms to run more iterations.

Genetic Programming and Reinforcement Learning on Learning Heuristics for Dynamic Scheduling: A Preliminary Comparison

Genetic Programming and Reinforcement Learning on Learning Heuristics for Dynamic Scheduling: A Preliminary Comparison

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

The flexible job shop scheduling problem (FJSP) is an important production scheduling problem in intelligent manufacturing. How to model the complex FJSP more accurately and improve the efficiency and generalization of scheduling policies is an urgent challenge to be solved. Therefore, a new end-to-end deep reinforcement learning (DRL) method combined with the meta-path-based heterogeneous graph neural network (MHGNN) is proposed to effectively solve FJSP. The task of solving FJSP is decomposed into two subtasks of operation selection and machine allocation. This dual-task FJSP is represented by introducing a heterogeneous graph and modeled as the dual Markov decision process (DMDP). A MHGNN is proposed to embed the global scheduling states of the dual-task FJSP. A heterogeneous graph neural network (GNN) framework is designed for the dual-task FJSP to efficiently encode the operation nodes and machine nodes, where the extracted embedded feature information is used to represent the operation selection policy and the machine allocation policy. Two policy networks are designed to efficiently predict the operation selection policy and the machine allocation policy, and a soft double-actors critic algorithm is proposed to train these two policy networks, where the trained policies can be used to efficiently solve FJSP instances of different scales. The experiments on three public benchmarks show that the proposed method outperforms the well-known heuristic scheduling rules and some advanced methods for solving FJSP. In particular, when solving large-scale FJSPs, the proposed method performs more prominently in terms of solution quality and solution time.

Hybrid Residual Multiexpert Reinforcement Learning for Spatial Scheduling of High-Density Parking Lots.

Industries, such as manufacturing, are accelerating their embrace of the metaverse to achieve higher productivity, especially in complex industrial scheduling. In view of the growing parking challenges in large cities, high-density vehicle spatial scheduling is one of the potential solutions. Stack-based parking lots utilize parking robots to densely park vehicles in the vertical stacks like container stacking, which greatly reduces the aisle area in the parking lot, but requires complex scheduling algorithms to park and take out the vehicles. The existing high-density parking (HDP) scheduling algorithms are mainly heuristic methods, which only contain simple logic and are difficult to utilize information effectively. We propose a hybrid residual multiexpert (HIRE) reinforcement learning (RL) approach, a method for interactive learning in the digital industrial metaverse, which efficiently solves the HDP batch space scheduling problem. In our proposed framework, each heuristic scheduling method is considered as an expert. The neural network trained by RL assigns the expert strategy according to the current parking lot state. Furthermore, to avoid being limited by heuristic expert performance, the proposed hierarchical network framework also sets up a residual output channel. Experiments show that our proposed algorithm outperforms various advanced heuristic methods and the end-to-end RL method in the number of vehicle maneuvers, and has good robustness to the parking lot size and the estimation accuracy of vehicle exit time. We believe that the proposed HIRE RL method can be effectively and conveniently applied to practical application scenarios, which can be regarded as a key step for RL to enter the practical application stage of the industrial metaverse.

Connection-Aware Heuristics for Scheduling and Distributing Jobs under Dynamic Dew Computing Environments

Due to the widespread use of mobile and IoT devices, coupled with their continually expanding processing capabilities, dew computing environments have become a significant focus for researchers. These environments enable resource-constrained devices to contribute computing power to a local network. One major challenge within these environments revolves around task scheduling, specifically determining the optimal distribution of jobs across the available devices in the network. This challenge becomes particularly pronounced in dynamic environments where network conditions constantly change. This work proposes integrating the “reliability” concept into cutting-edge human-design job distribution heuristics named ReleSEAS and RelBPA as a means of adapting to dynamic and ever-changing network conditions caused by nodes’ mobility. Additionally, we introduce a reinforcement learning (RL) approach, embedding both the notion of reliability and real-time network status into the RL agent. Our research rigorously contrasts our proposed algorithms’ throughput and job completion rates with their predecessors. Simulated results reveal a marked improvement in overall throughput, with our algorithms potentially boosting the environment’s performance. They also show a significant enhancement in job completion within dynamic environments compared to baseline findings. Moreover, when RL is applied, it surpasses the job completion rate of human-designed heuristics. Our study emphasizes the advantages of embedding inherent network characteristics into job distribution algorithms for dew computing. Such incorporation gives them a profound understanding of the network’s diverse resources. Consequently, this insight enables the algorithms to manage resources more adeptly and effectively.

Survey on Genetic Programming and Machine Learning Techniques for Heuristic Design in Job Shop Scheduling

Job shop scheduling is a process of optimising the use of limited resources to improve the production efficiency. Job shop scheduling has a wide range of applications such as order picking in the warehouse and vaccine delivery scheduling under a pandemic. In real-world applications, the production environment is often complex due to dynamic events such as job arrivals over time and machine breakdown. Scheduling heuristics, e.g., dispatching rules, have been popularly used to prioritise the candidates such as machines in manufacturing to make good schedules efficiently. Genetic programming, has shown its superiority in learning scheduling heuristics for job shop scheduling automatically due to its flexible representation. This survey firstly provides comprehensive discussions of recent designs of genetic programming algorithms on different types of job shop scheduling. In addition, we notice that in the recent years, a range of machine learning techniques such as feature selection and multitask learning, have been adapted to improve the effectiveness and efficiency of scheduling heuristic design with genetic programming. However, there is no survey to discuss the strengths and weaknesses of these recent approaches. To fill this gap, this paper provides a comprehensive survey on genetic programming and machine learning techniques on automatic scheduling heuristic design for job shop scheduling. In addition, current issues and challenges are discussed to identify promising areas for automatic scheduling heuristic design in the future.

A Containerized Service-Based Integration Framework for Heterogeneous-Geospatial-Analysis Models

The integration of geospatial-analysis models is crucial for simulating complex geographic processes and phenomena. However, compared to non-geospatial models and traditional geospatial models, geospatial-analysis models face more challenges owing to extensive geographic data processing and complex computations involved. One core issue is how to eliminate model heterogeneity to facilitate model combination and capability integration. In this study, we propose a containerized service-based integration framework named GeoCSIF, specifically designed for heterogeneous-geospatial-analysis models. Firstly, by designing the model-servicized structure, we shield the heterogeneity of model structures so that different types of geospatial-analysis models can be effectively described and integrated based on standardized constraints. Then, to tackle the heterogeneity in model dependencies, we devise a prioritization-based orchestration method, facilitating optimized combinations of large-scale geospatial-analysis models. Lastly, considering the heterogeneity in execution modes, we design a heuristic scheduling method that establishes optimal mappings between models and underlying computational resources, enhancing both model stability and service performance. To validate the effectiveness and progressiveness of GeoCSIF, a prototype system was developed, and its integration process for flood disaster models was compared with mainstream methods. Experimental results indicate that GeoCSIF possesses superior performance in model management and service efficiency.

Heuristic Scheduling for Robotic Job Shops Using Petri Nets and Artificial Potential Fields

Heuristic Scheduling for Robotic Job Shops Using Petri Nets and Artificial Potential Fields

Task Relatedness-Based Multitask Genetic Programming for Dynamic Flexible Job Shop Scheduling

Multitasking learning has been successfully used in handling multiple related tasks simultaneously. In reality, there are often many tasks to be solved together, and the relatedness between them is unknown in advance. In this paper, we focus on multitask genetic programming for the dynamic flexible job shop scheduling problems, and address two challenges. The first is how to measure the relatedness between tasks accurately. The second is how to select task pairs to transfer knowledge during the multitask learning process. To measure the relatedness between dynamic flexible job shop scheduling tasks, we propose a new relatedness metric based on the behaviour distributions of the variable-length genetic programming individuals. In addition, for more effective knowledge transfer, we develop an adaptive strategy to choose the most suitable assisted task for the target task based on the relatedness information between tasks. The findings show that in all of the multitask scenarios studied, the proposed algorithm can substantially increase the effectiveness of the learned scheduling heuristics for all the desired tasks. The effectiveness of the proposed algorithm has also been verified by the analyses of task relatedness and structures of the evolved scheduling heuristics, and the discussions of population diversity and knowledge transfer.

Comparison study of dispatching rules and heuristics for online scheduling of single machine scheduling problem with predicted release time jobs

In real-life dynamic scheduling systems, the release times of some important and/or urgent jobs are usually known deterministically or predictably in advance. When making dynamic scheduling decisions, we are expected to get better scheduling solutions if these jobs are considered before they actually arrive. However, to the best of our knowledge, we have not yet found corresponding research to date. This paper addresses a dynamic single machine scheduling problem with predicted arriving jobs (DSMSP/PRJs). We give a detailed description and the corresponding mathematical programming model with the objective of minimizing total weighted tardiness (TWT). With respect to the inclusion strategy of PRJs, which is one of the key technologies affecting the solution quality, we propose a job-based inclusion (JBI) procedure that focuses on the influence on the tardiness of PRJs rather than on a certain length of lookahead window. The novel JBI procedure shows obvious superior performance to the time-based method in the literature with respect to TWT. The discrete event simulation based on dispatching rules/heuristics (RHs) is adopted to address the DSMSP/PRJs. Six weighted dispatching rules and two improved heuristics based on the DT-TD (Decision Theory-Tactically Delayed) are proposed. Large-scale and systematic experiments show that the proposed RHs perform well. Especially, the DT-TD2 heuristic has overwhelming superior performance to the others. For the huge volumes of experimental raw data produced by 1350 simulation runs under 135 problem scenarios, we innovatively put forward the overall procedure and methods of data processing and analysis oriented to the different purposes. The systematic data analyses show that, under the various problem scenarios, considering PRJs in advance can significantly improve the scheduling solutions as long as appropriate RHs are used. The analysis results also reveal that, in the scenarios with lower shop utilization, lower percentage of PRJ, higher weight of PRJ, and looser due date of normal job (NJ), it is more obvious in the improvement of solutions, implying that it is more necessary to consider PRJs in advance.

Deep reinforcement learning for solving steelmaking-continuous casting scheduling problems under time-of-use tariffs

This paper proposes a novel intelligent scheduling method based on deep reinforcement learning (DRL) to solve the multi-objective steelmaking-continuous casting (SCC) scheduling problem, under time-of-use (TOU) tariffs for the first time. The intelligent scheduling system architecture is designed, and a mathematical model is established to minimise the total sojourn time and electricity cost. To effectively reduce production costs by avoiding peak periods of electricity consumption, the ‘start time’ of the system is generated based on the Markov Decision Process (MDP), and heuristic scheduling rules related to power cost are used as the action space, with corresponding reward functions designed according to the characteristics of these two objectives. To satisfy the continuous casting which is a particular SCC constraint, a backward strategy is developed. Additionally, a branching duelling double deep Q-network (BD3QN) is adapted to guide action selection and avoid blind search in the iteration process, and then applied to real-time scheduling. Numerical experiments demonstrate that the proposed method outperforms comparison algorithms in terms of solution quality and CPU times by a large margin.