Job Completion Time Research Articles

Optimizing makespan and resource utilization for multi-DNN training in GPU cluster

Deep neural network (DNN) has been widely applied in many fields of artificial intelligence (AI), gaining great popularity both in industry and academia. Increasing the size of DNN models does dramatically improve the learning accuracy. However, training large-scale DNN models on a single GPU takes unacceptable waiting time. In order to speed up the training process, many distributed deep learning (DL) systems and frameworks have been published and designed for parallel DNN training with multiple GPUs. However, most of the existing studies concentrate only on improving the training speed of a single DNN model under centralized or decentralized systems with synchronous or asynchronous approaches. Few works consider the issue of multi-DNN training on the GPU cluster, which is the joint optimization problem of job scheduling and resource allocation. This paper proposes an optimizing makespan and resource utilization (OMRU) approach to minimize job completion time and improve resource utilization for multi-DNN training in a GPU cluster. Specifically, we first collect the training speed/time data of all DNN models by running a job for one epoch on a different number of GPUs. The OMRU algorithm, integrating job scheduling, resource allocation, and GPU reuse strategies, is then devised to minimize the total job completion time (also called makespan) and improve GPU cluster resource utilization. The linear scaling rule (LSR) is adopted for adjusting the learning rate when a DNN model is trained on multiple GPUs with large minibatch size, which can guarantee model accuracy without the other hyper-parameters tune-up. We implement the OMRU algorithm on the Pytorch with Ring-Allreduce communication architecture and a GPU cluster with 8 nodes, each of which has 4 NVIDIA V100 GPUs. Experimental results on image classification and action recognition show that OMRU achieves a makespan reduction of up to 30% compared to the baseline scheduling algorithms and reach an average of 98.4% and 99.2% resource utilization on image classification and action recognition, respectively, with the state-of-the-art model accuracy.

Integrated production planning and preventive maintenance scheduling for synchronized parallel machines

Production synchronization is an emerging paradigm in advanced manufacturing systems, with which the inventory space and cost of finished products can be significantly reduced. This paradigm has been studied in the past years, and several mathematical models have been developed to evaluate production synchronization for a diversity of manufacturing systems. Most of the existing studies on production synchronization, however, assume that machines are always functioning without malfunctions in production process. It is no doubt that machine failures can definitely lead to unexpected breakdowns that may produce non-ignorable impact on production synchronization. In this study, a new evaluation model for production synchronization is put forth for parallel machines. The production time uncertainty caused by unexpected breakdowns of machines is explicitly quantified, and the distribution of the job completion time on a machine is approximated by the Gamma distribution through the moment match method. By integrating production planning and preventive maintenance scheduling together, a new joint optimization problem is formulated for synchronized parallel machines, and it is resolved by a modified cooperative co-evolutionary genetic algorithm with a heuristic initialization. Two illustrative examples along with a set of comparative studies are presented to examine the effectiveness of the proposed model and solution algorithm.

Probability distribution based resource management for multitenant cloud clusters

AbstractAccurate resource allocation and colocating jobs are effective ways to increase resource utilization and reduce costs in modern datacenters. The main challenges are fluctuations in resource consumption and interference among colocated jobs. Therefore, we propose a resource management schema based on the probability distributions of resource consumption and completion time for multitenant cloud clusters. First, we found that the characteristics of the task can be well described by the probability distribution of resource consumption and completion time, and the probability distribution function can be obtained by Gaussian fitting. Second, we propose a probability distribution based resource allocation (PDRA) strategy for batch jobs. Third, we design a tail latency aware allocation (TLAA) strategy to use transient resources efficiently while ensuring tail latency requirements. Finally, we design a cost‐effective resource revocation (CERR) strategy to revoke transient resources with minimal eviction costs. Experimental results demonstrate the efficiency of our resource management. Our resource allocation strategies (PDRA and TLAA) can effectively improve resource utilization and reduce job completion time. CERR can reduce the impact of resource revocation on batch jobs and perform better than existing resource revocation strategies.

Combinatory Least Slack and Kuhn Tucker Optimization for Multiobjective Flexible Job Shop Scheduling

The Multi-objective Flexible Job Shop Scheduling (MFJSS) problem, in which the order or processing sequence for a set of jobs through several machines is assigned in an optimal patterns have accepted sizeable attentiveness. Different types of scheduling rules and procedures for certain types of MFJSS have made progress from these endeavors. The scheduling problem is cumbersome to be made consistent due to different types of criteria involved. This is because in MFJSS, after the completion of a job on a machine, it may be transited between different machines and transit time may affect scheduling. However, the transit times are frequently ignored in the literature. In this paper, the transit time and the processing time are considered as the independent time into the MFJSS. In this paper, we propose Combinatory Least Slack and Kuhn Tucker Optimization (CLS-KTO) for the MFJSS problem with the objectives to minimize the mean tardiness, makespan and job completion time. The problem is addressed via assignment and scheduling model. First, by using well-designed processing and transit time and least slack variable, Combinatory Least Slack (CLS) algorithm is adapted for the CLS-KTO. Then, an Additive function is formulated by incorporating Combinatory Dispatching Rules into the adapted CLS, where some good machines are assigned for the corresponding job, therefore reducing mean tardiness. Furthermore, in the proposed sequencing part, an optimization model based on the Kuhn Tucker conditions and Lagrange Multiplier is adopted to handle the three objectives. In the experimental studies, the influence of mean tardiness on the performance of the proposed CLS-KTO is first examined. Afterwards, the effectiveness of makespan and job completion time in CLS-KTO is verified. Finally, extensive comparisons are carried out with the state-of-the-art methods for the MFJSSon benchmark OR instances. The results show that CLS-KTO performs better than other algorithms.

Makespan minimization with OR-precedence constraints

We consider a variant of the NP-hard problem of assigning jobs to machines to minimize the completion time of the last job. Usually, precedence constraints are given by a partial order on the set of jobs, and each job requires all its predecessors to be completed before it can start. In this paper, we consider a different type of precedence relation that has not been discussed as extensively and is called OR-precedence. In order for a job to start, we require that at least one of its predecessors is completed—in contrast to all its predecessors. Additionally, we assume that each job has a release date before which it must not start. We prove that a simple List Scheduling algorithm due to Graham (Bell Syst Tech J 45(9):1563–1581, 1966) has an approximation guarantee of 2 and show that obtaining an approximation factor of 4/3 - varepsilon is NP-hard. Further, we present a polynomial-time algorithm that solves the problem to optimality if preemptions are allowed. The latter result is in contrast to classical precedence constraints where the preemptive variant is already NP-hard. Our algorithm generalizes previous results for unit processing time jobs subject to OR-precedence constraints, but without release dates. The running time of our algorithm is O(n^2) for arbitrary processing times and it can be reduced to O(n) for unit processing times, where n is the number of jobs. The performance guarantees presented here match the best-known ones for special cases where classical precedence constraints and OR-precedence constraints coincide.

Short‐term scheduling with machine calibration

AbstractIn most of the past research on scheduling, it is conventional to model machine deterioration by decreasing production speed over time. However, as is observed, there is a fair number of scenarios in practice where deterioration has a little impact on machine speed but a great impact on production accuracy and a machine is unusable unless it has been calibrated recently. In this paper, we consider a single‐machine short‐term scheduling issue in distributed production systems where the engineer calibrates the machines in a round‐robin manner. The objective is to minimize total job completion time, which is a popular criterion indicating total holding or inventory cost. We consider two cases of the problem and show that one of them can be solved optimally by providing an extended shortest processing time first rule. As for the other case, which is intractable, we propose a polynomial‐time approximation scheme based on the extended rule and an exchange procedure. Computational experiments show that the proposed polynomial‐time approximation scheme is very promising even if limited number of exchangeable jobs is allowed.

Interference-aware execution framework with Co-scheML on GPU clusters

Recently, improving the overall resource utilization through efficient scheduling of applications on graphic processing unit (GPU) clusters has been a concern. Traditional cluster-orchestration platforms providing GPUs exclusively for applications constrain high resource utilization. Co-execution of GPU applications is suggested to utilize limited resources. However, the co-execution of GPU applications without considering their diverse characteristics can lead to their unpredictable performances owing to interference resulting from contention and unbalanced usage of resources among applications. This paper proposes an interference-aware execution framework with Co-scheML for various GPU applications such as high performance computing (HPC), deep learning (DL) training, and DL inference. Various resource-usage characteristics of GPU applications are analyzed and profiled to identify various degrees of their application interference. As interference prediction is challenging owing to the complexity of GPU systems, an interference model is generated by applying defined GPU metrics to machine learning (ML) models. A Co-scheML scheduler deploys applications to minimize the interference using the predicted interference from the constructed model. Experimental results of our framework demonstrated that the resource utilization improved by 24%, the average job completion time (JCT) improved by 23%, and the makespan shortened by 22% on average, compared to baseline schedulers.

Enhancing Balanced Graph Edge Partition with Effective Local Search

Graph partition is a key component to achieve workload balance and reduce job completion time in parallel graph processing systems. Among the various partition strategies, edge partition has demonstrated more promising performance in power-law graphs than vertex partition and thereby has been more widely adopted as the default partition strategy by existing graph systems. The graph edge partition problem, which is to split the edge set into multiple balanced parts with the objective of minimizing the total number of copied vertices, has been widely studied from the view of optimization and algorithms. In this paper, we study local search algorithms for this problem to further improve the partition results from existing methods. More specifically, we propose two novel concepts, namely adjustable edges and blocks. Based on these, we develop a greedy heuristic as well as an improved search algorithm utilizing the property of max-flow model. To evaluate the performance of our algorithms, we first provide adequate theoretical analysis in terms of approximation quality. We significantly improve the previous known approximation ratio for this problem. Then we conduct extensive experiments on a large number of benchmark datasets and state-of-the-art edge partition strategies. The results show that our proposed local search framework can further improve the quality of graph partition by a wide margin.

Minimizing total job completion time in MapReduce scheduling

We follow up an earlier studied multiple-task parallel-machine scheduling model that captures the core challenges in MapReduce scheduling, with the optimization goal to minimize the total job completion time. The problem is a novel generalization of the classic two-stage flow-shop scheduling, in which we are given a set of jobs each is associated with multiple map tasks and multiple reduce tasks. All these tasks are non-preemptive to be processed on multiple parallel identical map machines and multiple parallel identical reduce machines, respectively, under the strict precedence constraints that, for each job, any reduce task cannot start before all its map tasks have been finished. We prove a new lower bound on the total job completion time, and based on which we present an O(nlogn+N+m1+m2)-time (9-3m1)-approximation algorithm, where n and N are the number of jobs and the total number of tasks, respectively, m1 and m2 are the numbers of map and reduce machines, respectively, and they can even be part of the input. Our algorithm improves the previous best 12-approximation, and it reduces to the best approximation algorithms for several interesting or well studied special cases. We confirm through numerical experiments on 828,100 instances that both our lower bound and our algorithm significantly outperform: the empirical mean improvement ratio of the new lower bound is as high as 37.75%, and the empirical mean approximation ratio of our algorithm is only 1.7582.

A scatter search algorithm with a novel solution representation for flexible open shop scheduling: a multi-objective optimization

The flexible open shop scheduling problem is the combination of two classical open shop and a parallel machine scheduling problems. Due to the computational complexity of solving the open shop scheduling problem, so far, limited research has considered the assumption of flexibility in this problem. In this study, a bi-objective flexible open shop scheduling with independent setup time is considered. The considered objective functions are to minimize the maximum completion time of jobs and total tardiness. Using the first objective function, an attempt is made to balance the amount of workloads on the machines so that the completion time of the last job is minimized, while using the second one, from the customers' point of view, an attempt is made to reduce the amount of tardiness in delivery of jobs to the customers. Here, firstly, a mixed-integer nonlinear programming model is proposed. Then, to deal with the multi-objective problem, the weighted Lp-metric method is used. Since, this problem is a non-deterministic polynomial-time hard problem (NP-hard), a scatter search algorithm is developed to obtain near-optimal solutions in a reasonable time. In this regard, for the first time, a novel solution representation method is also presented. To ensure the efficiency of the proposed algorithm, the results were compared with NSGA-II using several instances. After analyzing the results using four evaluation metrics, it is observed that the scatter search algorithm performs better than the NSGA-II.

P104 Orion xTask: Improving neurosurgical workflow using an electronic tasks system

IntroductionThe dominant form of communication within hospitals remains the antiquated paging system. Pitfalls to pagers include: overwhelming volume of non-urgent bleeps; inability to prioritise at the point-of-bleep; no written record of communication - potentially compromising patient safety.Orion xTask, an electronic tasks system, was implemented at Cambridge neurosurgery in April 2019 to mitigate some of these pitfalls.MethodsBleep logs, covering 2-months before and 4-months after xTask implementation with total 3181 bleeps, were obtained. xTask data for the same period (total 4195 electronic requests) was exported. Bleep response times for neurosurgery and for other specialties without xTask were collected. Questionnaires were distributed to nursing and medical staff.ResultsDaily averages of 14.4 in-hours and 23.7-out-of-hours electronic requests were made. The exception to greater out-of-hours utilisation was the ward without a dedicated doctor, where requests peaked in the morning. Prescription was most commonly requested (49%), followed by clinical review (17%).There were daily averages of 13.0 in-hours and 8.3 out-of-hours bleeps. Changes in bleep volume with xTask implementation were inconsistent: Calls to ward bleeps decreased (p = 0.044), but increased to the on-call (p = 0.042).Bleep response times, and (bleeped-)job completion times were significantly quicker for neurosurgery than for other specialties (p < 0.001).Nursing and doctors’ feedback was overwhelmingly positive, with agreement that xTask improves patient safety. xTask scored particularly highly on “Improves Communication”, “Improves Record Keeping” and “Useful Out-of-Hours”. Doctors indicated that xTask also “Helps Prioritisation” and “Facilitates Educational Opportunities”.ConclusionThe feedback and pattern-of-use indicates that Orion xTask successfully mitigates some of the pitfalls of pagers, thereby improving patient safety.

Stargazer: Toward efficient data analytics scheduling via task completion time inference

The fundamental challenge of data analytics scheduling is the heterogeneity of both data analytics jobs and resources. Although many scheduling solutions have been developed to improve the efficiency of data analytics frameworks (e.g., Spark), they either (1) focus on the scheduling of a single type of resource, without considering the coordination between different resources; or (2) schedule multiple resources by factoring in limited information about analytics jobs without considering the heterogeneity of resources. This paper presents Stargazer, a novel, efficient system that tackles diversity data analytics jobs on heterogeneous cluster by inferring the completion times of their decomposed tasks. Specifically, Stargazer adopts a deep learning model, which takes into considerations multiple key factors of diversity data analytics jobs and heterogeneous resources, to accurately infer the completion time of different tasks. A prototype of Stargazer is fully implemented in the Spark framework. Extensive experiments show that Stargazer can reduce the average job completion time by 21% and improve average performance by 20%, while incurring little overhead.

IMSM: An Interval Migration Based Approach for Skew Mitigation in MapReduce

Background: Extreme growth of data necessitates the need for high-performance computing. MapReduce is among the most sought-after platform for processing large-scale data. Research work and analysis of the existing system has revealed its performance bottlenecks and areas of concern. MapReduce has the problem of skew on its processing nodes. This paper proposes an algorithm for MapReduce to balance the load and eliminate the skew on Map tasks. It reduces the execution time of job by lowering the completion time of the slowest task. Methods: The proposed method performs one-time settlement of load balancing among the Map tasks by analyzing the expected completion time of the Map tasks and redistributes the load. It uses intervals to migrate the overloaded or slows tasks and append them on the under loaded tasks. Results: Experiments revealed an improvement of up to 1.3x by implementing the proposed strategy. Comparison of the proposed technique with other relevant strategies exhibits a better distribution of load among Map tasks and lower level of the skew. Evaluation is done using different workloads. Conclusion: A significant improvement is observed in the performance and reduced completion time of job.

FPGA Resource Pooling in Cloud Computing

Cloud providers have started to deploy various FPGA accelerators in their datacenters because the performance of many applications can be significantly improved by implementing their core routines in FPGAs. In conventional datacenters with FPGA accelerated servers, if a tenant wants to use FPGA accelerators, it requests for a VM instance residing in a server equipped with an FPGA accelerator. This paradigm to integrate FPGA into Cloud leads to poor resource sharing of the precious FPGA resources. In this paper, we propose FPGAPooling, an FPAG-enabled Cloud system where all FPGA accelerators are managed as a single resource pool and shared among all VMs. For a VM, instead of requesting the Cloud to run the VM on an FPGA accelerated server, at run time, when a VM needs to use FPGA acceleration, it requests an FPGA accelerator from the pool. After the VM finishes using the FPGA accelerator, it releases the FPGA accelerator back to the pool. We design a centralized scheduler to handle acceleration requests from VMs and assign each request to an idle FPGA accelerator at run time; We implemented a system prototype on IBM's OpenPower Cloud system. The key challenging of FPGAPooling is scheduling. We designed and implemented a group of scheduling algorithms for the FPGAPooling system. With extensive evaluations on both a small testbed and a large-scale simulation, we found that our algorithms can improve the average and tail job completion time by up to 7 and 4 times, respectively.

HeSRPT

Modern data centers serve workloads which can exploit parallelism. When a job parallelizes across multiple servers it completes more quickly. However, it is unclear how to share a limited number of servers between many parallelizable jobs. In this paper we consider a typical scenario where a data center composed of N servers will be tasked with completing a set of M parallelizable jobs. Typically, M is much smaller than N. In our scenario, each job consists of some amount of inherent work which we refer to as a job's size. We assume that job sizes are known up front to the system, and each job can utilize any number of servers at any moment in time. These assumptions are reasonable for many parallelizable workloads such as training neural networks using TensorFlow [2]. Our goal in this paper is to allocate servers to jobs so as to minimize the mean slowdown across all jobs, where the slowdown of a job is the job's completion time divided by its running time if given exclusive access to all N servers. Slowdown measures how a job was interfered with by other jobs in the system, and is often the metric of interest in the theoretical parallel scheduling literature (where it is also called stretch), as well as the HPC community (where it is called expansion factor).

Coflow Scheduling in Data Centers: Routing and Bandwidth Allocation

In distributed computing frameworks like MapReduce, Spark, and Dyrad, a coflow is a set of flows transferring data between two stages of a job. The job cannot start its next stage unless all flows in the coflow finish. To improve the execution performance of such a job, it is crucial to reduce the completion time of a coflow, as it can contribute more than 50 percent of the job completion time. While several coflow schedulers have been proposed, we observe that routing, as a factor greatly impacting the Coflow Completion Time (CCT), has not been well considered. In this article, we focus on the coflow scheduling problem and jointly consider routing and bandwidth allocation. We begin by providing an analytical solution to the problem of optimal bandwidth allocation with pre-determined routes. In the following, we formulate the problem of scheduling a single coflow as a Non-linear Mixed Integer Programming problem and present its relaxed convex optimization problem. We further propose two algorithms, CoRBA and its simplified version: CoRBA-fast that solve the single coflow scheduling problem with a joint consideration of routing and bandwidth allocation. Lastly, to address multiple coflows in online scheduling, we propose an online scheduler named OnCoRBA. By comparing with the start-of-the-art algorithms and schedulers via simulations, we demonstrate that CoRBA and CoRBA-fast reduce the CCT by 30-400 percent and the OnCoRBA scheduler reduces the average online CCT by 20-230 percent. In addition, CoRBA-fast can be hundreds times faster than CoRBA with around 8 percent performance degradation compared to CoRBA, which makes the use of CoRBA-fast very appropriate in practice.

Deadline-Aware Cost Optimization for Spark

We present OptEx, a closed-form model of job execution on Apache Spark, a popular parallel processing engine. To the best of our knowledge, OptEx is the first work that analytically models job completion time on Spark. The model can be used to estimate the completion time of a given Spark job on a cloud, with respect to the size of the input dataset, the number of iterations, and the number of nodes comprising the underlying cluster. Experimental results demonstrate that OptEx yields a mean relative error of 6 percent in estimating the job completion time. Furthermore, the model can be applied for estimating the cost-optimal cluster composition for running a given Spark job on a cloud under a completion deadline specified in the SLO (i.e., Service Level Objective). We show experimentally that OptEx is able to correctly estimate the required cluster composition for running a given Spark job under a given SLO deadline with an accuracy of 98 percent. We also provide a tool which can classify Spark jobs into job categories based on bisimilarity analysis on lineage graphs collected from the given jobs.

Makespan Minimization in Job Shop Scheduling

In industries, the completion time of job problems in the manufacturing unit has risen significantly. In several types of current study, the job's completion time, or makespan, is reduced by taking straight paths, which is time-consuming. In this paper, we used an Improved Ant Colony Optimization and Tabu Search (ACOTS) algorithm to solve this problem by precisely defining the fault occurrence location in order to rollback. We have used a short-term memory-based rollback recovery strategy to minimise the job's completion time by rolling back to its own short-term memory. The recent movements in Tabu quest are visited using short term memory. As compared to the ACO algorithm, our proposed ACOTS-Cmax solution is more efficient and takes less time to complete.

Optimizing Resource Allocation for Data-Parallel Jobs Via GCN-Based Prediction

Under-allocating or over-allocating computation resources (e.g., CPU cores) can prolong the completion time of data-parallel jobs in a distributed system. We present a predictor, ReLocag, to find the near-optimal number of CPU cores to minimize job completion time (JCT). ReLocag includes a graph convolutional network (GCN) and a fully-connected network (FCNN). The GCN learns the dependency between operations from the workflow of a job, and then the FCNN takes the workflow dependency together with other features (e.g., the input size, the number of CPU cores, the amount of memory, and the number of computation tasks) as input for JCT prediction. The prediction result can guide the user to determine the near-optimal number of CPU cores. Besides, we propose two effective strategies to overcome the time-consuming issue of training sample collection in big data applications. First, we develop an adaptive sampling method to collect essential samples judiciously. Second, we further design a cross-application transfer learning model to exploit the training samples collected from other applications. We conduct extensive experiments in a Spark cluster for 7 types of exemplary Spark applications. Results show that ReLocag improves the JCT prediction accuracy by 4-14 percent. Moreover, the CPU core consumption decreases by 58.2 percent.

Parallel-Machine Scheduling with DeJong’s Learning Effect, Delivery Times, Rate-Modifying Activity, and Resource Allocation

We investigate parallel-machine scheduling with past-sequence-dependent (p-s-d) delivery times, DeJong’s learning effect, rate-modifying activity, and resource allocation. Each machine has a rate-modifying activity. We consider two versions of the problem to minimize the sum of the total completion times, the total absolute deviation of job completion times, and the total resource allocation and the sum of the total waiting times, the total absolute deviation of job waiting times, and the total resource allocation, respectively. The problems under our present model can be solved in polynomial time.