Job Sizes Research Articles

Simple Near-Optimal Scheduling for the M/G/1

We consider the problem of preemptively scheduling jobs to minimize mean response time of an M/G/1 queue. When we know each job's size, the shortest remaining processing time (SRPT) policy is optimal. Unfortunately, in many settings we do not have access to each job's size. Instead, we know only the job size distribution. In this setting the Gittins policy is known to minimize mean response time, but its complex priority structure can be computationally intractable. A much simpler alternative to Gittins is the shortest expected remaining processing time (SERPT) policy. While SERPT is a natural extension of SRPT to unknown job sizes, it is unknown whether or not SERPT is close to optimal for mean response time.

Characterizing Policies with Optimal Response Time Tails under Heavy-Tailed Job Sizes

We consider the tail behavior of the response time distribution in anM/G/1 queue with heavy-tailed job sizes, specifically those with intermediately regularly varying tails. In this setting, the response time tail of many individual policies has been characterized, and it is known that policies such as Shortest Remaining Processing Time (SRPT) and Foreground-Background (FB) have response time tails of the same order as the job size tail, and thus such policies are tail-optimal. Our goal in this work is to move beyond individual policies and characterize the set of policies that are tail-optimal. Toward that end, we use the recently introduced SOAP framework to derive sufficient conditions on the form of prioritization used by a scheduling policy that ensure the policy is tail-optimal. These conditions are general and lead to new results for important policies that have previously resisted analysis, including the Gittins policy, which minimizes mean response time among policies that do not have access to job size information. As a by-product of our analysis, we derive a general upper bound for fractional moments of M/G/1 busy periods, which is of independent interest.

Dynamic Computation of Threshold Value for Classifying Jobs in Cloud Computing for Efficient Resource Utilization

Cloud computing is an instant use of resources and it is a trending technology in the field of computer science. Here, many jobs will be arriving continuously with different job size, at that point of time, allocating of resources for suitable virtual machines without allowing virtual machine to starving is a hindrance job. So, to avoid this hindrance, an algorithm Dynamic Computation of Threshold Value is proposed (DCTV) and based on the threshold value the jobs are classified in the initial stage, so this classification leads to allocation of resources precisely and efficient resource utilization. The experimental result shows that by using dynamic computation of threshold value the allocation of resource time is reduced and classification accuracy is improved compared to manual computation of threshold value.

Understanding Well Interference and Parent-Child Well Relationships With Liquid Molecular Chemical Tracer Technology

In an era where capital markets are hitting the brakes on funding the US shale sector, operators have increasingly pivoted from production growth to maximizing the rates of return via lower-cost wells. One of the major challenges of this new era is the determination of optimal stage and well spacing for a drilling area. For much of the US, the trend has been toward increased job size and rapid downspacing of infill or “child” wells. The unintended consequence of this trend has been an increase in fracture interference and excessive cross-well communication, which results in an overcapitalization of acreage and underperforming child wells as the drainage areas of wells overlap and compete for depleted resources. Within the SCOOP/STACK play, child wells completed in 2017/2018 have been half as productive as their 2015/2016 parent wells, a trend theorized to be directly related to negative fracture interactions. Determining an optimal spacing between parent and infill wells is not a straightforward endeavor. An ideal development strategy is not just about well spacing, as decisions in completion design can exacerbate issues stemming from tight spacing. For this multivariate problem, operators have attempted to use bottomhole pressure measurements to extrapolate fracture interference or to use large data sets in a “trial-and-error” approach to understand well interference behavior. These approaches have significant downsides, relying either on data sets of questionable value or on extrapolating data to and from wells with significantly different geological conditions. One operator in the SCOOP/STACK has attempted to understand well interference with chemical tracer technology. Using liquid molecular tracers, this operator achieved a quantified understanding of flow between parent/child wells with a high degree of confidence. This inexpensive technology has allowed the operator to experiment with well spacing and repressurization of parent wells. This, in turn, permits the rapid evaluation of mitigation techniques to optimize its field development strategy based on an accurate measurement of interwell fluid movement. Tracer Selection Two categories of chemical tracer are commonly used in conjunction with hydraulic fracturing operations: infused solid particulate tracers and liquid molecular tracers. While both tracer products have their strengths and weaknesses, liquid molecular tracers were chosen as a best practice in this study as they allow for quantification of the traced fluid phase, unlike infused solid particulate tracers.

Analysis of the Task Assignment based on Guessing Size policy

We study the Task Assignment based on Guessing Size (TAGS) policy in a parallel and homogeneous server system. The policy parameters are the number of servers h and a set of cutoffs s1<s2,ldots,sh−1. In this policy, all the incoming jobs are routed to the first server and jobs are run up to s1 time units. If they complete they leave the system, but jobs that do not complete after s1 time units are killed and moved to the end of the queue of the second server, where service starts from scratch. Likewise, jobs that are executed in server i and complete service before si units of time, leave the system, whereas jobs that do not complete are killed and routed to the next server. We first study the stability of such system and provide a precise utilization threshold for the existence of stable parameters for a given job size distribution. We compute the threshold for several families of distributions and provide bounds for others. We show that TAGS is most stable for bounded Pareto distributions with parameter α=1. Besides, we provide tight bounds on the performance of the TAGS policy where the cutoffs are chosen to minimize average waiting time in the asymptotic regime where the largest job size tends to infinity for the Bounded Pareto distribution and the system load smaller than one. In this case, we show that the performance ratio between TAGS and a version called SITA which does require knowledge of job size is at most 2. We then consider more broadly the same asymptotic regime and consider a bound on the average waiting time for any distribution. This is compatible with having a conservative policy which will work well for any job size distribution. We show rather tight upper and lower bounds which again match those of SITA up to a factor of 2. These bounds are considerably lower than the corresponding bounds of competing policies such as Random or Least Work Remaining. We also show that for all these policies the bounded Pareto distribution with α=1 is close to being the worst possible among all distributions with the same job size range. We show using the stability results that in the same regime, if we increase the load the gap between SITA and TAGS grows dramatically and eventually, the TAGS system cannot be stabilized. The conclusion from all our analysis is that TAGS is best used as a conservative policy with minimal requirements on small systems with relatively low utilization and where the range of job sizes is large.

Problem on the integration between production and delivery with parallel batching machines of generic job sizes and processing times

In recent years the manufacturing industries are currently focused on adopting optimal hybrid mass/customized strategies. Working solely on customized production or mass production is not a desirable strategy for many manufacturers anymore. The rapid advances in technology, applications, and customers’ requirements are increasing the competitiveness and, therefore, productivity. Given this new scenario, the integration between production and delivery with parallel batching machines comes as an essential issue in logistics. Efficient integrated production and delivery operation is a critical factor for minimizing costs and increasing customer satisfaction. The particular case discussed in this paper consists of scheduling jobs of generic sizes and processing times on identical and parallel batching machines, aiming to maximize profits. The objective is to get a joint production and delivery schedule that maximizes total profits, focusing on attending consumers on time. We propose a mathematical formulation and show that the addition of a set of constraints improves the commercial solver performance. Additionally, we focus on the development and analys is of two fast polynomial methods. The first, RKP, aims to obtain relaxed solutions, while the last, CMFFDA, aims to obtain feasible solutions. Furthermore, computational experiments and sensitivity analysis with random instances of different sizes are conducted to evaluate the proposed methods and the integrated system. The results show tight gaps for the mathematical formulation and algorithms RKP and CMFFDA. The proposed algorithms present short computational times for small, moderate, and large size instances, an essential quality for daily operation.

Characterizing Policies with Optimal Response Time Tails under Heavy-Tailed Job Sizes

We consider the tail behavior of the response time distribution in an M/G/1 queue with heavy-tailed job sizes, specifically those with intermediately regularly varying tails. In this setting, the response time tail of many individual policies has been characterized, and it is known that policies such as Shortest Remaining Processing Time (SRPT) and Foreground-Background (FB) have response time tails of the same order as the job size tail, and thus such policies are tail-optimal. Our goal in this work is to move beyond individual policies and characterize the set of policies that are tail-optimal. Toward that end, we use the recently introduced SOAP framework to derive sufficient conditions on the form of prioritization used by a scheduling policy that ensure the policy is tail-optimal. These conditions are general and lead to new results for important policies that have previously resisted analysis, including the Gittins policy, which minimizes mean response time among policies that do not have access to job size information. As a by-product of our analysis, we derive a general upper bound for fractional moments of M/G/1 busy periods, which is of independent interest.

Energy-Efficient Job-Assignment Policy With Asymptotically Guaranteed Performance Deviation

We study a job-assignment problem in a large-scale server farm system with geographically deployed servers as abstracted computer components (e.g., storage, network links, and processors) that are potentially diverse. We aim to maximize the energy efficiency of the entire system by effectively controlling carried load on networked servers. A scalable, near-optimal job-assignment policy is proposed. The optimality is gauged as, roughly speaking, energy cost per job. Our key result is an upper bound on the deviation between the proposed policy and the asymptotically optimal energy efficiency, when job sizes are exponentially distributed and blocking probabilities are positive. Relying on Whittle relaxation and the asymptotic optimality theorem of Weber and Weiss, this bound is shown to decrease exponentially as the number of servers and the arrival rates of jobs increase arbitrarily and in proportion. In consequence, the proposed policy is asymptotically optimal and, more importantly, approaches asymptotic optimality quickly (exponentially). This suggests that the proposed policy is close to optimal even for relatively small systems (and indeed any larger systems), and this is consistent with the results of our simulations. Simulations indicate that the policy is effective, and robust to variations in job-size distributions.

Minimizing Mean Response Time in Nonobservable Distributed Processing Systems with Nodes Operating under Egalitarian Processor-Sharing Policy

Abstract—Consideration is given to the dispatching problem in an almost nonobservable distributed processing system with M, M ≥ 2, single-server queues operating in parallel, each under the processor-sharing discipline. Jobs having the same job size distribution arrive one-by-one to the dispatcher, which immediately routes it to one of the queues. When making a routing decision, the dispatcher has no online information about the system (like current queues sizes, size of arriving job, etc.). The only information available to the dispatcher is job-size distribution, job-interarrival-time distribution, server speeds, time instants of previously arrived jobs, and previous routing decisions. Under these conditions, one is interested in the routing policies that minimize the job long-run mean response time. A new class of dispatching policies is proposed that, according to the numerical experiments, may significantly outperform all classical dispatching policies available for such system: (optimal) probabilistic policy and the round-robin policy.

Simple Near-Optimal Scheduling for the M/G/1

We consider the problem of preemptively scheduling jobs to minimize mean response time of an M/G/1 queue. When we know each job's size, the shortest remaining processing time (SRPT) policy is optimal. Unfortunately, in many settings we do not have access to each job's size. Instead, we know only the job size distribution. In this setting the Gittins policy is known to minimize mean response time, but its complex priority structure can be computationally intractable. A much simpler alternative to Gittins is the shortest expected remaining processing time (SERPT) policy. While SERPT is a natural extension of SRPT to unknown job sizes, it is unknown whether or not SERPT is close to optimal for mean response time. We present a new variant of SERPT called monotonic SERPT (M-SERPT) which is as simple as SERPT but has provably near-optimal mean response time at all loads for any job size distribution. Specifically, we prove the mean response time ratio between M-SERPT and Gittins is at most 3 for load ρ łeq 8/9$ and at most 5 for any load. This makes M-SERPT the only non-Gittins scheduling policy known to have a constant-factor approximation ratio for mean response time.

A Multi-Optimization Technique for Improvement of Hadoop Performance with a Dynamic Job Execution Method Based on Artificial Neural Network

The improvement of Hadoop performance has received considerable attention from researchers in cloud computing fields. Most studies have focused on improving the performance of a Hadoop cluster. Notably, various parameters are required to configure Hadoop and must be adjusted to improve performance. This paper proposes a mechanism to improve Hadoop, schedule jobs, and allocate and utilize resources. Specifically, we present an improved ant colony optimization method to schedule jobs according to the job size and the time expected for execution. Priority is given to the job with the minimum data size and minimum response time. The resource usage and running jobs by data node are predicted using an artificial neural network, and job activity and resource usage are monitored using the resource manager. Moreover, we enhance the Hadoop Name node performance by adding an aggregator node to the default HDFS framework architecture. The changes involve four entities: the name node, secondary name node, aggregator nodes, and data nodes, where the aggregator node is responsible for assigning the jobs among the data node, and the Name node keeps tracking only the aggregator nodes. We test the overall scheme among Amazon EC2 and S3, and show the results of throughput and CPU response time for different data sizes. Finally, we show that the proposed approach shows significant improvement compare to native Hadoop and other approaches.

Feasibility of on-line speed policies in real-time systems

We consider a real-time system where a single processor with variable speed executes an infinite sequence of sporadic and independent jobs. We assume that job sizes and relative deadlines are bounded by C and ∆ respectively. Furthermore, S max denotes the maximal speed of the processor. In such a real-time system, a speed selection policy dynamically chooses (i.e., on-line) he speed of the processor to execute the current, not yet finished, jobs. We saythat an on-line speed policy is feasible if it is able to execute any sequence ofjobs while meeting two constraints: the processor speed is always belowSmaxand no job misses its deadline. In this paper, we compare the feasibility regionof four on-line speed selection policies in single-processor real-time systems,namely Optimal Available (OA) (Yao, Demers, and Shenker, 1995), Aver-age Rate (AVR) (Yao, Demers, and Shenker, 1995), (BKP) (Bansal, Kimbrel,and Pruhs, 2007), and a Markovian Policy based on dynamic programming(MP) (Gaujal, Girault, and Plassart, 2017). We prove the following results: – (OA) is feasible if and only if Smax ≥C(h∆−1+ 1), where hn is the n-th harmonic number (hn=∑ni=11/i≈logn). – (AVR) is feasible if and only if Smax ≥Ch∆. – (BKP) is feasible if and only if Smax≥ eC (where e= exp(1)). – (MP) is feasible if and only if Smax≥C. This is an optimal feasibility condition because when Smax< C no policy can be feasible. This reinforces the interest of (MP) that is not only optimal for energy con-sumption (on average) but is also optimal regarding feasibility.

Low Latency Big Data Processing without Prior Information

Job scheduling plays an important role in improving the overall system performance in big data processing frameworks. Simple job scheduling policies, such as Fair and FIFO scheduling, do not consider job sizes and may degrade the performance when jobs of varying sizes arrive. More elaborate job scheduling policies make the convenient assumption that jobs are recurring, and complete information about their sizes is available from their prior runs. In this paper, we design and implement an efficient and practical job scheduler for big data processing systems to achieve better performance even without prior information about job sizes. The superior performance of our job scheduler originates from the design of multiple level priority queues, where jobs are demoted to lower priority queues if the amount of service consumed so far reaches a certain threshold. In this case, jobs in need of a small amount of service can finish in the topmost several levels of queues, while jobs that need a large amount of service to complete are moved to lower priority queues to avoid head-of-line blocking. Our new job scheduler can effectively mimic the shortest job first scheduling policy without knowing the job sizes in advance. To demonstrate its performance, we have implemented our new job scheduler in YARN, a popular resource manager used by Hadoop/Spark, and validated its performance with experiments in both real testbeds including Amazon EC2 and large-scale trace-driven simulations. Our experimental and simulation results have strongly confirmed the effectiveness of our design: our new job scheduler can reduce the average job response time of the Fair scheduler by up to 45 percent and achieve better fairness at the same time.

Bi-criteria parallel batch machine scheduling to minimize total weighted tardiness and electricity cost

A bi-criteria scheduling problem for parallel identical batch processing machines in semiconductor wafer fabrication facilities is studied. Only jobs belonging to the same family can be batched together. The performance measures are the total weighted tardiness and the electricity cost where a time-of-use (TOU) tariff is assumed. Unequal ready times of the jobs and non-identical job sizes are considered. A mixed integer linear program (MILP) is formulated. We analyze the special case where all jobs have the same size, all due dates are zero, and the jobs are available at time zero. Properties of Pareto-optimal schedules for this special case are stated. They lead to a more tractable MILP. We design three heuristics based on grouping genetic algorithms that are embedded into a non-dominated sorting genetic algorithm II framework. Three solution representations are studied that allow for choosing start times of the batches to take into account the energy consumption. We discuss a heuristic that improves a given near-to-optimal Pareto front. Computational experiments are conducted based on randomly generated problem instances. The varepsilon -constraint method is used for both MILP formulations to determine the true Pareto front. For large-sized problem instances, we apply the genetic algorithms (GAs). Some of the GAs provide high-quality solutions.

Performance balancing size-interval routing policies

We study a parallel-queue system with Poisson arrivals, in which a dispatcher sends the incoming traffic to K queues using Size Interval Task Assignment (SITA) policy that aims to equalize the performance of all queues. We study existence and uniqueness of the allocation thresholds for a large set of performance functions of the queues. We also provide a family of performance functions of the queues such that the performance of the system is characterized. For a particular case of the latter family of functions, we show that the performance of the SITA policy we study coincides with the performance of the SITA policy in which the load is balanced. We investigate the optimality of the SITA policy under study and, according to our numerical experiments, for FCFS queues and Bounded Pareto distributed job sizes, the SITA policy we study is almost optimal, when we consider the mean queue length and as well as when we consider the mean slowdown.

Single-Machine Parallel-Batch Scheduling with Nonidentical Job Sizes and Rejection

We investigate the single-machine parallel-batch scheduling problem with nonidentical job sizes and rejection. In this problem, a set of jobs with different processing times and nonidentical sizes is given to be possibly processed on a parallel-batch processing machine. Each job is either accepted and then processed on the machine or rejected by paying its rejection penalty. Preemption is not allowed. Our task is to choose the accepted jobs and schedule them as batches on the machine to minimize the makespan of the accepted jobs plus the total rejection penalty of the rejected jobs. We provide an integer programming formulation to exactly solve our problem. Then, we propose three fast heuristic algorithms to solve the problem and evaluate their performances by using a small numerical example.

Efficient Heuristics for Total Completion Time and Maximum Lateness Bicriteria Scheduling Problems

This paper considers the bicriteria scheduling problem of minimizing the total completion time and maximum lateness with zero release dates. The problem is NP hard, though Early Due Date (EDD) and Shortest Processing Time (SPT) rules yielded optimal solutions for maximum lateness and total completion time, respectively if each criterion were to be considered singly. Thus, the values of each of the criteria for the two proposed heuristics; SII and PI were compared to the optimal solution of the sub problems. Results of computational experiment on job sizes varies from 5-100 jobs showed that the two proposed solution methods yielded results not significantly different from the optimal. This is because the two heuristic yielded results not significantly from the optimal sub-problems for the two performance measures at 95% significant level.

PARALLEL PROCESSORS SCHEDULING ALGORITHMS TO MINIMISE MAKESPAN IN A GALVANISING PLANT

Ponte Academic JournalMar 2020, Volume 76, Issue 3 PARALLEL PROCESSORS SCHEDULING ALGORITHMS TO MINIMISE MAKESPAN IN A GALVANISING PLANTAuthor(s): Mendon Dewa ,Bakhe NleyaJ. Ponte - Mar 2020 - Volume 76 - Issue 3 doi: 10.21506/j.ponte.2020.3.18 Abstract:Galvanising lines consists of load/loading stations and a series of processing tanks that are generally energy-intensive. Each raw part needs to go through a number of processing stages sequentially. Job sizes and processing time vary from part to part, hence the need to derive an optimal schedule to minimise total processing time in a batch. The problem of minimizing the makespan on parallel processing machines using different scheduling algorithms is studied in this paper. A set of independent tasks were scheduled on parallel processors in order to minimize schedule length using Integer Linear Programming, Shortest Processing Time, Longest Processing Time, and Greedy Genetic algorithms. The experimental results demonstrated that our Greedy Genetic algorithm outperformed other algorithms on minimizing makespan on parallel processing machines. Download full text:Check if you have access through your login credentials or your institution Username Password

Using GRASP Approach and Path Relinking to Minimize Total Number of Tardy Jobs on a Single Batch Processing Machine

This paper considers the problem of scheduling a single batch processing machine such that the total number of tardy jobs is minimized. The machine can simultaneously process several jobs as a batch as long as the machine capacity is not violated. The batch processing time is equal to the largest processing time among those jobs in the batch. Two decisions are made to schedule jobs on the batch processing machine, namely grouping jobs to form batches and sequencing the batches on the machines. Both the decisions are interdependent as the composition of the batch affects the processing time of the batch. The problem under study is NP-hard. Consequently, solving a mathematical formulation to find an optimal solution is computationally intensive. A Greedy Randomized Adaptive Search Procedure (GRASP) is proposed to solve the problem under study with the assumption of arbitrary job sizes, arbitrary processing times and arbitrary due dates. A novel construction phase for the GRASP approach is proposed to improve the solution quality. In addition, a path relinking procedure is proposed for solving large-sized problems effectively. The performance of the proposed GRASP approach is evaluated by comparing its results to a commercial solver (which was used to solve the mathematical model) and a construction heuristic. Experimental studies suggest that the solution obtained from the GRASP approach is superior compared to the commercial solver and the construction heuristic.

Estimation of mean response time in supercomputer systems with rigid jobs and inaccurate job size information.

Estimation of mean response time in supercomputer systems with rigid jobs and inaccurate job size information.