Average Job Response Time Research Articles

Light traffic performance under the power of two load balancing strategy

We consider the power-of-two policy with d = 2, Poisson job arrivals, heterogeneous servers and a general job requirement distribution. With the help of the first two light traffic derivatives for the average job response time, we point to interesting structural features associated with server heterogeneity in light traffic.

Analytical Performance Models for MapReduce Workloads

MapReduce is a currently popular programming model to support parallel computations on large datasets. Among the several existing MapReduce implementations, Hadoop has attracted a lot of attention from both industry and research. In a Hadoop job, map and reduce tasks coordinate to produce a solution to the input problem, exhibiting precedence constraints and synchronization delays that are characteristic of a pipeline communication between maps (producers) and reduces (consumers). We here address the challenge of designing analytical models to estimate the performance of MapReduce workloads, notably Hadoop workloads, focusing particularly on the intra-job pipeline parallelism between map and reduce tasks belonging to the same job. We propose a hierarchical model that combines a precedence graph model and a queuing network model to capture the intra-job synchronization constraints. We first show how to build a precedence graph that represents the dependencies among multiple tasks of the same job. We then apply it jointly with an approximate Mean Value Analysis (aMVA) solution to predict mean job response time, throughput and resource utilization. We validate our solution against a queuing network simulator and a real setup in various scenarios, finding very close agreement in both cases. In particular, our model produces estimates of average job response time that deviate from measurements of a real setup by less than 15 %.

A runtime resolution scheme for priority boost conflict in implicit coscheduling

High-performance parallel and scientific applications are composed of multiple processes running on distinct CPUs that communicate frequently. Due to the synchronization needs of such applications, performance is greatly hampered if their processes are not scheduled simultaneously on the CPUs. Implicit coscheduling (ICS) is a well-known technique to address this problem in multi-programmed clusters, however, traditional ICS schemes do not incorporate steps to adequately deal with priority boost conflicts, leading to significantly degraded performance. In this paper, we propose the use of runtime difference in contention across nodes to provide more sophisticated coscheduling decisions in response to the conflicts. We also present a novel coscheduling scheme termed PROC (Process ReOrdering-based Coscheduling) that adaptively regulates the scheduling sequence of conflicting processes based on the rescheduling latency of their correspondents in remote nodes. We perform extensive simulation-based experiments using both synthetic and realistic workloads to analyze the performance of PROC compared to alternatives such as local scheduling, a widely used batch scheduling, gang scheduling, and existing ICS schemes. The results show that all ICS schemes commonly experience priority boost conflicts, and that the proposed PROC significantly outperforms other ICS alternatives (or batch scheduling) by up to 50.4% (or 72.5%) in the average job response time. This improvement is achieved by reducing wasted idle time and spinning time without sacrificing fairness.

Scheduling flows with unknown sizes

Previous job scheduling studies indicate that providing rapid response to interactive jobs which place frequent but small demands, can reduce the overall system average response time [1], especially when the job size distribution is skewed (see [2] and references therein). Since the distribution of Internet flows is skewed, it is natural to design a network system that favors short file transfers through service differentiation. However, to maintain system scalability, detailed per-flow state such as flow length is generally not available inside the network. As a result, we usually resort to a threshold-based heuristic to identify and give preference to short flows. Specifically, packets from a new flow are always given the highest priority. However, the priority is reduced once the flow has transferred a certain amount of packets.In this paper, we use the MultiLevel (ML) feedback queue [3] to characterize this discriminatory system. However, the solution given in [3] is in the form of an integral equation, and to date the equation has been solved only for job size distribution that has the form of mixed exponential functions. We adopt an alternative approach, namely using a conservation law by Kleinrock [1], to solve for the average response time in such system. To that end, we approximate the average response time of jobs by a linear function in the job size and solve for the stretch (service slowdown) factors. We show by simulation that such approximation works well for job (flow) size distributions that possess the heavy-tailed property [2], although it does not work so well for exponential distributions.Due to the limited space available, in Section 2 we briefly describe the queueing model and summarize our approximation approach to solving for the average response time of the M/G/1/ML queueing system. We conclude our paper in Section 3.

Efficient processor management schemes for mesh-connected multicomputers

This paper investigates various processor management techniques for improving the performance of mesh-connected multicomputers. Unlike almost all prior work where the focus was on improving the submesh recognition ability of the processor allocation algorithms, this research examines other alternatives to improve system performance beyond what is achievable with usually assumed first come first served (FCFS) scheduling and any allocation. First, we use the smallest job first (SJF) policy to improve the spatial parallelism in a mesh. Next, we introduce a generic processor management scheme called multitasking and multiprogramming (M 2). Then, an M 2 policy for mesh-connected multicomputers called virtual mesh (VM) is proposed and analyzed. The proposed VM scheme allows multiprogramming of jobs on several VMs. Finally, a novel approach called limit allocation is used for job allocation. With this scheme, a job (submesh) size is reduced if the job cannot be allocated. The objective here is to reduce the job waiting time and hence improve the overall performance. While all of the three approaches are viable alternatives to reduce the average job response time under various workloads, the VM and the limit allocation techniques are especially attractive for providing some additional features. The VM scheme brings in the concept of time-sharing execution for better efficiency and limit allocation shows how job size restriction can be beneficial for performance and fault-tolerance in a mesh topology. Moreover, the limit allocation scheme using even the simplest allocation policy can outperform any other approach.

Analytical Modelling of a Refined Fcfs Queueing Discipline in Multiprogrammed Parallel Processing Systems

A multiprogrammed (or partitionable) parallel computer system can be partitioned into subsystems which are allocated to independent jobs. The performance of such a parallel processing system is deter-mined by the system partitioning strategy and the job scheduling policy adopted. In this paper, we consider the impact of a job scheduling algorithm on the system performance, mainly the average job response time. A refined First-Come-First-Serve (FCFS) queueing discipline called FCFS/SJE is studied, which, when there are not enough processors available for large jobs in the front of a waiting queue, schedules small jobs behind these large jobs earlier. We develop a multi-server queueing system for modelling multiprogrammed parallel processing systems. A system is specified by the number of processors and jobs are characterized by a probability distribution of the number of processors requested, a Poisson arrival process, and an exponential distribution of execution time. Our analysis as well as simulation indicate that FCFS/SJE results in a noticeable improvement on system performance compared with FCFS.

Optimizing Average Job Response Time via Decentralized Probabilistic Job Dispatching in Heterogeneous Multiple Computer Systems

In this paper, we study decentralized probabilistic job dispatching and load balancing strategies which optimize the performance of heterogeneous multiple computer systems. We present a model to study a heterogeneous multiple computer system with a decentralized stochastic job dispatching mechanism, where nodes are treated as M/G/1 servers. We discuss a way to implement a virtual centralized job dispatcher using a distributed control mechanism. We derive closed form solutions for optimal job dispatching probabilities which minimize the average job response time, when all nodes have an identical coefficient of variation for job execution times. We also generalize the results to the case where nodes have different coefficients of variation for job execution times.

Dynamic resource management on distributed systems using reconfigurable applications

Efficient management of distributed resources, under conditions of unpredictable and varying workload, requires enforcement of dynamic resource management policies. Execution of such policies requires a relatively fine-grain control over the resources allocated to jobs in the system. Although this is a difficult task using conventional job management and program execution models, reconfigurable applications can be used to make it viable. With reconfigurable applications, it is possible to dynamically change, during the course of program execution, the number of concurrently executing tasks of an application as well as the resources allocated. Thus, reconfigurable applications can adapt to internal changes in resource requirements and to external changes affecting available resources. In this paper, we discuss dynamic management of resources on distributed systems with the help of reconfigurable applications. We first characterize reconfigurable parallel applications. We then present a new programming model for reconfigurable applications and the Distributed Resource Management System (DRMS), an integrated environment for the design, development, execution, and resource scheduling of reconfigurable applications. Experiments were conducted to verify the functionality and performance of application reconfiguration under DRMS. A detailed breakdown of the costs in reconfiguration is presented with respect to several different applications. Our results indicate that application reconfiguration is effective under DRMS and can be beneficial in improving individual application performance as well as overall system performance. We observe a significant reduction in average job response time and an improvement in overall system utilization.

An approximate analysis of the join the shortest queue (JSQ) policy

This paper presents an accurate analytical model for evaluating the performance of the join the shortest queue (JSQ) policy. The system considered consists of N identical queues each of which may have single or multiple servers. A birth-death Markov process is used to model the evolution of the number of jobs in the system. Our results show that this method provides very accurate estimates of the average job response times.

Static and decentralized-adaptive load balancing in a star configured distributed computing system

The problem of minimizing the average job response time (sojourn time) in a star configured distributed computing system (DCS) with N peripheral processors and one central processor is studied. Each peripheral processor (satellite) receives a stream of Poisson arrivals. A job arriving at a satellite S/sub i/ will be processed with a probability (1-P/sub i/) by itself and routed to the central site for remote processing with probability P/sub i/. The service time distribution at each processor is arbitrary with means 1//spl mu//sub i/ and second moments E[Y/sub i//sup 2/], i=0,...,N. We first develop an algorithm to compute the optimal routing probabilities when the arrival and the service parameters are known a priori. We then develop an adaptive estimator to be used by each satellite to compute its optimal routing probability, when the parameters (everywhere) are unknown a priori, a copy of the algorithm runs on each peripheral processor and uses the sojourn time measurements that the peripheral processor can gather on its own from those jobs that are sent to the central processor and returned to the originating site after service.

A dynamic load-balancing policy with a central job dispatcher (LBC)

A dynamic load-balancing policy is proposed with a central job dispatcher called the LBC policy for distributed systems. The design of this policy is motivated by the operation of a single-queue multiserver queueing system, and the average job response time is the same as that of a single-queue multiserver system, which is the best achievable performance when the communication delay is reduced to zero. Hence, near-minimum average job response time is expected for distributed systems with high-speed communication subnets. The performance is studied for systems with nonnegligible job transfer delays in the following three aspects: average job response time, overhead due to information exchanges, and sensitivity to heterogeneous load. >

Processor-pool-based scheduling for large-scale NUMA multiprocessors

Large-scale Non-Uniform Memory Access (NUMA) multiprocessors are gaining increased attention due to their potential for achieving high performance through the replication of relatively simple components. Because of the complexity of such systems, scheduling algorithms for parallel applications are crucial in realizing the performance potential of these systems. In particular, scheduling methods must consider the scale of the system, with the increased likelihood of creating bottlenecks, along with the NUMA characteristics of the system, and the benefits to be gained by placing threads close to their code and data.We propose a class of scheduling algorithms based on processor pools . A processor pool is a software construct for organizing and managing a large number of processors by dividing them into groups called pools. The parallel threads of a job are run in a single processor pool, unless there are performance advantages for a job to span multiple pools. Several jobs may share one pool. Our simulation experiments show that processor pool-based scheduling may effectively reduce the average job response time. The performance improvements attained by using processor pools increase with the average parallelism of the jobs, the load level of the system, the differentials in memory access costs, and the likelihood of having system bottlenecks. As the system size increasesr, while maintaining the workload composition and intensity, we observed that processor pools can be used to provide significant performance improvements. We therefore conclude that processor pool-based scheduling may be an effective and efficient technique for scalable systems.

Adaptive optimal load balancing in a nonhomogeneous multiserver system with a central job scheduler

A model comprising several servers, each equipped with its own queue and with possibly different service speeds, is considered. Each server receives a dedicated arrival stream of jobs; there is also a stream of generic jobs that arrive to a job scheduler and can be individually allocated to any of the servers. It is shown that if the arrival streams are all Poisson and all jobs have the same exponentially distributed service requirements, the probabilistic splitting of the generic stream that minimizes the average job response time is such that it balances the server idle times in a weighted least-squares sense, where the weighting coefficients are related to the service speeds of the servers. The corresponding result holds for nonexponentially distributed service times if the service speeds are all equal. This result is used to develop adaptive quasi-static algorithms for allocating jobs in the generic arrival stream when the load parameters are unknown. The algorithms utilize server idle-time measurements which are sent periodically to the central job scheduler. A model is developed for these measurements, and the result mentioned is used to cast the problem into one of finding a projection of the root of an affine function, when only noisy values of the function can be observed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Optimal Load Balancing in a Multiple Processor System with Many Job Classes

A loosely coupled multiprocessor system contains multiple processors which have their own local memories. To balance the load among multiple processors is of fundamental importance in enhancing the performance of such a multiple processor system. Probabilistic load balancing in a heterogeneous multiple processor system with many job classes is considered in this study. The load balancing scheme is formulated as a nonlinear programming problem with linear constraints. An optimal probabilistic load balancing algorithm is proposed to solve this nonlinear programming problem. The proposed load balancing method is proven globally optimum in the sense that it results in a minimum overall average job response time on a probabilistic basis.