Size Interval Task Assignment Research Articles

Heavy-Traffic Optimal Size- and State-Aware Dispatching

Dispatching systems, where arriving jobs are immediately assigned to one of multiple queues, are ubiquitous in computer systems and service systems. A natural and practically relevant model is one in which each queue serves jobs in FCFS (First-Come First-Served) order. We consider the case where the dispatcher is size-aware, meaning it learns the size (i.e. service time) of each job as it arrives; and state-aware, meaning it always knows the amount of work (i.e. total remaining service time) at each queue. While size- and state-aware dispatching to FCFS queues has been extensively studied, little is known about optimal dispatching for the objective of minimizing mean delay. A major obstacle is that no nontrivial lower bound on mean delay is known, even in heavy traffic (i.e. the limit as load approaches capacity). This makes it difficult to prove that any given policy is optimal, or even heavy-traffic optimal. In this work, we propose the first size- and state-aware dispatching policy that provably minimizes mean delay in heavy traffic. Our policy, called CARD (Controlled Asymmetry Reduces Delay), keeps all but one of the queues short, then routes as few jobs as possible to the one long queue. We prove an upper bound on CARD's mean delay, and we prove the first nontrivial lower bound on the mean delay of any size- and state-aware dispatching policy. Both results apply to any number of servers. Our bounds match in heavy traffic, implying CARD's heavy-traffic optimality. In particular, CARD's heavy-traffic performance improves upon that of LWL (Least Work Left), SITA (Size Interval Task Assignment), and other policies from the literature whose heavy-traffic performance is known.

Performance degradation in parallel-server systems with shared resources and lack of coordination

Parallel server systems are ubiquitous. Multicore CPUs are in practically every personal device from mobile handsets to high-end desktop PCs. At larger scale, data centers consist of a huge number of physical servers often shared by multiple users (for economic reasons). Moreover, the simultaneous users are typically unaware of each other due to reasons that can be technical (cf. security & privacy), practical (coordination layer would add complexity) and business related (usage can be business sensitive information). The workload patterns of different users may also vary significantly. This results in unpredictable and often long response times. We study means for tackling these challenges. In particular, we consider a model where multiple users (dispatchers) route their jobs to a pool of servers using uncoordinated static dispatching policies. The goal is to determine how different policies interact: whether users’ decisions support each other, or if some decisions are simply counterproductive. The lack of proper coordination is shown to increase the mean response times, with two common and robust dispatching policies: Size-Interval-Task Assignment (SITA) and Round-Robin (RR). We refer to this phenomenon as the price of ignorance.

Size-Based Routing Policies: Non-Asymptotic Analysis and Design of Decentralized Systems.

Size-based routing policies are known to perform well when the variance of the distribution of the job size is very high. We consider two size-based policies in this paper: Task Assignment with Guessing Size (TAGS) and Size Interval Task Assignment (SITA). The latter assumes that the size of jobs is known, whereas the former does not. Recently, it has been shown by our previous work that when the ratio of the largest to shortest job tends to infinity and the system load is fixed and low, the average waiting time of SITA is, at most, two times less than that of TAGS. In this article, we first analyze the ratio between the mean waiting time of TAGS and the mean waiting time of SITA in a non-asymptotic regime, and we show that for two servers, and when the job size distribution is Bounded Pareto with parameter , this ratio is unbounded from above. We then consider a system with an arbitrary number of servers and we compare the mean waiting time of TAGS with that of Size Interval Task Assignment with Equal load (SITA-E), which is a SITA policy where the load of all the servers are equal. We show that in the light traffic regime, the performance ratio under consideration is unbounded from above when (i) the job size distribution is Bounded Pareto with parameter and an arbitrary number of servers as well as (ii) for Bounded Pareto distributed job sizes with and the number of servers tends to infinity. Finally, we use the result of our previous work to show how to design decentralized systems with quality of service constraints.

Performance balancing size-interval routing policies

We study a parallel-queue system with Poisson arrivals, in which a dispatcher sends the incoming traffic to K queues using Size Interval Task Assignment (SITA) policy that aims to equalize the performance of all queues. We study existence and uniqueness of the allocation thresholds for a large set of performance functions of the queues. We also provide a family of performance functions of the queues such that the performance of the system is characterized. For a particular case of the latter family of functions, we show that the performance of the SITA policy we study coincides with the performance of the SITA policy in which the load is balanced. We investigate the optimality of the SITA policy under study and, according to our numerical experiments, for FCFS queues and Bounded Pareto distributed job sizes, the SITA policy we study is almost optimal, when we consider the mean queue length and as well as when we consider the mean slowdown.

Asymptotically Optimal Size-Interval Task Assignments

Size-based routing provides robust strategies to improve the performance of computer and communication systems with highly variable workloads because it is able to isolate small jobs from large ones in a static manner. The basic idea is that each server is assigned all jobs whose sizes belong to a distinct and continuous interval. In the literature, dispatching rules of this type are referred to as SITA (Size Interval Task Assignment) policies. Though their evident benefits, the problem of finding a SITA policy that minimizes the overall mean (steady-state) waiting time is known to be intractable. In particular it is not clear when it is preferable to balance or unbalance server loads and, in the latter case, how. In this paper, we provide an answer to these questions in the celebrated limiting regime where the system capacity grows linearly with the system demand to infinity. Within this framework, we prove that the minimum mean waiting time achievable by a SITA policy necessarily converges to the mean waiting time achieved by SITA-E, the SITA policy that equalizes server loads, provided that servers are homogeneous. However, within the set of SITA policies we also show that SITA-E can perform arbitrarily bad if servers are heterogeneous. In this case we prove that there exist exactly $C!$C! asymptotically optimal policies, where $C$C denotes the number of server types, and all of them are linked to the solution of a single strictly convex optimization problem. It turns out that the mean waiting time achieved by any of such asymptotically optimal policies does not depend on how job-size intervals are mapped to servers. Our theoretical results are validated by numerical simulations with respect to realistic parameters and suggest that the above insights are also accurate in small systems composed of a few servers, i.e., ten.

Combining Size-Based Load Balancing with Round-Robin for Scalable Low Latency

When dispatching jobs to parallel servers, or queues, the highly scalable round-robin (RR) scheme reduces the variance of interarrival times at all queues to a great extent but has no impact on the variances of service processes. Contrariwise, size-interval task assignment (SITA) routing has little impact on the variances of interarrival times but makes the service processes as deterministic as possible. In this paper, we unify both ‘static’ approaches to design a scalable load balancing framework able to control the variances of the arrival and service processes jointly. It turns out that the resulting combination significantly improves performance and is able to drive the mean job delay to zero in the large-system limit; it is known that this property is not achieved when both approaches are considered separately. Within realistic parameters, we show that the optimal number of size intervals that partition the support of the job size distribution is small with respect to the system size. This enhances the applicability of the proposed load balancing scheme at a large scale. In fact, we find that adding a little bit of information about job sizes to a dispatcher operating under RR improves performance a lot. Under the optimal scaling of size intervals and assuming highly variable job sizes, numerical simulations indicate that the proposed algorithm is competitive with the (less scalable) join-the-shortest-workload algorithm even when the system size grows large.

Performance Degradation in Parallel-Server Systems

We consider a parallel-server system with homogeneous servers where incoming tasks, arriving at rate $\lambda $ , are dispatched by $n$ dispatchers, each of them balancing a fraction $1/n$ of the load to $K/n$ servers. Servers are first-come-first-served (FCFS) queues and dispatchers implement size interval task assignment policy with equal load (SITA-E), a size-based policy such that the servers are equally loaded. We compare the performance of a system with $n>1$ dispatchers and a single dispatcher. We show that the performance of a system with $n$ dispatchers, $K$ servers, and arrival rate $\lambda $ coincides with that of a system with one dispatcher, $K/n$ servers, and arrival rate $\lambda /n$ . We define the degradation factor as the ratio between the performance of a system with $K$ servers and arrival rate $\lambda $ and the performance of a system with $K/n$ servers and arrival rate $\lambda /n$ . We establish a partial monotonicity on $n$ for the degradation factor and, therefore, the degradation factor is lower bounded by one. We then investigate the upper bound of the degradation factor for particular distributions. We consider two continuous service time distributions: uniform and bounded Pareto and a discrete distribution with two values, which is the distribution that maximizes the variance for a given mean. We show that the performance degradation is small for uniformly distributed job sizes but that for Bounded Pareto and two points distributions it can be unbounded. We have investigated the degradation using the distribution obtained from real traces.

Steady-State Analysis for Multiserver Queues Under Size Interval Task Assignment in the Quality-Driven Regime

We study the steady-state behavior of multiserver queues with general job size distributions under size interval task assignment (SITA) policies. Assuming Poisson arrivals and the existence of the αth moment of the job size distribution for some α > 1, we show that if the job arrival rate and the number of servers increase to infinity with the traffic intensity held fixed, the SITA policy parameterized by α minimizes in a large deviation sense the steady-state probability that the total number of jobs in the system is greater than or equal to the number of servers. The optimal large deviation decay rate can be arbitrarily close to the one for the corresponding probability in an infinite-server queue, which only depends on the system traffic intensity but not on any higher moments of the job size distribution. This supports in a many-server asymptotic framework the common wisdom that separating large jobs from small jobs protects system performance against job size variability.

Analysis of SITA policies

We analyze the performance of Size Interval Task Assignment (SITA) policies, for multi-host assignment in a non-preemptive environment. Assuming Poisson arrivals, we provide general bounds on the average waiting time independent of the job size distribution. We establish a general duality theory for the performance of SITA policies. We provide a detailed analysis of the performance of SITA systems when the job size distribution is Bounded Pareto and the range of job sizes tends to infinity. In particular, we determine asymptotically optimal cutoff values and provide asymptotic formulas for average waiting time and slowdown. We compare the results with the Least Work Remaining policy and compute which policy is asymptotically better for any given set of parameters. In the case of inhomogeneous hosts, we determine their optimal ordering.

Analysis of size interval task assignment policies

We analyze the performance of Size Interval task assignment (SITA) scheduling policies, for multi-host scheduling in a non-preemptive environment. We establish a general duality theory for the performance analysis of SITA policies. When the job size distribution is Bounded Pareto and the range of job sizes tends to infinity. we determine asymptotically optimal cutoff values and provide asymptotic formulas for average waiting time and slowdown. In the case of inhomogeneous hosts we determine their optimal ordering. We also consider TAGS policies. We provide a general formula that describes their load handling capabilities and examine their performance when the job size distribution is Bounded Pareto.

Task assignment in a distributed system (extended abstract)

We consider the problem of task assignment in a distributed system (such as a distributed Web server) in which task sizes are drawn from a heavy-tailed distribution. Many task assignment algorithms are based on the heuristic that balancing the load at the server hosts will result in optimal performance. We show this conventional wisdom is less true when the task size distribution is heavy-tailed (as is the case for Web file sizes). We introduce a new task assignment policy, called Size Interval Task Assignment with Variable Load (SITA-V). SITA-V purposely operates the server hosts at different loads, and directs smaller tasks to the lighter-loaded hosts. The result is that SITA-V provably decreases the mean task slowdown by significant factors (up to 1000 or more) where the more heavy-tailed the workload, the greater the improvement factor. We evaluate the tradeoff between improvement in slowdown and increase in waiting time in a system using SITA-V, and show conditions under which SITA-V represents a particularly appealing policy.