Shortest Remaining Processing Time Research Articles

Performance of the Gittins Policy in the G/G/1 and G/G/k, With and Without Setup Times

We consider the classic problem of preemptively scheduling jobs in a queue to minimize mean number-in-system, or equivalently mean response time. Even in single-server queueing models, this can be a nontrivial problem whose answer depends on the information available to the scheduler. The simplest case is when the scheduler knows each job's size, for which the optimal policy is Shortest Remaining Processing Time (SRPT) [9]. In the more realistic case of scheduling with unknown or partially known job sizes, people consider the Gittins policy [1-3, 12]. Roughly speaking, Gittins assigns each job a scalar rank based on an estimate of its remaining work, then serves the job of least rank.

When Does the Gittins Policy Have Asymptotically Optimal Response Time Tail in the M/G/1?

Looking Beyond Mean Delay in Queue Scheduling A central aim of queueing theory is to design scheduling policies that reduce delays. Much of literature focuses on minimizing mean delay, but, in practice, it can be more important to reduce the delay tail—that is, the probability of especially large delays. There is a subtle interplay between these objectives. For instance, the SRPT policy (Shortest Remaining Processing Time) prioritizes short jobs over long jobs, which optimizes mean delay. Sometimes, SRPT also has optimal delay tail, but other times, its delay tail is pessimal. Thus, good mean delay need not imply good delay tail. The article When Does the Gittins Policy Have Asymptotically Optimal Response Time Tail in the M/G/1? explores the interplay between mean delay and delay tail for systems with unknown job durations. The article studies the Gittins policy, which optimizes mean delay in this setting. Curiously, its delay tail can be optimal, pessimal, or in between, depending on the job duration distribution. Fortunately, the worst case can be avoided: the article shows how to modify Gittins to avoid pessimal delay tail while maintaining near-optimal mean delay.

SEH: Size Estimate Hedging Scheduling of Queues

For a single server system, Shortest Remaining Processing Time (SRPT) is an optimal size-based policy. In this article, we discuss scheduling a single-server system when exact information about the jobs’ processing times is not available. When the SRPT policy uses estimated processing times, the underestimation of large jobs can significantly degrade performance. We propose an index-based policy with a single parameter, Size Estimate Hedging (SEH), that only uses estimated processing times for scheduling decisions. A job’s priority is increased dynamically according to an SRPT rule until it is determined that it is underestimated, at which time the priority is frozen. Numerical results suggest that SEH has desirable performance for estimation error variance that is consistent with what is seen in practice.

A New Toolbox for Scheduling Theory

Queueing delays are ubiquitous in many domains, including computer systems, service systems, communication networks, supply chains, and transportation. Queueing and scheduling theory provide a rigorous basis for understanding how to reduce delays with scheduling, including evaluating policy performance and guiding policy design. Unfortunately, stateof- the-art theory fails to address many practical concerns. For example, scheduling theory seldom treats nontrivial preemption limitations, and there is very little theory for scheduling in multiserver queues. My thesis presents two new, broadly applicable tools that greatly expand the reach of scheduling theory, using each to solve multiple open problems. The first tool, called "SOAP", is a new unifying theory of scheduling in single-server queues, specifically the M/G/1 model. SOAP characterizes the delay distribution of a broad space of policies, most of which have never been analyzed before. Such policies include the Gittins index policy, which minimizes mean delay in low-information settings, and many policies with preemption limitations. The second tool, called "WINE", is a new queueing identity that complements Little's law. WINE enables a new method of analyzing complex queueing systems by relating them to simpler systems. This results in the first delay bounds for Shortest Remaining Processing Time (SRPT) and the Gittins policy in multiserver queues, specifically the M/G/k model. This abstract gives a brief overview of my thesis, describing what the SOAP and WINE tools do, the key ideas underlying them, and the open problems they help solve.

Heavy traffic scaling limits for shortest remaining processing time queues with heavy tailed processing time distributions

We study a single server queue operating under the shortest remaining processing time (SRPT) scheduling policy; that is, the server preemptively serves the job with the shortest remaining processing time first. Since one needs to keep track of the remaining processing times of all jobs in the system in order to describe the evolution, a natural state descriptor for an SRPT queue is a measure valued process in which the state of the system at a given time is the finite nonnegative Borel measure on the nonnegative real line that puts a unit atom at the remaining processing time of each job in system. In this work we are interested in studying the asymptotic behavior of the suitably scaled measure valued state descriptors for a sequence of SRPT queuing systems. Gromoll, Kruk and Puha (Stoch. Syst. 1 (2011) 1–16) have studied this problem under diffusive scaling (time is scaled by r2 and the mass of the measure normalized by r, where r is a scaling parameter approaching infinity). In the setting where the processing time distributions have bounded support, under suitable conditions, they show that the measure valued state descriptors converge in distribution to the process that at any given time is a single atom located at the right edge of the support of the processing time distribution with the size of the atom fluctuating randomly in time. In the setting where the processing time distributions have unbounded support, under suitable conditions, they show that the diffusion scaled measure valued state descriptors converge in distribution to the process that is identically zero. In Puha (Ann. Appl. Probab. 25 (2015) 3381–3404) for the setting where the processing time distributions have unbounded support and light tails, a nonstandard scaling of the queue length process is shown to give rise to a form of state space collapse that results in a nonzero limit. In the current work we consider the case where processing time distributions have finite second moments and regularly varying tails. Results of Puha (Ann. Appl. Probab. 25 (2015) 3381–3404) suggest that the right scaling for the measure valued process is governed by a parameter cr that is given as a certain inverse function related to the tails of the first moment of the processing time distribution. Using this parameter we consider a novel scaling for the measure valued process in which the time is scaled by a factor of r2, the mass is scaled by the factor cr/r and the space (representing the remaining processing times) is scaled by the factor 1/cr. We show that the scaled measure valued process converges in distribution (in the space of paths of measures). In a sharp contrast to results for bounded support and light tailed service time distributions, this time there is no state space collapse and the limiting measures are not concentrated on a single atom. Nevertheless, the description of the limit is simple and given explicitly in terms of a certain R+ valued random field which is determined from a single Brownian motion. Along the way we establish convergence of suitably scaled workload and queue length processes. We also show that as the tail of the distribution of job processing times becomes lighter in an appropriate fashion, the difference between the limiting queue length process and the limiting workload process converges to zero, thereby approaching the behavior of state space collapse.

Size-Based Congestion Control Using Network Utility Maximization

This article presents a method for allocating transmission rates in a communication network that uses the knowledge of the size of the file to be transmitted to seek to obtain the benefits of shortest remaining processing time (SRPT) scheduling. It provides fairness among file transmissions, taking into account their sizes and the congestion on the links they use. The proposed approach is based on network utility maximization (NUM), but with utility functions that vary in time depending on the remaining file size. A parameterized family of utility functions is proposed. For each value of the parameter, there is a threshold load such that an arbitrary network will be stable, provided that no link load exceeds that threshold. This is in contrast with SRPT, for which any nonzero load becomes unstable for topologies with sufficiently long paths. The method can be implemented by a distributed algorithm analogous to that of traditional NUM.

Approximate and Deployable Shortest Remaining Processing Time Scheduler

The scheduling policy installed on switches of datacenters plays a significant role on congestion control. Shortest-Remaining-Processing-Time (SRPT) achieves the near-optimal average message completion time (MCT) in various scenarios, but is difficult to deploy as viewed by the industry. The reasons are two-fold: 1) many commodity switches only provide FIFO queues, and 2) the information of remaining message size is not available. Recently, the idea of emulating SRPT using only a few FIFO queues and the original message size has been coined as the approximate and deployable SRPT (ADS) design. In this paper, we provide the first theoretical study on the optimal ADS design. Specifically, we first characterize a wide range of feasible ADS scheduling policies via a unified framework, and then derive the steady-state MCT, slowdown, and impoliteness in the M/G/1 setting. Hence we formulate the optimal ADS design as a non-linear combinatorial optimization problem, which aims to minimize the average MCT given the available FIFO queues. We also take into account the proportional fairness and temporal fairness constraints based on the maximal slowdown and impoliteness, respectively. The optimal ADS design problem is NP-hard in general, and does not exhibit monotonicity or sub-modularity. We leverage its decomposable structure and devise an efficient algorithm to solve the optimal ADS policy. We carry out extensive flow-level simulations and packet-level experiments to evaluate the proposed optimal ADS design. Results show that the optimal ADS policy installed on eight FIFO queues is capable of emulating the true SRPT.

Instability of SRPT, SERPT and SJF multiclass queueing networks

We provide two examples of strictly subcritical multiclass queueing networks which are unstable under the shortest remaining processing time (SRPT) service protocol. Both of them are reentrant lines with two servers and eight customer classes. The customer service times in our first system are deterministic, yielding an example of an unstable shortest remaining expected processing time (SERPT) network. In the second one, the service times in one customer class are randomized. Both our examples show also system instability under the shortest job first (SJF) discipline. A simulation study of robustness of our results with respect to changes in the customer interarrival and service times is also provided. Our results indicate that size-based service policies may not use the available resources efficiently in a multiserver network setting and in fact cause instability effects. This is in sharp contrast with their satisfactory performance for single-server queues.

SRPT Scheduling Discipline in Many-Server Queues with Impatient Customers

The shortest-remaining-processing-time (SRPT) scheduling policy has been extensively studied, for more than 50 years, in single-server queues with infinitely patient jobs. Yet, much less is known about its performance in multiserver queues. In this paper, we present the first theoretical analysis of SRPT in multiserver queues with abandonment. In particular, we consider the [Formula: see text] queue and demonstrate that, in the many-sever overloaded regime, performance in the SRPT queue is equivalent, asymptotically in steady state, to a preemptive two-class priority queue where customers with short service times (below a threshold) are served without wait, and customers with long service times (above a threshold) eventually abandon without service. We prove that the SRPT discipline maximizes, asymptotically, the system throughput, among all scheduling disciplines. We also compare the performance of the SRPT policy to blind policies and study the effects of the patience-time and service-time distributions. This paper was accepted by Baris Ata, stochastic models & simulation.

Anti-aging scheduling in single-server queues: A systematic and comparative study

The age of information (AoI) is a new performance metric recently proposed for measuring the freshness of information in information-update systems. In this work, we conduct a systematic and comparative study to investigate the impact of scheduling policies on the AoI performance in single-server queues and provide useful guidelines for the design of AoI-efficient scheduling policies. Specifically, we first perform extensive simulations to demonstrate that the update-size information can be leveraged for achieving a substantially improved AoI compared to non-size-based (orarrival-time-based) policies. Then, by utilizing both the update-size and arrival-time information, we propose three AoI-based policies. Observing improved AoI performance of policies that allow service preemption and that prioritize informative updates, we further propose preemptive, informative, AoI-based scheduling policies. Our simulation results show that such policies empirically achieve the best AoI performance among all the considered policies. However, compared to the best delay-efficient policies (such as shortest remaining processing time (SRPT)), the AoI improvement is rather marginal in the settings with exogenous arrivals. Interestingly, we also prove sample-path equivalence between some size-based policies and AoI-based policies. This provides an intuitive explanation for why some size-based policies (such as SRPT) achieve a very good AoI performance.

Minimality of SRPT Networks With Resource Sharing

A general multi-resource network with users requiring service from a number of shared resources simultaneously is considered. It is demonstrated that the Shortest Remaining Processing Time (SRPT) service protocol minimizes, in a suitable sense, the system resource idleness with respect to customers with residual service times not greater than any threshold value on every network route. Our arguments are pathwise, with no assumptions on the model stochastic primitives and the network topology.

Heavy traffic analysis for single-server SRPT and LRPT queues via EDF diffusion limits

Extending the results of Kruk (Queueing theory and network applications. QTNA 2019. Lecture notes in computer science, vol 11688. Springer, Cham, pp 263–275, 2019), we derive heavy traffic limit theorems for a single server, single customer class queue in which the server uses the Shortest Remaining Processing Time (SRPT) policy from heavy traffic limits for the corresponding Earliest Deadline First queueing systems. Our analysis allows for correlated customer inter-arrival and service times and heavy-tailed inter-arrival and service time distributions, as long as the corresponding stochastic primitive processes converge weakly to continuous limits under heavy traffic scaling. Our approach yields simple, concise justifications and new insights for SRPT heavy traffic limit theorems of Gromoll et al. (Stoch Syst 1(1):1–16, 2011). Corresponding results for the longest remaining processing time policy are also provided.

Efficient Online Scheduling for Coflow-Aware Machine Learning Clusters

Distributed machine learning (DML) is an increasingly important workload. In a DML job, each communication phase can comprise a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">coflow</i> , and there are dependencies among its coflows. Thus, efficient coflow scheduling becomes critical for DML jobs. However, the majority of existing solutions focus on scheduling single-stage coflows with no dependencies. While there are a few studies schedule dependent coflows of multi-stage jobs, they suffer from either practical or theoretical issues. Motivated by this situation, we study how to schedule dependent coflows of multiple DML jobs to minimize the total JCT in a shared cluster. We present a formal mathematical formulation for this problem and prove its NP-hardness. To solve this problem without job size information, we present an online coflow-aware optimization framework called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Parrot</i> . The core idea in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Parrot</i> is to infer the job with the shortest remaining processing time (SRPT) each time and dynamically control the inferred job's bandwidth based on how confident it is an SRPT job while being mindful of not starving any other job. Specifically, in the design of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Parrot</i> , we present a least per-coflow attained service (LPCAS) policy to infer the SRPT job. We further propose a dynamic job weight assignment mechanism and a linear program (LP) based weighted bandwidth scaling strategy for sharing bandwidth among DML jobs. We have proved that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Parrot</i> algorithm has a non-trivial competitive ratio. The results from large-scale trace-driven simulations further demonstrate that our <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Parrot</i> can reduce the total JCT by up to 58.4 percent, compared to the state-of-the-art Aalo solution.

Boosting Edge Intelligence With Collaborative Cross-Edge Analytics

Edge intelligence is emerging as a new interdiscipline to accelerate the convergence of artificial intelligence (AI) and Internet of Things (IoT). By applying edge computing, edge intelligence enables resource-limited IoT devices to offload computation-intensive AI applications to the network edge for execution. While existing efforts have focused on optimizing the model training and inference phases of edge intelligence, the initial phase of edge intelligence-collaborative cross-edge analytics that preprocess the unlabeled raw data dispersed at multiple edge sites to obtain labeled and trainable data samples-has greatly overlooked. To bridge this gap, in this article, we study how to jointly optimize the input data and task placement, with the goal of speeding up collaborative cross-edge analytics at low network traffic cost. This problem is by no means trivial since it is nonconvex and involving future uncertainty of the query characteristics. To accommodate these dual challenges, we blend the advantages of the convex relaxation method and a two-stage optimization. Specifically, based on a prediction of the query characteristics, the input data placement is first determined when it is generated, and then when the query job arrives, the actual value of the query characteristics is used to optimize the task placement. The problem becomes more complicated when we have multiple queries simultaneously. In response, we develop an efficient flow scheduling for the intermediate data transfers of competing queries by extending the classic shortest remaining processing time (SRPT) policy. Extensive trace-driven simulations verify the efficacy of our proposed solution.

Size-based scheduling for TCP flows: Implementation and performance evaluation

In many theoretical works, the benefits of size-based scheduling disciplines have been proved. The Foreground–Background discipline (or Least Attained Service (LAS)) and the multi-level processor sharing discipline are two examples of scheduling algorithms that provide an improvement of the expected flow completion time for heavily-tailed TCP flows with respect to the widely adopted Processor Sharing (PS) discipline.In this work, we provide an implementation of the 2-level processor sharing queue (2LPS) for a Linux kernel and we measure its performance with TCP flow sizes taken from real-world datasets. 2LPS is the simplest implementation of a size-based scheduling and is parametrised with a threshold a: at each epoch, flows which received less than a service have hard priority on those that have crossed the threshold.We give a numerical procedure to solve the 2LPS queue which allows for the computation of the optimal threshold for arbitrary job size distributions (approximated arbitrary well by generalised hyper-exponential distributions) and compare the results with those obtained by other disciplines (LAS and Shortest Remaining Processing Time (SRPT)).Finally, we measure the performance of its implementation. Our findings show that the benefits of 2LPS, although slightly lower than those predicted by the theoretical model, are still significant and that using just two levels with optimal threshold seems the best trade-off between the complexity and resource demand of the implementation of a size-based scheduler and the improvements in the expected response time.

Multiple Server SRPT With Speed Scaling Is Competitive

Can the popular shortest remaining processing time (SRPT) algorithm achieve a constant competitive ratio on multiple servers when server speeds are adjustable (speed scaling) with respect to the flow time plus energy consumption metric? This question has remained open for a while, where a negative result in the absence of speed scaling is well known. The main result of this paper is to show that multi-server SRPT with speed scaling can be constant competitive, with a competitive ratio that only depends on the power-usage function of the servers, but not on the number of jobs/servers or the job sizes (unlike when speed scaling is not allowed). When all job sizes are unity, we show that round-robin routing is optimal and can achieve the same competitive ratio as the best known algorithm for the single server problem. Finally, we show that a class of greedy dispatch policies, including policies that route to the least loaded or the shortest queue, do not admit a constant competitive ratio. When job arrivals are stochastic, with Poisson arrivals and i.i.d. job sizes, we show that random routing and a simple gated-static speed scaling algorithm achieves a constant competitive ratio.

Simple Near-Optimal Scheduling for the M/G/1

We consider the problem of preemptively scheduling jobs to minimize mean response time of an M/G/1 queue. When we know each job's size, the shortest remaining processing time (SRPT) policy is optimal. Unfortunately, in many settings we do not have access to each job's size. Instead, we know only the job size distribution. In this setting the Gittins policy is known to minimize mean response time, but its complex priority structure can be computationally intractable. A much simpler alternative to Gittins is the shortest expected remaining processing time (SERPT) policy. While SERPT is a natural extension of SRPT to unknown job sizes, it is unknown whether or not SERPT is close to optimal for mean response time.

Characterizing Policies with Optimal Response Time Tails under Heavy-Tailed Job Sizes

We consider the tail behavior of the response time distribution in anM/G/1 queue with heavy-tailed job sizes, specifically those with intermediately regularly varying tails. In this setting, the response time tail of many individual policies has been characterized, and it is known that policies such as Shortest Remaining Processing Time (SRPT) and Foreground-Background (FB) have response time tails of the same order as the job size tail, and thus such policies are tail-optimal. Our goal in this work is to move beyond individual policies and characterize the set of policies that are tail-optimal. Toward that end, we use the recently introduced SOAP framework to derive sufficient conditions on the form of prioritization used by a scheduling policy that ensure the policy is tail-optimal. These conditions are general and lead to new results for important policies that have previously resisted analysis, including the Gittins policy, which minimizes mean response time among policies that do not have access to job size information. As a by-product of our analysis, we derive a general upper bound for fractional moments of M/G/1 busy periods, which is of independent interest.

Characterizing Policies with Optimal Response Time Tails under Heavy-Tailed Job Sizes

We consider the tail behavior of the response time distribution in an M/G/1 queue with heavy-tailed job sizes, specifically those with intermediately regularly varying tails. In this setting, the response time tail of many individual policies has been characterized, and it is known that policies such as Shortest Remaining Processing Time (SRPT) and Foreground-Background (FB) have response time tails of the same order as the job size tail, and thus such policies are tail-optimal. Our goal in this work is to move beyond individual policies and characterize the set of policies that are tail-optimal. Toward that end, we use the recently introduced SOAP framework to derive sufficient conditions on the form of prioritization used by a scheduling policy that ensure the policy is tail-optimal. These conditions are general and lead to new results for important policies that have previously resisted analysis, including the Gittins policy, which minimizes mean response time among policies that do not have access to job size information. As a by-product of our analysis, we derive a general upper bound for fractional moments of M/G/1 busy periods, which is of independent interest.

Simple Near-Optimal Scheduling for the M/G/1

We consider the problem of preemptively scheduling jobs to minimize mean response time of an M/G/1 queue. When we know each job's size, the shortest remaining processing time (SRPT) policy is optimal. Unfortunately, in many settings we do not have access to each job's size. Instead, we know only the job size distribution. In this setting the Gittins policy is known to minimize mean response time, but its complex priority structure can be computationally intractable. A much simpler alternative to Gittins is the shortest expected remaining processing time (SERPT) policy. While SERPT is a natural extension of SRPT to unknown job sizes, it is unknown whether or not SERPT is close to optimal for mean response time. We present a new variant of SERPT called monotonic SERPT (M-SERPT) which is as simple as SERPT but has provably near-optimal mean response time at all loads for any job size distribution. Specifically, we prove the mean response time ratio between M-SERPT and Gittins is at most 3 for load ρ łeq 8/9$ and at most 5 for any load. This makes M-SERPT the only non-Gittins scheduling policy known to have a constant-factor approximation ratio for mean response time.