Processor Sharing Research Articles

Approximations to Study the Impact of the Service Discipline in Systems with Redundancy

In this paper we develop the first methods to approximate the queue length distribution in a queueing system with redundancy under various service disciplines. We focus on a system with exponential job sizes, i.i.d. copies, and a large number of servers. In the case of Processor Sharing, we provide a pair and a triplet approximation. We also develop a pair approximation for First-Come-First-Served, Limited Processor Sharing and Last-Come-First-Served servers. We present numerical evidence that shows that all the approximations presented in the paper are highly accurate, but that none of them are asymptotically exact (as the number of servers goes to infinity).

Approximations to Study the Impact of the Service Discipline in Systems with Redundancy

As job redundancy has been recognized as an effective means to improve performance of large-scale computer systems, queueing systems with redundancy have been studied by various authors. Existing results include methods to compute the queue length distribution and response time but only when the service discipline is First-Come-First-Served (FCFS). For other service disciplines, such as Processor Sharing (PS), or Last-Come-First-Served (LCFS), only the stability conditions are known. In this paper we develop the first methods to approximate the queue length distribution in a queueing system with redundancy under various service disciplines. We focus on a system with exponential job sizes, i.i.d. copies, and a large number of servers. We first derive a mean field approximation that is independent of the scheduling policy. In order to study the impact of service discipline, we then derive refinements of this approximation to specific scheduling policies. In the case of Processor Sharing, we provide a pair and a triplet approximation. The pair approximation can be regarded as a refinement of the classic mean field approximation and takes the service discipline into account, while the triplet approximation further refines the pair approximation. We also develop a pair approximation for three other service disciplines: First-Come-First-Served, Limited Processor Sharing and Last-Come-First-Served. We present numerical evidence that shows that all the approximations presented in the paper are highly accurate, but that none of them are asymptotically exact (as the number of servers goes to infinity). This makes these approximations suitable to study the impact of the service discipline on the queue length distribution. Our results show that FCFS yields the shortest queue length, and that the differences are more substantial at higher loads.

Efficient and accurate Lyapunov function-based truncation technique for multi-dimensional Markov chains with applications to discriminatory processor sharing and priority queues

Online service providers aim to satisfy the tail performance requirements of customers through Service-Level Objectives (SLOs). One approach to ensure tail performance requirements is to model the service as a Markov chain and obtain its steady-state probability distribution. However, obtaining the distribution can be challenging, if not impossible, for certain types of Markov chains, such as those with multi-dimensional or infinite state-space and state-dependent transitions. Examples include M/M/1 with Discriminatory Processor Sharing (DPS) and preemptive M/M/c with multiple priority classes and customer abandonment.To address this fundamental problem, we propose a Lyapunov function-based state-space truncation technique that leverages moments or bounds on moments of the state variables. This technique allows us to obtain tight truncation bounds while ensuring arbitrary probability mass guarantees for the truncated chain. We highlight the efficacy of our technique for multi-dimensional DPS and M/M/c priority queue with abandonment, demonstrating a substantial reduction in state space (up to 74%) compared to existing approaches. Additionally, we present three practical use cases that highlight the applicability of our truncation technique by analyzing the performance of the DPS system.

CoMP-Enhanced Flexible Functional Split for Mixed Services in Beyond 5G Wireless Networks

With explosively escalating service demands, beyond fifth generation (B5G) aims to realize various requirements for multi-service networks, i.e., higher performance of mixed enhanced mobile broadband (eMBB) and ultra-reliable low-latency communication (URLLC) services than 5G. To flexibly serve diverse traffic, various functional split options (FSOs) are specified by 5G protocols enabling different network functions. In order to improve signal qualities for edge users, we consider FSO-based coordinated multi-point (CoMP) transmission as a prominent technique capable of supporting high traffic demands. However, due to conventional confined hardware processing capability, a processor sharing (PS) model is introduced to deal with high latency for multi-service FSO-based networks. Therefore, it becomes essential to assign CoMP-enhanced functional split modes under PS model. A more tractable FSO-based network in terms of ergodic rate and reliability is derived by stochastic geometry approach. Moreover, we have proposed CoMP-enhanced functional split mode allocation (CFSMA) scheme to adaptively assign FSOs to provide enhanced mixed throughput and latency-aware services. The simulation results have validated analytical derivation and demonstrated that the proposed CFSMA scheme optimizes system spectrum efficiency while guaranteeing stringent latency requirement. The proposed CFSMA scheme with the designed PS FFS-CoMP system outperforms the benchmarks of conventional FCFS scheduling, non-FSO network, fixed FSOs, and limited available FSO selections in open literature.

WCFS: a new framework for analyzing multiserver systems

Multiserver queueing systems are found at the core of a wide variety of practical systems. Many important multiserver models have a previously-unexplained similarity: identical mean response time behavior is empirically observed in the heavy traffic limit. We explain this similarity for the first time. We do so by introducing the work-conserving finite-skip (WCFS) framework, which encompasses a broad class of important models. This class includes the heterogeneous M/G/k, the Limited Processor Sharing policy for the M/G/1, the Threshold Parallelism model and the Multiserver-Job model under a novel scheduling algorithm. We prove that for all WCFS models, scaled mean response time \(E[T](1-\rho )\) converges to the same value, \(E[S^2]/(2E[S])\), in the heavy-traffic limit, which is also the heavy traffic limit for the M/G/1/FCFS. Moreover, we prove additively tight bounds on mean response time for the WCFS class, which hold for all load \(\rho \). For each of the four models mentioned above, our bounds are the first known bounds on mean response time.

Improving the Mean-Field Fluid Model of Processor Sharing Queueing Networks for Dynamic Performance Models in Cloud Computing

Resource management in cloud computing is a difficult problem, as one needs to balance between adequate service to clients and cost minimization in a dynamic environment of interconnected components. To make correct decisions in such an environment, good performance models are necessary. A common modeling methodology is to use networks of queues, but as these are prohibitively expensive to evaluate for many applications, approximation methods for key metrics are frequently employed. One such method-that provides both transient solutions and short, scalable computation times-is the fluid model, which approximates the dynamics of the mean queue lengths using a system of ordinary differential equations. However, finding a fluid model that can adequately approximate an arbitrary queueing network is in general difficult. In the paper, we extend the state of the art with the following three contributions. First, we show that for any mixed multiclass queueing network of processor sharing and delay queues with phase-type service time distributions, such a fluid model can be found via the meanfield approximation. Furthermore, we propose an improved model based on smoothing of the processor share function that improves the performance of certain systems. Finally, using the smoothed mean-field model, we introduce an accurate closed-form approximation of the response time CDF over any subset of classes and queues.

Approximations and Optimal Control for State-Dependent Limited Processor Sharing Queues

The paper studies approximations and control of a processor sharing (PS) server where the service rate depends on the number of jobs occupying the server. The control of such a system is implemented by imposing a limit on the number of jobs that can share the server concurrently, with the rest of the jobs waiting in a first-in-first-out (FIFO) buffer. A desirable control scheme should strike the right balance between efficiency (operating at a high service rate) and parallelism (preventing small jobs from getting stuck behind large ones). We use the framework of heavy-traffic diffusion analysis to devise near optimal control heuristics for such a queueing system. However, although the literature on diffusion control of state-dependent queueing systems begins with a sequence of systems and an exogenously defined drift function, we begin with a finite discrete PS server and propose an axiomatic recipe to explicitly construct a sequence of state-dependent PS servers that then yields a drift function. We establish diffusion approximations and use them to obtain insightful and closed-form approximations for the original system under a static concurrency limit control policy. We extend our study to control policies that dynamically adjust the concurrency limit. We provide two novel numerical algorithms to solve the associated diffusion control problem. Our algorithms can be viewed as “average cost” iteration: The first algorithm uses binary-search on the average cost, while the second faster algorithm uses Newton-Raphson method for root finding. Numerical experiments demonstrate the accuracy of our approximation for choosing optimal or near-optimal static and dynamic concurrency control heuristics.

Convergence Bounds for Limited Processor Sharing Queue with Impatience for Analyzing Non-Stationary File Transfer in Wireless Network

The data transmission in wireless networks is usually analyzed under the assumption of non-stationary rates. Nevertheless, they strictly depend on the time of day, that is, the intensity of arrival and daily workload profiles confirm this fact. In this article, we consider the process of downloading a file within a single network segment and unsteady speeds—for arrivals, file sizes, and losses due to impatience. To simulate the scenario, a queuing system with elastic traffic with non-stationary intensity is used. Formulas are given for the main characteristics of the model: the probability of blocking a new user, the average number of users in service, and the queue. A method for calculating the boundaries of convergence of the model is proposed, which is based on the logarithmic norm of linear operators. The boundaries of the rate of convergence of the main limiting characteristics of the queue length process were also established. For clarity of the influence of the parameters, a numerical analysis was carried out and presented.

Processor Sharing Queues With Impatient Customers and State-Dependent Rates

We study queues with impatient customers and Processor Sharing (PS) discipline as well as other variants of PS discipline, namely, Discriminatory Processor Sharing (DPS) and Generalized Processor Sharing (GPS) disciplines, where customers have deadlines until the end of service (DES). Customers arrive according to a state-dependent Poisson process and have general impatience. Customers have exponential service times with state-dependent service rates. Analytical methods based on simple Markov chains are given for the performance analysis of such queues. The principal measures of performance are the steady-state probability of missing deadline and the steady-state probability of blocking. Similar results are obtained for related queues with Random Order Service (ROS) discipline where customers have deadlines until the beginning of service (DBS). In view of a lack of exact analytical results for First Come First Served (FCFS) queues with state-dependent rates, a highly accurate approximation method is also given for these latter queues. The efficacy and accuracy of the approach are illustrated by some numerical examples and simulation experiments.

Heavy-Traffic Comparison of a Discrete-Time Generalized Processor Sharing Queue and a Pure Randomly Alternating Service Queue

This paper compares two discrete-time single-server queueing models with two queues. In both models, the server is available to a queue with probability 1/2 at each service opportunity. Since obtaining easy-to-evaluate expressions for the joint moments is not feasible, we rely on a heavy-traffic limit approach. The correlation coefficient of the queue-contents is computed via the solution of a two-dimensional functional equation obtained by reducing it to a boundary value problem on a hyperbola. In most server-sharing models, it is assumed that the system is work-conserving in the sense that if one of the queues is empty, a customer of the other queue is served with probability 1. In our second model, we omit this work-conserving rule such that the server can be idle in case of a non-empty queue. Contrary to what we would expect, the resulting heavy-traffic approximations reveal that both models remain different for critically loaded queues.

Improving the mean-field fluid model of processor sharing queueing networks for dynamic performance models in cloud computing

Resource management in cloud computing is a difficult problem, as one is often tasked with balancing between adequate service to clients and cost minimization in dynamic environments of many interconnected components. To make correct decisions in these environments, good performance models are necessary. A common modeling methodology is to use networks of queues, but as these are prohibitively expensive to evaluate for many real-time applications, different approximation methods for important metrics are frequently employed. One such method—that provides both transient solutions and short, scalable computation times—is the fluid model, which approximates the dynamics of the mean queue lengths using a system of ordinary differential equations. However, finding a fluid model that can adequately approximate an arbitrary queueing network is in general difficult. In this paper, we extend the state of the art with the following three contributions. First, we show that for any mixed multiclass queueing network of processor sharing and delay queues with phase-type service time distributions, such a fluid model can be found via the mean-field approximation. Furthermore, we propose an improved model based on smoothing of the processor share function that improves the performance of certain systems. Finally, using the smoothed mean-field model, we introduce an accurate closed-form approximation of the response time CDF over any subset of classes and queues. The contributions are further evaluated in a large simulation experiment, which shows that they can be used to accurately predict performance metrics under some system perturbations common in cloud computing.

Instability of LIFO queueing networks

Under the last-in, first-out (LIFO) discipline, jobs arriving later at a class always receive priority of service over earlier arrivals at any class belonging to the same station. Subcritical LIFO queueing networks with Poisson external arrivals are known to be stable, but an open problem has been whether this is also the case when external arrivals are given by renewal processes. Here, we show that this weaker assumption is not sufficient for stability by constructing a family of examples where the number of jobs in the network increases to infinity over time. This behavior contrasts with that for the other classical disciplines: processor sharing (PS), infinite server (IS) and first-in, first-out (FIFO), which are stable under general conditions on the renewals of external arrivals. Together with LIFO, PS and IS constitute the classical symmetric disciplines; with the inclusion of FIFO, these disciplines constitute the classical homogeneous disciplines. Our examples show that a general theory for stability of either family is doubtful.

On the Asymptotic Insensitivity of the Supermarket Model in Processor Sharing Systems

The supermarket model is a popular load balancing model where each incoming job is assigned to a server with the least number of jobs among d randomly selected servers. Several authors have shown that the large scale limit in case of processor sharing servers has a unique insensitive fixed point, which naturally leads to the belief that the queue length distribution in such a system is insensitive to the job size distribution as the number of servers tends to infinity. Simulation results that support this belief have also been reported. However, global attraction of the unique fixed point of the large scale limit was not proven except for exponential job sizes, which is needed to formally prove asymptotic insensitivity. The difficulty lies in the fact that with processor sharing servers, the limiting system is in general not monotone. In this paper we focus on the class of hyperexponential distributions of order $2$ and demonstrate that for this class of distributions global attraction of the unique fixed point can still be established using monotonicity by picking a suitable state space and partial order. This allows us to formally show that we have asymptotic insensitivity within this class of job size distributions. We further demonstrate that our result can be leveraged to prove asymptotic insensitivity within this class of distributions for other load balancing systems.

On the Asymptotic Insensitivity of the Supermarket Model in Processor Sharing Systems

The supermarket model is a popular load balancing model where each incoming job is assigned to a server with the least number of jobs among d randomly selected servers. Several authors have shown that the large scale limit in case of processor sharing servers has a unique insensitive fixed point, which naturally leads to the belief that the queue length distribution in such a system is insensitive to the job size distribution as the number of servers tends to infinity. Simulation results that support this belief have also been reported. However, global attraction of the unique fixed point of the large scale limit was not proven except for exponential job sizes, which is needed to formally prove asymptotic insensitivity. The difficulty lies in the fact that with processor sharing servers, the limiting system is in general not monotone. In this paper we focus on the class of hyperexponential distributions of order 2 and demonstrate that for this class of distributions global attraction of the unique fixed point can still be established using monotonicity by picking a suitable state space and partial order. This allows us to formally show that we have asymptotic insensitivity within this class of job size distributions. We further demonstrate that our result can be leveraged to prove asymptotic insensitivity within this class of distributions for other load balancing systems.

Individual Service Curves for Bandwidth-Sharing Policies Using Network Calculus

Bandwidth-sharing policies guarantee a proportion of the service offered by a server to each class of traffic. These policies include GPS (Generalized Processor Sharing) and DRR (Deficit Round Robin). In this letter, we compute service curves for each class of traffic for such scheduling policies using network calculus. These individual service curves take into account the characteristics of the cross-traffic and thus improve the state-of-the-art performance bounds like delay or backlog.

On the Stability of Redundancy Models

The main motivation to investigate redundancy models comes from empirical evidence suggesting that redundancy can help improve the performance of real-world applications. Although there are several variants of a redundancy-based system, the general notion of redundancy is to create multiple copies of the same job that will be sent to a subset of servers. By allowing for redundant copies, the aim is to minimize the system latency by exploiting the variability in the queue lengths and the capacity of the different servers. In this article, the stability condition of redundancy multiserver systems is investigated. Several popular scheduling disciplines are considered, such as first-come-first-serve (FCFS), processor sharing (PS), and random order of service (ROS) and show that whereas with ROS the performance is not reduced, with both PS and FCFS the performance can severely degrade.

Frequency scaling in multilevel queues

In this paper, we study a variant of PS+PS multilevel scheduling, which we call the PS+IS queue. Specifically, we use Processor Sharing (PS) at both queues, but with linear frequency scaling on the second queue, so that the latter behaves like an Infinite Server (IS) queue. The goals of the system are low response times for small jobs in the first queue, and reduced power consumption for large jobs in the second queue. The novelty of our model includes the frequency scaling at the second queue, and the batch arrival process at the second queue induced by the busy period structure of the first queue which has strictly higher priority. We derive a numerical solution for the PS+IS queueing system in steady-state, and then study its properties under workloads obtained from fitting of TCP flow traces. The simulation results confirm the e

Stability and tail behavior of redundancy systems with processor sharing

We investigate the stability condition for redundancy-d systems where each of the servers follows a processor-sharing (PS) discipline. We allow for generally distributed job sizes, with possible dependence among the d replica sizes being governed by an arbitrary joint distribution. We establish that for homogeneous servers the stability condition for the associated fluid-limit model is characterized by the expectation of the minimum of d replica sizes being less than the mean interarrival time per server. In the special case of identical replicas, the stability condition is insensitive to the job size distribution given its mean, and the stability threshold is inversely proportional to the number of replicas. In the special case of i.i.d. replicas, the stability threshold decreases (increases) in the number of replicas for job size distributions that are NBU (NWU). We also discuss the extension to heterogeneous servers.For heavy-tailed job sizes we characterize the tail behavior of the response time distribution. In particular, for regularly varying job sizes with tail index −ν it is shown that the tail index of the response time equals −ν and −dν for identical and i.i.d. replicas, respectively.

Size-based scheduling for TCP flows: Implementation and performance evaluation

In many theoretical works, the benefits of size-based scheduling disciplines have been proved. The Foreground–Background discipline (or Least Attained Service (LAS)) and the multi-level processor sharing discipline are two examples of scheduling algorithms that provide an improvement of the expected flow completion time for heavily-tailed TCP flows with respect to the widely adopted Processor Sharing (PS) discipline.In this work, we provide an implementation of the 2-level processor sharing queue (2LPS) for a Linux kernel and we measure its performance with TCP flow sizes taken from real-world datasets. 2LPS is the simplest implementation of a size-based scheduling and is parametrised with a threshold a: at each epoch, flows which received less than a service have hard priority on those that have crossed the threshold.We give a numerical procedure to solve the 2LPS queue which allows for the computation of the optimal threshold for arbitrary job size distributions (approximated arbitrary well by generalised hyper-exponential distributions) and compare the results with those obtained by other disciplines (LAS and Shortest Remaining Processing Time (SRPT)).Finally, we measure the performance of its implementation. Our findings show that the benefits of 2LPS, although slightly lower than those predicted by the theoretical model, are still significant and that using just two levels with optimal threshold seems the best trade-off between the complexity and resource demand of the implementation of a size-based scheduler and the improvements in the expected response time.

Frequency scaling in multilevel queues

In this paper, we study a variant of PS+PS multilevel scheduling, which we call the PS+IS queue. Specifically, we use Processor Sharing (PS) at both queues, but with linear frequency scaling on the second queue, so that the latter behaves like an Infinite Server (IS) queue. The goals of the system are low response times for small jobs in the first queue, and reduced power consumption for large jobs in the second queue. The novelty of our model includes the frequency scaling at the second queue, and the batch arrival process at the second queue induced by the busy period structure of the first queue which has strictly higher priority. We derive a numerical solution for the PS+IS queueing system in steady-state, and then study its properties under workloads obtained from fitting of TCP flow traces. The simulation results confirm the efficacy of the PS+IS model.