Processor-sharing Service Research Articles

Approximations and Optimal Control for State-Dependent Limited Processor Sharing Queues

The paper studies approximations and control of a processor sharing (PS) server where the service rate depends on the number of jobs occupying the server. The control of such a system is implemented by imposing a limit on the number of jobs that can share the server concurrently, with the rest of the jobs waiting in a first-in-first-out (FIFO) buffer. A desirable control scheme should strike the right balance between efficiency (operating at a high service rate) and parallelism (preventing small jobs from getting stuck behind large ones). We use the framework of heavy-traffic diffusion analysis to devise near optimal control heuristics for such a queueing system. However, although the literature on diffusion control of state-dependent queueing systems begins with a sequence of systems and an exogenously defined drift function, we begin with a finite discrete PS server and propose an axiomatic recipe to explicitly construct a sequence of state-dependent PS servers that then yields a drift function. We establish diffusion approximations and use them to obtain insightful and closed-form approximations for the original system under a static concurrency limit control policy. We extend our study to control policies that dynamically adjust the concurrency limit. We provide two novel numerical algorithms to solve the associated diffusion control problem. Our algorithms can be viewed as “average cost” iteration: The first algorithm uses binary-search on the average cost, while the second faster algorithm uses Newton-Raphson method for root finding. Numerical experiments demonstrate the accuracy of our approximation for choosing optimal or near-optimal static and dynamic concurrency control heuristics.

Redundancy with Processor Sharing servers

The main motivation to investigate redundancy models comes from empirical evidence suggesting that redundancy can help improve the performance of real-world applications. Under redundancy, a job that arrives to this system is dispatched to d servers uniformly chosen at random in order to benefit from the variability of the length of these queues. As soon as one of the copies finishes service, the job (and its copies) is removed from the system, and as a consequence, a job's delay is given by the minimum delay among the servers its copies are sent to. Most of the literature on performance evaluation of redundancy systems has been carried out when First Come First Served (FCFS) is implemented in the servers. In particular, for exponential service time distributions, Gardner et al. [4, 5] and Bonald and Comte [2] show that the stability region is not reduced due to adding redundant copies. In this extended abstract, we focus instead on Processor Sharing (PS) service policy and study how redundancy impacts the stability condition. In particular, we aim to study the impact that the correlation structure of the copies has on the performance of the redundancy-d model. In a recent paper, Gardner et al. [3] showed that the assumption of independent and identically distributed (i.i.d.) copies, can be unrealistic, and that it might lead to theoretical results that do not reflect the results of replication schemes in real-life computer systems. We consider the two extreme cases of correlation; (i) the copies are i.i.d. (ii) the copies of a job are exact replicas (identical copies). We observe that the stability condition strongly depends on the correlation structure, as well as on the number of redundant copies.

Cross-Layer Design for Mission-Critical IoT in Mobile Edge Computing Systems

In this work, we propose a cross-layer framework for optimizing user association, packet offloading rates, and bandwidth allocation for Mission-Critical Internet-of-Things (MC-IoT) services with short packets in Mobile Edge Computing (MEC) systems, where enhanced Mobile BroadBand (eMBB) services with long packets are considered as background services. To reduce communication delay, the 5th generation new radio is adopted in radio access networks. To avoid long queueing delay for short packets from MC-IoT, Processor-Sharing (PS) servers are deployed at MEC systems, where the service rate of the server is equally allocated to all the packets in the buffer. We derive the distribution of latency experienced by short packets in closed-form, and minimize the overall packet loss probability subject to the end-to-end delay requirement. To solve the non-convex optimization problem, we propose an algorithm that converges to a near optimal solution when the throughput of eMBB services is much higher than MC-IoT services, and extend it into more general scenarios. Furthermore, we derive the optimal solutions in two asymptotic cases: communication or computing is the bottleneck of reliability. Simulation and numerical results validate our analysis and show that the PS server outperforms first-come-first-serve servers.

Asymptotically Optimal Job Assignment for Energy-Efficient Processor-Sharing Server Farms

We study the problem of job assignment in a large-scale realistically dimensioned server farm comprising multiple processor-sharing servers with different service rates, energy consumption rates, and buffer sizes. Our aim is to optimize the energy efficiency of such a server farm by effectively controlling carried load on networked servers. To this end, we propose a job assignment policy, called Most energy-efficient available server first Accounting for Idle Power (MAIP), which is both scalable and near optimal. MAIP focuses on reducing the productive power used to support the processing service rate. Using the framework of semi-Markov decision process, we show that, with exponentially distributed job sizes, MAIP is equivalent to the well-known Whittle’s index policy. This equivalence and the methodology of Weber and Weiss enable us to prove that, in server farms where a loss of jobs happens if and only if all buffers are full, MAIP is asymptotically optimal, as the number of servers tends to infinity under certain conditions associated with the large number of servers, as we have in a real server farm. Through extensive numerical simulations, we demonstrate the effectiveness of MAIP and its robustness to different job-size distributions, and observe that significant improvement in energy efficiency can be achieved by utilizing the knowledge of energy consumption rate of idle servers.

Energy-Efficient Heuristics for Insensitive Job Assignment in Processor-Sharing Server Farms

Energy efficiency of server farms is an important design consideration of the green datacenter initiative. One effective approach is to optimize power consumption of server farms by controlling the carried load on the networked servers. In this paper, we propose a robust heuristic policy called E* for stochastic job assignment in a server farm, aiming to improve the energy efficiency by maximizing the ratio of job throughput to power consumption. Our model of the server farm considers a parallel system of finite-buffer processor-sharing queues with heterogeneous server speeds and energy consumption rates. We devise E* as an insensitive policy so that the stationary distribution of the number of jobs in the system depends on the job size distribution only through its mean. We provide a rigorous analysis of E* and compare it with a baseline approach, known as most energy-efficient server first (MEESF), that greedily chooses the most energy-efficient servers for job assignment. We show that E* has always a higher job throughput than that of MEESF, and derive realistic conditions under which E* is guaranteed to outperform MEESF in energy efficiency. Extensive numerical results are presented and demonstrate that E* can improve the energy efficiency by up to 100%.

Insensitive Job Assignment With Throughput and Energy Criteria for Processor-Sharing Server Farms

We study the problem of stochastic job assignment in a server farm comprising multiple processor-sharing servers with various speeds and finite buffer sizes. We consider two types of assignment policies: without jockeying, where an arriving job is assigned only once to an available server, and with jockeying, where a job may be reassigned at any time. We also require that the underlying Markov process under each policy is insensitive. Namely, the stationary distribution of the number of jobs in the system is independent of the job size distribution except for its mean. For the case without jockeying, we derive two insensitive heuristic policies: One aims at maximizing job throughput, and the other trades off job throughput for energy efficiency. For the case with jockeying, we formulate the optimal assignment problem as a semi-Markov decision process and derive optimal policies with respect to various optimization criteria. We further derive two simple insensitive heuristic policies with jockeying: One maximizes job throughput, and the other aims at maximizing energy efficiency. Numerical examples demonstrate that, under a wide range of system parameters, the latter policy performs very close to the optimal policy. Numerical examples also demonstrate energy/throughput tradeoffs for the various policies and, in the case with jockeying, they show a potential of substantial energy savings relative to a policy that optimizes throughput.

Staffing and Control of Instant Messaging Contact Centers

In addition to traditional call centers, many companies have started building a new kind of customer contact center, in which agents communicate with customers via instant messaging (IM) over the Internet rather than phone calls. A distinctive feature of the service centers based on IM is that one agent can serve multiple customers in parallel. We choose to model such a center as a server pool consisting of many limited processor-sharing servers. We characterize the underlying stochastic processes by establishing a fluid approximation in the many-server heavy-traffic regime. The limiting behavior of the stochastic processes is shown to involve a stochastic averaging principle, and the fluid approximation provides insights into the optimal staffing and control for such service centers.

Performance Evaluation of Carrier Aggregation for Elastic Traffic in LTE-Advanced Systems

Carrier aggregation is a potential technology for the LTE-Advanced system to support wider bandwidth than the LTE system. This paper analyzes the performance of carrier aggregation under elastic traffic, and compares it to that of a simpler approach for the same purpose, referred to as the independent carrier approach. The queueing behaviors of these two approaches are formulated as one fast versus multiple slow state-dependent Processor Sharing servers, respectively. Both analytical and simulation results show that when there are L component carriers with uniform bandwidth in the system, the performance of the carrier aggregation approach is L times better than that of the independent carrier approach in terms of the average user delay and throughput under the same traffic load.

Insensitivity for PS server farms with JSQ routing

Join-the-Shortest-Queue (JSQ) is a very old and popular routing policy for server farms. Figure 1 shows two examples of server farm architectures employing JSQ routing. In both cases, each incoming job is immediately dispatched, via a front-end router, to the queue with the fewest number of jobs, designated as the shortest queue (ties are broken at random). In Figure 1(a), jobs at a queue are served in First-Come-First-Served (FCFS) order. In Figure 1(b), jobs within a queue are served according to Processor-Sharing (PS), meaning that when there are n jobs at a queue, they share the processing capacity, each simultaneously receiving 1/nth of the service. We refer to Figure 1(a) as a JSQ/FCFS server farm and to Figure 1(b) as a JSQ/PS farm. If more detail is needed, we use the notation: M/G/K/JSQ/PS, denoting a Poisson arrival process, i.i.d. job sizes from a general distribution, K servers, JSQ routing; and PS scheduling at queues.

Analysis of join-the-shortest-queue routing for web server farms

Join the Shortest Queue (JSQ) is a popular routing policy for server farms. However, until now all analysis of JSQ has been limited to First-Come-First-Serve (FCFS) server farms, whereas it is known that web server farms are better modeled as Processor Sharing (PS) server farms. We provide the first approximate analysis of JSQ in the PS server farm model for general job-size distributions, obtaining the distribution of queue length at each queue. To do this, we approximate the queue length of each queue in the server farm by a one-dimensional Markov chain, in a novel fashion. We also discover some interesting insensitivity properties of PS server farms with JSQ routing, and discuss the near-optimality of JSQ.

Bandwidth Optimization Algorithm Based on Weight Vector Adjustment in Generalized Processor Sharing Servers

We consider the bandwidth optimization problem in a Generalized Processor Sharing (GPS) server to minimize the total bandwidth such that QoS requirements for each class queue are satisfied. Our previous optimization algorithm [6] requires rather long optimization time to solve the problem. We propose a new optimization algorithm based on weight vector adjustment. Numerical results show that the required time to find the optimal resource in GPS servers is significantly reduced, compared to the previous algorithm. key words: Internet traffic, QoS, GPS, bandwidth allocation, optimization

Process Algebraic Non-product-forms

A generalization of the Reversed Compound Agent Theorem of Markovian process algebra is derived that yields separable, but non-product-form solutions for collections of interacting processes such as arise in multi-class queueing networks with Processor Sharing servers. It is based on an analysis of the minimal cycles in the state space of a multi-agent cooperation, which can be simply identified. The extended methodology leads to what we believe are new separable solutions and, more generally, the results represent a viable practical application of the theory of Markovian process algebras in stochastic modelling.

Individual Equilibrium and Learning in Processor Sharing Systems

We consider a processor-sharing service system, where the service rate to individual customers decreases as the load increases. Each arriving customer may observe the current load and should then choose whether to join the shared system. The alternative is a constant-cost option, modeled here for concreteness as a private server (e.g., a personal computer that serves as an alternative to a central mainframe computer). The customers wish to minimize their individual service times (or an increasing function thereof). However, the optimal choice for each customer depends on the decisions of subsequent ones, through their effect on the future load in the shared server. This decision problem is analyzed as a noncooperative dynamic game among the customers. We first show that any Nash equilibrium point consists of threshold decision rules and establish the existence and uniqueness of a symmetric equilibrium point. Computation of the equilibrium threshold is demonstrated for the case of Poisson arrivals, and some of its properties are delineated. We next consider a reasonable dynamic learning scheme, which converges to the symmetric Nash equilibrium point. In this model customers simply choose the better option based on available performance history. Convergence of this scheme is illustrated here via a simulation example and is established analytically in subsequent work.

Fluid approximations for a processor-sharing queue

In this paper a fluid approximation, also known as a functional strong law of large numbers (FSLLN) for a GI/G/1 queue under a processor-sharing service discipline is established and its properties are analysed. The fluid limit depends on the arrival rate, the service time distribution of the initial customers, and the service time distribution of the arriving customers. This is in contrast to the known result for the GI/G/1 queue under a FIFO service discipline, where the fluid limit is piecewise linear and depends on the service time distribution only through its mean. The piecewise linear form of the limit can be recovered by an equilibrium type choice of the initial service distribution.

The Unconditional Sojourn Time Distribution in the Queue GI/M/1‐K with Processor Sharing Service

We consider a processor shared GI/M/1 queue which can accommodate a finite number K of customers. Using singular perturbation techniques, we construct asymptotic expansions for the distribution of a tagged customer's sojourn time. We assume that K is large and treat several different cases of the model parameters.

Efficient algorithms for simulating service disciplines

Efficient techniques for simulating the Round-Robin and Processor-Sharing service disciplines are presented. These schemes are general in that they can be applied in any discrete-event simulation view to improve execution performance. For the Round-Robin discipline, the proposed approach is based on computation instead of process switching. For the Processor-Sharing discipline, we introduce a lazy approach in combination with any efficient priority queue structure to reduce scheduling-relating computation. Using the S i simulation testbed for Simulating (process) interactions, the proposed algorithms are shown to be more efficient than the standard algorithms for simulating these service disciplines.

The M/G/1 processor-sharing model: transient behavior

This paper deals with the M/G/1 model with processor-sharing service discipline. LetL*(t, x) denote the number of jobs present at timet whose attained service time is not greater thanx,x⩾0, andV0(t,z) the sojourn time of a tagged job placed in the system at timet and requiringz units of service. Explicit analytical expressions are obtained for the joint distribution ofL*(t, ·) andV0(t, ·) under various initial conditions in terms of the Laplace transform with respect tot. It is shown that for initial conditions of special kind (there is one job or none) the results can be expressed in a closed form.

The Conditional Sojourn Time Distribution in the GI/M/1 Processor-Sharing Queue in Heavy Traffic

We treat the GI/M/1 queue with a processor-sharing server, in the heavy traffic case. Using perturbation methods, we construct asymptotic expansions for the conditional sojourn time distribution of a tagged customer conditioned on the tagged customer's service time. The resulting approximation is simple in form and involves only the first three moments of the interarrival time distribution.

The conditional sojourn time distribution in the GI/M/1 processor-sharing queue in heavy traffic

We treat the GI/M/1 queue with a processor-sharing server, in the heavy traffic case. Using perturbation methods, we construct asymptotic expansions for the conditional sojourn time distribution of a tagged customer conditioned on the tagged customer's service time. The resulting approximation is simple in form and involves only the first three moments of the interarrival time distribution.

Brownian Models of Feedforward Queueing Networks: Quasireversibility and Product Form Solutions

We consider a very general type of $d$-station open queueing network, with multiple customer classes and a more or less arbitrary service discipline at each station, but restricted by the requirement that customers always flow from lower numbered stations to higher numbered ones. To approximate the behavior of such a queueing network under heavy traffic conditions, a corresponding Brownian network model is proposed and it is shown that the approximating Brownian model reduces to a $d$-dimensional reflected Brownian motion $W$ whose state space is the nonnegative orthant. A necessary and sufficient condition for $W$ to have a product form stationary distribution (that is, a stationary distribution with independent components) and a probabilistic interpretation for that condition are given. Our interpretation involves a notion of quasireversibility analogous to that introduced by Kelly and elaborated by Walrand in their brilliant analysis of product form solutions for conventional queueing network models. Three illustrative queueing network models are discussed in detail and the analysis of these examples shows how a Brownian network approximation may have a product form stationary distribution even when the original or exact model is intractable. Particularly intriguing in that regard are two examples involving non-Poisson inputs, deterministic routing, deterministic service times and processor-sharing service disciplines.