Parallel Queues Research Articles

Guaranteeing performance based on time-stability for energy-efficient data centers

ABSTRACTWe consider a system of multiple parallel single-server queues where servers are heterogeneous with resources of different capacities and can be powered on or off while running at different speeds when they are powered on. In addition, we assume that application requests are heterogeneous with different workload distributions and resource requirements and the arrival, rates of request are time-varying. Managing such a heterogeneous, transient, and non-stationary system is a tremendous challenge. We take an unconventional approach, in that we force the queue lengths in each powered-on server to be time-stable (i.e., stationary). It allows the operators to guarantee performance and effectively monitor the system. We formulate a mixed-integer program to minimize energy costs while satisfying time-stability. Simulation results show that our suggested approach can stabilize queue length distributions and provide probabilistic performance guarantees on waiting times.

On Round-Robin routing with FCFS and LCFS scheduling

We study the Round-Robin (RR) routing to a system of parallel queues, which serve jobs according to FCFS or preemptive LCFS scheduling disciplines. The cost structure comprises two components: a service fee and a queueing delay related component. With Poisson arrivals, the inter-arrival time to each queue obeys the Erlang distribution. This allows us to study the mean and transient behavior of the queues separately. The service fee is independent of the scheduling and queueing, and we obtain the corresponding mean cost rate and value function in closed forms. With respect to queueing delay, we first derive integral expressions enabling efficient computation of the corresponding size-aware value functions. By decomposition, these yield also the value function for the whole system of m parallel queues fed by RR. Given the value function, one can carry out the first policy iteration step with arbitrary holding cost rates (e.g., delay, slowdown, etc.) yielding efficient size-, cost- and state-aware policies. Moreover, the mean waiting and sojourn times in the corresponding systems get resolved at the same time. The results are demonstrated in the numerical examples, where we compute near optimal job routing policies for sample systems.

Markovian inventory model with two parallel queues, jockeying and impatient customers

This article presents a perishable stochastic inventory system under continuous review at a service facility consisting of two parallel queues with jockeying. Each server has its own queue, and jockeying among the queues is permitted. The capacity of each queue is of finite size L. The inventory is replenished according to an (s; S) inventory policy and the replenishing times are assumed to be exponentially distributed. The individual customer is issued a demanded item after a random service time, which is distributed as negative exponential. The life time of each item is assumed to be exponential. Customers arrive according to a Poisson process and on arrival; they join the shortest feasible queue. Moreover, if the inventory level is more than one and one queue is empty while in the other queue, more than one customer are waiting, then the customer who has to be received after the customer being served in that queue is transferred to the empty queue. This will prevent one server from being idle while the customers are waiting in the other queue. The waiting customer independently reneges the system after an exponentially distributed amount of time. The joint probability distribution of the inventory level, the number of customers in both queues, and the status of the server are obtained in the steady state. Some important system performance measures in the steady state are derived, so as the long-run total expected cost rate.

МОДЕЛЬ УПРАВЛЕНИЯ ДВУМЯ ПАРАЛЛЕЛЬНЫМИ FIFO-ОЧЕРЕДЯМИ, ДВИГАЮЩИМИСЯ ДРУГ ЗА ДРУГОМ В ОБЩЕЙ ПАМЯТИ

МОДЕЛЬ УПРАВЛЕНИЯ ДВУМЯ ПАРАЛЛЕЛЬНЫМИ FIFO-ОЧЕРЕДЯМИ, ДВИГАЮЩИМИСЯ ДРУГ ЗА ДРУГОМ В ОБЩЕЙ ПАМЯТИ

Maximum weight matching with hysteresis in overloaded queues with setups

We consider a system of parallel queues where arriving service tasks are buffered, according to type. Available service resources are dynamically configured and allocated to the queues to process the tasks. At each point in time, a scheduler chooses a service configuration across the queues, in response to queue backlogs. Switching from one service configuration to another incurs a setup time, during which idling occurs and service bandwidth is lost. Such setup times are inherent in manufacturing and computer systems. Frequent switchings can significantly compromise the service capacity of such systems. A maximum weight matching (MWM) scheduler, which is known to maximize throughput in the absence of setups, can easily become unstable with setups, even under low load. To remedy this problem, we propose a new MWM-H scheduler which utilizes a controller introduced hysteresis and achieves maximum throughput even with setups, without requiring knowledge of arrival rates and average traffic loads. During prolonged traffic bursts, the queues may become overloaded and the issue becomes how to reasonably distribute the growing backlog under MWM-H. It is shown that by appropriately selecting the MWM-H parameters, one can control the backlog among the individual queues in order to achieve a desired balance.

Dynamic Subtask Dispersion Reduction in Heterogeneous Parallel Queueing Systems

Fork-join and split-merge queueing systems are mathematical abstractions of parallel task processing systems in which entering tasks are split into N subtasks which are served by a set of heterogeneous servers. The original task is considered completed once all the subtasks associated with it have been serviced. Performance of split-merge and fork-join systems are often quantified with respect to two metrics: task response time and subtask dispersion. Recent research effort has been focused on ways to reduce subtask dispersion, or the product of task response time and subtask dispersion, by applying delays to selected subtasks. Such delays may be pre-computed statically, or varied dynamically. Dynamic in our context refers to the ability to vary the delay applied to a subtask according to the state of the system, at any time before the service of that subtask has begun. We assume that subtasks in service cannot be preempted. A key dynamic optimisation that benefits both metrics of interest is to remove delays on any subtask with a sibling that has already completed service. This paper incorporates such a policy into existing methods for computing optimal subtask delays in split-merge and fork-join systems. In the context of two case studies, we show that doing so affects the optimal delays computed, and leads to improved subtask dispersion values when compared with existing techniques. Indeed, in some cases, it turns out to be beneficial to initially postpone the processing of non-bottleneck subtasks until the bottleneck subtask has completed service.

STRATEGIC DYNAMIC JOCKEYING BETWEEN TWO PARALLEL QUEUES

Consider a two-station, heterogeneous parallel queueing system in which each station operates as an independent M/M/1 queue with its own infinite-capacity buffer. The input to the system is a Poisson process that splits among the two stations according to a Bernoulli splitting mechanism. However, upon arrival, a strategic customer initially joins one of the queues selectively and decides at subsequent arrival and departure epochs whether to jockey (or switch queues) with the aim of reducing her own sojourn time. There is a holding cost per unit time, and jockeying incurs a fixed non-negative cost while placing the customer at the end of the other queue. We examine individually optimal joining and jockeying policies that minimize the strategic customer's total expected discounted (or undiscounted) costs over finite and infinite time horizons. The main results reveal that, if the strategic customer is in station 1 with ℓ customers in front of her, and q1 and q2 customers in stations 1 and 2, respectively (excluding herself), then the incentive to jockey increases as either ℓ increases or q2 decreases. Numerical examples reveal that it may not be optimal to join, and/or jockey to, the station with the shortest queue or the fastest server.

An Explicit Solution for a Series and Parallel Queue with Retrial, Losses, and Bernoulli Schedule

This paper deals with the series and parallel queueing system in which there are two servers whose service time follow two exponential distributions. Each arriving customer either enters into the tandem service with probability or joins the service of the single server with complementary probability. We assume that the customers of arriving at the first server who find the first server is busy join an orbit and retry to enter the server after some time and of arriving at the second server who find the second server is busy are lost. For this model, we obtain the explicit expressions of the joint stationary distribution between the number of customers in the orbit and the states of the servers.

Two Parallel Queues with Infinite Servers and Join the Shortest Queue Discipline

We study the stationary distribution of a system of two parallel M/M/∞ queues managed by the Join the Shortest Queue load balancing policy; one motivation for characterizing the efficiency of that policy is its potential application to resource allocation issues in cloud computing. For the general system with distinct service rates, we first show that the tail of each marginal queue length distribution exhibits a much faster decay than that of a Poisson distribution. Second, the determination of the joint stationary distribution is shown to reduce to the resolution of a pair of linear integral equations. In the case when service rates are identical (“symmetric case”), that pair of integral equations simplifies to a single Fredholm integral equation of the first kind whose solution is explicitly given in terms of Legendre polynomials; this enables us to entirely determine the stationary distribution of the system. We provide, in particular, asymptotics for the second moment and the tail of the queue length distribution.

A coupled processor model with simultaneous arrivals and ordered service requirements

We study a coupled processor model with simultaneous arrivals. Under the assumption of ordered input, the Laplace–Stieltjes transform of the joint stationary amount of work in the system can be explicitly calculated by relating it to the amount of work in a parallel queueing system without coupling. This relation is first exploited for the system with two coupled queues. The method is then extended to higher dimensions.

Can Server Behavior and Service System Design Outweigh the Benefits of Pooling?

It is widely accepted that multi-server single-queue (SQ) systems outperform multi-server parallel-queue (PQ) systems due to the pooling effect. However, humans are not machines, and the assumption that servers perform identically regardless of the queueing system, is not necessarily true in practice. In this paper, we investigate the impact of two human server behaviors - social loafing and workload-dependent speedup - and one physical factor - walking time - on the performance of the SQ and PQ systems. We develop theoretical models and comparatively analyze them with respect to three queueing performance measures: expected wait time, pre-service time, and total system time. We propose functional forms for social loafing and speedup effects and evaluate the performance of the models numerically. We find that social loafing worsens the performance of the SQ system. Server speedup improves performance of the PQ system. Moreover, this improvement is greater than the direct impact of a comparable average increase in the server's speed. We derive thresholds beyond which PQ outperform SQ systems for all three performance measures. Our findings provide managers with guidance in the design of their queueing operations and serve as a foundation for future research on service systems with behavioral elements.

Performance Analysis of A Centralized Burst-Mode Traffic Shaping for Distributed Parallel Queues

Accurate control of the data flows in distributed queues is a critical issue in a cloud service or a scalable network system. This paper proposes a centralized traffic control mechanism using burst-mode scheduling. To minimize the amount of control signaling bandwidth and to achieve a high level of control accuracy, transmissions of distributed packets are controlled by a centralized traffic shaping method using a virtual queue that is configured by reported packet information without aggregation of the packets. The performance critical metrics of the proposed mechanism are analyzed using a queueing model and the derivation of the feasible batch service size in the given delay constraints is also achieved. The accuracy of the proposed scheme is validated by numerical analysis and practical tests.

Necessary condition for null controllability in many-server heavy traffic

Throughput sub-optimality (TSO), introduced in Atar and Shaikhet [Ann. Appl. Probab. 19 (2009) 521-555] for static fluid models of parallel queueing networks, corresponds to the existence of a resource allocation, under which the total service rate becomes greater than the total arrival rate. As shown in Atar, Mandelbaum and Shaikhet [Ann. Appl. Probab. 16 (2006) 1764-1804] and Atar and Shaikhet (2009), in the many server Halfin-Whitt regime, TSO implies null controllability (NC), the existence of a routing policy under which, for every finite $T$, the measure of the set of times prior to $T$, at which at least one customer is in the buffer, converges to zero in probability at the scaling limit. The present paper investigates the question whether the converse relation is also true and TSO is both sufficient and necessary for the NC behavior. In what follows we do get the affirmation for systems with either two customer classes (and possibly more service pools) or vice-versa and state a condition on the underlying static fluid model that allows the extension of the result to general structures.

Strategic Arrivals into Queueing Networks: The Network Concert Queueing Game

Queueing networks models typically assume that the arrival process is exogenous and unaffected by admission control, scheduling policies, etc. In many situations, however, users choose the time of their arrival strategically, taking delay and other metrics into account. In this paper, we develop a framework to study such strategic arrivals into queueing networks. We study the population game wherein users strategically choose when to arrive at a parallel queueing network and upon arrival, which of the queues to join. The queues start service at given times, which can potentially be different. We characterize the (strategic) arrival process at each of the queues and the price of anarchy of the ensuing strategic arrival game. We then extend the analysis to multiple populations of users, each with a different cost metric. The equilibrium arrival profile and price of anarchy are derived. Finally, we extend this to general feedforward network architectures by modeling the arrival timing game as a two-stage extensive form game. We prove the existence and essential uniqueness of equilibria. We also study more specific network topologies, like tandem and trellis networks, and we derive the equilibrium arrival and routing profiles. We show that there exists an equivalent parallel queueing network that has the same equilibrium arrival profile. Thus, the price of anarchy of the arrival game is then implied by that of the parallel queueing network.

On minimizing delay with probabilistic splitting of traffic flow in heterogeneous wireless networks

In the paper, we propose a framework to investigate how to effectively perform traffic flow splitting in heterogeneous wireless networks from a queue point. The average packet delay in heterogeneous wireless networks is derived in a probabilistic manner. The basic idea can be understood via treating the integrated heterogeneous wireless networks as different coupled and parallel queuing systems. The integrated network performance can approach that of one queue with maximal the multiplexing gain. For the purpose of illustrating the effectively of our proposed model, the Cellular/WLAN interworking is exploited. To minimize the average delay, a heuristic search algorithm is used to get the optimal probability of splitting traffic flow. Further, a Markov process is applied to evaluate the performance of the proposed scheme and compare with that of selecting the best network to access in terms of packet mean delay and blocking probability. Numerical results illustrate our proposed framework is effective and the flow splitting transmission can obtain more performance gain in heterogeneous wireless networks.

Heavy-traffic asymptotics for networks of parallel queues with Markov-modulated service speeds

We study a network of parallel single-server queues, where the speeds of the servers are varying over time and governed by a single continuous-time Markov chain. We obtain heavy-traffic limits for the distributions of the joint workload, waiting-time and queue length processes. We do so by using a functional central limit theorem approach, which requires the interchange of steady-state and heavy-traffic limits. The marginals of these limiting distributions are shown to be exponential with rates that can be computed by matrix-analytic methods. Moreover, we show how to numerically compute the joint distributions, by viewing the limit processes as multi-dimensional semi-martingale reflected Brownian motions in the non-negative orthant.

Queues and Risk Models with Simultaneous Arrivals

We focus on a particular connection between queueing and risk models in a multidimensional setting. We first consider the joint workload process in a queueing model with parallel queues and simultaneous arrivals at the queues. For the case that the service times are ordered (from largest in the first queue to smallest in the last queue), we obtain the Laplace-Stieltjes transform of the joint stationary workload distribution. Using a multivariate duality argument between queueing and risk models, this also gives the Laplace transform of the survival probability of all books in a multivariate risk model with simultaneous claim arrivals and the same ordering between claim sizes. Other features of the paper include a stochastic decomposition result for the workload vector, and an outline of how the two-dimensional risk model with a general two-dimensional claim size distribution (hence, without ordering of claim sizes) is related to a known Riemann boundary-value problem.

Queues and Risk Models with Simultaneous Arrivals

We focus on a particular connection between queueing and risk models in a multidimensional setting. We first consider the joint workload process in a queueing model with parallel queues and simultaneous arrivals at the queues. For the case that the service times are ordered (from largest in the first queue to smallest in the last queue), we obtain the Laplace-Stieltjes transform of the joint stationary workload distribution. Using a multivariate duality argument between queueing and risk models, this also gives the Laplace transform of the survival probability of all books in a multivariate risk model with simultaneous claim arrivals and the same ordering between claim sizes. Other features of the paper include a stochastic decomposition result for the workload vector, and an outline of how the two-dimensional risk model with a general two-dimensional claim size distribution (hence, without ordering of claim sizes) is related to a known Riemann boundary-value problem.

Design of TVM Based on Parallel Virtual Queuing Theory

From what has been discussed above, the design of parallel virtual queuing, means that the passenger chooses the trains, the time, departures and terminals, seat types on TVM by blackening a ticket purchasing card. All the job off the TVM can be done by all the passengers all at once, which is equal to every passenger is being served at the counter (TVM), and thus the queuing time are greatly reduced in peak hours. Therefore, the queuing is equal to “Parallel Virtual Queuing”, and the equivalent queuing model is M/M/∞/∞/∞/FCFS.

The Bluetooth Module Design of TVM Trans-Province Express Train

By installation of GC-02 Bluetooth module,inter-provincial express train TVM is relatively mature at present, so that passengers can, in accordance with the agreement, choose the available time, trains, departures and terminals and seats with their own Bluetooth-enabled phone. Hence, it will greatly reduce the time on the vending machine. In peak hours, passengers will experience “parallel virtual queuing”,and benefit from the less queuing time and fast tickets buying.