System Of Parallel Queues Research Articles

Asymptotically optimal routing of a many-server parallel queueing system with long-run average criterion

This paper considers a parallel queueing system with multiple stations, each of which contains many statistically identical servers and has a dedicated queue. Upon each customer arrival, the system manager must decide to which station the customer should be routed, with the objective of minimizing the system’s long-run average delay cost. One feature of this paper is that a customer’s delay cost depends not only on his/her delay, but also on the routed station. Considering this heterogeneity across stations, we propose a routing policy, which can be regarded as an extension of the MED–FSF policy. Under this policy, any arriving customer will be routed to: (i) the station with the minimum value, which depends on the station’s expected delay and the station index when servers in all stations are fully occupied; or otherwise (ii) the station with a fastest idle server. Using asymptotic analysis, we derive diffusion limits of queue-length processes and their stationary distributions under the proposed policy in the Halfin–Whitt regime. Combined with an asymptotic lower bound result for the long-run average delay cost, we show that the proposed routing policy is asymptotically optimal under the considered objective. Finally, we provide numerical experiments to validate the accuracy of our diffusion approximation, and we compare the performance metrics under the proposed policy with those under other commonly used routing policies.

Admission and routing control of multiple queues with multiple types of customers

We study the routing and admission control problem in a parallel queueing system with heterogeneous servers serving multiple types of customers. The system makes admission decision regarding whether to admit a customer upon arrival as well as routing decision of the queue to which the admitted customer is assigned. The objective is to maximize the expected profit, which includes customer-dependent revenues and holding cost and server-dependent cost. We first characterize the structure of the optimal policy for the case with two servers and two types of customers that have the same holding cost. We show that the optimal admission and routing policy has a complex non-monotone structure; however, we show that this non-monotone structure is the result of overlapping of three pairwise dominant policies that have a monotone structure. Utilizing the above structure, we propose three heuristics for the general case of multiple servers and multiple types of customers. Through a numerical study, we demonstrate the effectiveness of our heuristics, and provide conditions under which each heuristic performs well. Lastly, we provide insights on the effect of holding cost on customer rejection and the effect of fixed production cost on capacity allocation.

Delay-Join the Shortest Queue Routing for a Parallel Queueing System with Removable Servers

We introduce a new class of policies called “delay-join the shortest queue (delay-JSQ)” for use in parallel processing networks with removable servers. When jobs arrive to the system while all servers are on, jobs should be routed to the shortest queue. However, when servers are off, they take a random time to turn back on, which we allow to occur only when the number of jobs in each of the nonempty queues exceeds a fixed threshold. This new class of policies balances the load among all servers that are currently on and balances the capacity by keeping servers off until they are needed. A detailed numerical study shows that at moderate loads (where server farms and increasingly manufacturing facilities operate), delay-JSQ outperforms JSQ by up to 80%. In addition, it does so without precise knowledge of the input parameters and even when the input process is nonstationary.

Verification of the Effect on a Reduction “Shikake” of Queue Length in Fork-less Shaped Queueing Systems Using a Multi Agent System

This paper concentrates on parallel queueing systems, called fork-less shaped queueing systems, taking a checkout area of the supermarket as an example. In fork-less shaped queueing systems, customers choose the server with their criterion. Sometimes queue length become imbalanced due to customers freely choices. Imbalanced queue has negative effects on both store clerk and customer, so it is necessary to reduce the variation of queue length. “Shikake” is a method of changing human behavior. It can be possible to reduce the variation of queue length. In this paper, we suggest some reduction Shikake and verify the effects of them using a multi-agent simulation.

Optimal Open-Loop Routing and Threshold-Based Allocation in TWO Parallel QUEUEING Systems with Heterogeneous Servers

In this paper, we study the problem of optimal routing for the pair of two-server heterogeneous queues operating in parallel and subsequent optimal allocation of customers between the servers in each queue. Heterogeneity implies different servers in terms of speed of service. An open-loop control assumes the static resource allocation when a router has no information about the state of the system. We discuss here the algorithm to calculate the optimal routing policy based on specially constructed Markov-modulated Poisson processes. As an alternative static policy, we consider an optimal Bernoulli splitting which prescribes the optimal allocation probabilities. Then, we show that the optimal allocation policy between the servers within each queue is of threshold type with threshold levels depending on the queue length and phase of an arrival process. This dependence can be neglected by using a heuristic threshold policy. A number of illustrative examples show interesting properties of the systems operating under the introduced policies and their performance characteristics.

Synchronized Lévy queues

AbstractWe consider a multivariate Lévy process where the first coordinate is a Lévy process with no negative jumps which is not a subordinator and the others are non-decreasing. We determine the Laplace–Stieltjes transform of the steady-state buffer content vector of an associated system of parallel queues. The special structure of this transform allows us to rewrite it as a product of joint Laplace–Stieltjes transforms. We are thus able to interpret the buffer content vector as a sum of independent random vectors.

Open Problem—Load Balancing Using Delayed Information

Load-balancing algorithms are important for efficiently routing jobs in systems of parallel queues; however, there has been relatively little attention devoted to developing algorithms in the presence of messaging and/or routing delays. Given a system of parallel queues with infinite capacity buffers; first-come, first-serve service discipline; and a single stream of incoming jobs that are routed by a dispatcher upon arrival, it has been shown that join the shortest queue (JSQ) satisfies certain optimality properties, including minimizing mean wait time when the job sizes are exponentially distributed ( Winston 1977 , Ephremides et al. 1980 ) and state space collapse of the queue lengths under heavy traffic scaling for general service distributions ( Reiman 1984 ). However, implementation of JSQ uses up-to-date information about the state of the buffers, which requires instantaneous exchange of multiple messages between the dispatcher and the queues. This challenge has led to the development of efficient algorithms that require fewer messages, including join the shortest of d queues (JSQ(d); Vvedenskaya et al. 1996 , Mitzenmacher 2001 ), join the idle queue (JIQ; Badonnel and Burgess 2008 , Lu et al. 2011 ), and persistent idle load distribution (PILD; Atar et al. 2019a , b ); see van der Boor et al. (2018) for a survey of some recent results in the many-server limit. Although these algorithms do not use full information about the system, they still use up-to-date information about the state of some of the queues. Because of the physical separation between the dispatcher and the queues, processing effects, or periodic updates from the queues, the dispatcher may have access only to information about the delayed states of the queues. (There may also be routing delays because of the time it takes a job to travel between the dispatcher and its assigned server, for which these algorithms also do not account.) In such settings, it has been shown that JSQ can perform quite poorly and lead to sustained oscillations in queue lengths ( Mitzenmacher 2000 ).

An approximation method for 2-chain flexible queues with preemptive priority

Chaining configurations (e.g. 2-chain) have been widely applied in service systems to improve responsiveness to customer demand. In this paper, we propose an approximation method for the analysis of symmetric parallel queueing systems with 2-chain configuration and preemptive priority. In the queueing system of interest, the arrival processes of different types of customers follow independent Poisson processes with identical arrival rate λ, and service times of different types of customers are exponentially distributed with identical service rate μ. Each server serves its primary customers with preemptive priority. When there are no primary customers in the system, the server serves secondary customers. We first derive the system stability condition. Then, we analyse the system's various performance measures. Our approach relies on the matrix-analytic method coupled with the idle probability of each server, which is proved to be . The average queue length of each queue and the rate that each server serves its secondary customers (i.e. the helping rate) can be estimated using our proposed method. Finally, we conduct numerical studies to demonstrate the accuracy of our proposed approximation method.

Design heuristic for parallel many server systems

We study a parallel queueing system with multiple types of servers and customers. A bipartite graph describes which pairs of customer-server types are compatible. We consider the service policy that always assigns servers to the first, longest waiting compatible customer, and that always assigns customers to the longest idle compatible server if on arrival multiple compatible servers are available. For a general renewal stream of arriving customers, general service time distributions that depend both on customer and on server types, and general customer patience distributions, the behavior of such systems is very complicated. Key quantities for their performance are the matching rates, the fraction of services for each pair of compatible customer-server. Calculation of these matching rates in general is intractable, it depends on the entire shape of service time distributions. We suggest through a heuristic argument that if the number of servers becomes large, the matching rates are well approximated by matching rates calculated from the tractable bipartite infinite matching model. We present simulation evidence to support this heuristic argument, and show how this can be used to design systems with desired performance requirements.

On the stability of a class of non-monotonic systems of parallel queues

We investigate, under general stationary e`rgodic assumptions, the stability of systems of S parallel queues in which any incoming customer joins the queue of the server having the p + 1-th shortest workload (p

Guaranteeing performance based on time-stability for energy-efficient data centers

ABSTRACTWe consider a system of multiple parallel single-server queues where servers are heterogeneous with resources of different capacities and can be powered on or off while running at different speeds when they are powered on. In addition, we assume that application requests are heterogeneous with different workload distributions and resource requirements and the arrival, rates of request are time-varying. Managing such a heterogeneous, transient, and non-stationary system is a tremendous challenge. We take an unconventional approach, in that we force the queue lengths in each powered-on server to be time-stable (i.e., stationary). It allows the operators to guarantee performance and effectively monitor the system. We formulate a mixed-integer program to minimize energy costs while satisfying time-stability. Simulation results show that our suggested approach can stabilize queue length distributions and provide probabilistic performance guarantees on waiting times.

On Round-Robin routing with FCFS and LCFS scheduling

We study the Round-Robin (RR) routing to a system of parallel queues, which serve jobs according to FCFS or preemptive LCFS scheduling disciplines. The cost structure comprises two components: a service fee and a queueing delay related component. With Poisson arrivals, the inter-arrival time to each queue obeys the Erlang distribution. This allows us to study the mean and transient behavior of the queues separately. The service fee is independent of the scheduling and queueing, and we obtain the corresponding mean cost rate and value function in closed forms. With respect to queueing delay, we first derive integral expressions enabling efficient computation of the corresponding size-aware value functions. By decomposition, these yield also the value function for the whole system of m parallel queues fed by RR. Given the value function, one can carry out the first policy iteration step with arbitrary holding cost rates (e.g., delay, slowdown, etc.) yielding efficient size-, cost- and state-aware policies. Moreover, the mean waiting and sojourn times in the corresponding systems get resolved at the same time. The results are demonstrated in the numerical examples, where we compute near optimal job routing policies for sample systems.

Maximum weight matching with hysteresis in overloaded queues with setups

We consider a system of parallel queues where arriving service tasks are buffered, according to type. Available service resources are dynamically configured and allocated to the queues to process the tasks. At each point in time, a scheduler chooses a service configuration across the queues, in response to queue backlogs. Switching from one service configuration to another incurs a setup time, during which idling occurs and service bandwidth is lost. Such setup times are inherent in manufacturing and computer systems. Frequent switchings can significantly compromise the service capacity of such systems. A maximum weight matching (MWM) scheduler, which is known to maximize throughput in the absence of setups, can easily become unstable with setups, even under low load. To remedy this problem, we propose a new MWM-H scheduler which utilizes a controller introduced hysteresis and achieves maximum throughput even with setups, without requiring knowledge of arrival rates and average traffic loads. During prolonged traffic bursts, the queues may become overloaded and the issue becomes how to reasonably distribute the growing backlog under MWM-H. It is shown that by appropriately selecting the MWM-H parameters, one can control the backlog among the individual queues in order to achieve a desired balance.

Randomized longest-queue-first scheduling for large-scale buffered systems

We develop diffusion approximations for parallel-queueing systems with the randomized longest-queue-first scheduling (LQF) algorithm by establishing new mean-field limit theorems as the number of buffers n → ∞. We achieve this by allowing the number of sampled buffers d = d(n) to depend on the number of buffers n, which yields an asymptotic 'decoupling' of the queue length processes. We show through simulation experiments that the resulting approximation is accurate even for moderate values of n and d(n). To the best of the authors' knowledge, this is the first derivation of diffusion approximations for a queueing system in the large-buffer mean-field regime. Another noteworthy feature of our scaling idea is that the randomized LQF algorithm emulates the LQF algorithm, yet is computationally more attractive. The analysis of the system performance as a function of d(n) is facilitated by the multi-scale nature in our limit theorems: the various processes we study have different space scalings. This allows us to show the trade-off between performance and complexity of the randomized LQF scheduling algorithm.

Randomized longest-queue-first scheduling for large-scale buffered systems

We develop diffusion approximations for parallel-queueing systems with the randomized longest-queue-first scheduling (LQF) algorithm by establishing new mean-field limit theorems as the number of buffers n → ∞. We achieve this by allowing the number of sampled buffers d = d(n) to depend on the number of buffers n, which yields an asymptotic 'decoupling' of the queue length processes. We show through simulation experiments that the resulting approximation is accurate even for moderate values of n and d(n). To the best of the authors' knowledge, this is the first derivation of diffusion approximations for a queueing system in the large-buffer mean-field regime. Another noteworthy feature of our scaling idea is that the randomized LQF algorithm emulates the LQF algorithm, yet is computationally more attractive. The analysis of the system performance as a function of d(n) is facilitated by the multi-scale nature in our limit theorems: the various processes we study have different space scalings. This allows us to show the trade-off between performance and complexity of the randomized LQF scheduling algorithm.

Dynamic Subtask Dispersion Reduction in Heterogeneous Parallel Queueing Systems

Fork-join and split-merge queueing systems are mathematical abstractions of parallel task processing systems in which entering tasks are split into N subtasks which are served by a set of heterogeneous servers. The original task is considered completed once all the subtasks associated with it have been serviced. Performance of split-merge and fork-join systems are often quantified with respect to two metrics: task response time and subtask dispersion. Recent research effort has been focused on ways to reduce subtask dispersion, or the product of task response time and subtask dispersion, by applying delays to selected subtasks. Such delays may be pre-computed statically, or varied dynamically. Dynamic in our context refers to the ability to vary the delay applied to a subtask according to the state of the system, at any time before the service of that subtask has begun. We assume that subtasks in service cannot be preempted. A key dynamic optimisation that benefits both metrics of interest is to remove delays on any subtask with a sibling that has already completed service. This paper incorporates such a policy into existing methods for computing optimal subtask delays in split-merge and fork-join systems. In the context of two case studies, we show that doing so affects the optimal delays computed, and leads to improved subtask dispersion values when compared with existing techniques. Indeed, in some cases, it turns out to be beneficial to initially postpone the processing of non-bottleneck subtasks until the bottleneck subtask has completed service.

STRATEGIC DYNAMIC JOCKEYING BETWEEN TWO PARALLEL QUEUES

Consider a two-station, heterogeneous parallel queueing system in which each station operates as an independent M/M/1 queue with its own infinite-capacity buffer. The input to the system is a Poisson process that splits among the two stations according to a Bernoulli splitting mechanism. However, upon arrival, a strategic customer initially joins one of the queues selectively and decides at subsequent arrival and departure epochs whether to jockey (or switch queues) with the aim of reducing her own sojourn time. There is a holding cost per unit time, and jockeying incurs a fixed non-negative cost while placing the customer at the end of the other queue. We examine individually optimal joining and jockeying policies that minimize the strategic customer's total expected discounted (or undiscounted) costs over finite and infinite time horizons. The main results reveal that, if the strategic customer is in station 1 with ℓ customers in front of her, and q1 and q2 customers in stations 1 and 2, respectively (excluding herself), then the incentive to jockey increases as either ℓ increases or q2 decreases. Numerical examples reveal that it may not be optimal to join, and/or jockey to, the station with the shortest queue or the fastest server.

An Explicit Solution for a Series and Parallel Queue with Retrial, Losses, and Bernoulli Schedule

This paper deals with the series and parallel queueing system in which there are two servers whose service time follow two exponential distributions. Each arriving customer either enters into the tandem service with probability or joins the service of the single server with complementary probability. We assume that the customers of arriving at the first server who find the first server is busy join an orbit and retry to enter the server after some time and of arriving at the second server who find the second server is busy are lost. For this model, we obtain the explicit expressions of the joint stationary distribution between the number of customers in the orbit and the states of the servers.

A coupled processor model with simultaneous arrivals and ordered service requirements

We study a coupled processor model with simultaneous arrivals. Under the assumption of ordered input, the Laplace–Stieltjes transform of the joint stationary amount of work in the system can be explicitly calculated by relating it to the amount of work in a parallel queueing system without coupling. This relation is first exploited for the system with two coupled queues. The method is then extended to higher dimensions.

Queueing Dynamics and State Space Collapse in Fragmented Limit Order Book Markets

In modern equity markets, participants have a choice of many exchanges at which to trade. Exchanges typically operate as electronic limit order books operating under a “price-time” priority rule and, in turn, can be modeled as multi-class FIFO queueing systems. A market with multiple exchanges can be thought as a decentralized, parallel queueing system. Heterogeneous traders that submit limit orders select the exchange, i.e., the queue, in which to place their orders by trading off delays until their order may fill against financial considerations. These limit orders can be thought as jobs waiting for service. Simultaneously, traders that submit market orders select the exchange, i.e., the queue, to direct their order. These market orders trigger instantaneous service completions of queued limit orders. In this way, the “server” is the aggregation of self-interested, atomistic traders submitting market orders. Taking into account the effect of investors’ order routing decisions across exchanges, we find that the equilibrium of this decentralized market exhibits a state space collapse property, whereby: (a) the queue lengths at different exchanges are coupled in an intuitive manner; (b) the behavior of the market is captured through a one-dimensional process that can be viewed as a weighted aggregate queue length across all exchanges; and (c) the behavior at each exchange can be inferred via a mapping of the aggregated market depth process that takes into account the heterogeneous trader characteristics. This predicted dimension reduction is the result of high-frequency order routing decisions that essentially couple the dynamics across exchanges. We derive a characterization of the market equilibrium and the associated aggregated depth process. Analyzing a TAQ dataset for a sample of stocks over a one month period, we find empirical support for the predicted state space collapse.