Multi-server Queueing Research Articles

Modeling and Analysis of Multichannel Drive-Thru Internet Systems and Performance Improvement

Recently, the interest regarding Drive-thru Internet systems has been rapidly arising in industrial and academic fields in view of the widespread adoption of IEEE 802.11 networks and its great potential to provide cost-effective Internet access. Drive-thru Internet systems are multiple-access wireless networks in which users in moving vehicles request/receive services such as digital map update and MP3 download to/from a Road Side Unit (RSU) as the vehicles pass through the coverage range of the RSU. For the purpose of efficiently supporting various services, Wireless Access in Vehicular Environment (WAVE), which is the standard for VANETs communications, specifies multichannel utilization, where the overall bandwidth is subdivided into seven channels, namely, one Control Channel (CCH) and six Service Channels (SCHs). However, originally designed for quasi-static single-channel-based small-scale indoor applications, the performance of IEEE 802.11 in the outdoor vehicular environment, where a large number of fast-moving vehicles simultaneously contend for channel access in the multichannel environment, is still unclear. In this article, a unified analytical framework is established to study the performance of multichannel Drive-thru Internet systems. Specifically, taking account of channel access contention of vehicles and power reception probability of an RSU, the message arrival rate at the RSU on the uplink channel (i.e., CCH) is derived. Then, a multiserver queueing model, which plays the role of a bridge connecting the uplink and downlink (i.e., SCHs) communications, is developed for the purpose of accurately capturing the dynamics of the multichannel environment. Based on the developed framework, it can be noticed that as the intensity of channel contention increases, the saturated throughput of SCHs decreases rapidly, and the system becomes unstable due to the reason that vehicles have to wait for a very large amount of time to receive the requested service messages, or even worse, cannot receive the messages before leaving the coverage of RSU. In order to keep the throughput at the maximum level regardless of the channel contention intensity while maintaining the system stability, we propose a centralized coordination mechanism. Simulation experiments are carried out to validate the accuracy of the developed analytical framework and the effectiveness of the proposed centralized coordination mechanism.

Competitive queueing systems with comparative rating dependent arrivals

We consider the joint operation of two multi-server queueing systems. Both systems provide the same type of service and compete for customers. It is assumed that the first system is controlled by ourselves while the second one is controlled by our competitor. The arriving customers are shared between the systems depending on the comparative rating of our system. The stationary behavior of the considered model under the fixed parameters of both systems and the fixed mechanism of calculation of the rating is described by a continuous-time multi-dimensional Markov chain. The stationary distribution of this Markov chain is computed. Expressions for the key performance measures of the systems are derived. Obtained results provide an opportunity to analyse possible consequences of various managerial actions aiming to maximize the profit of our system. A numerical experiment illustrates the application of the results for making a decision about the rationality of establishing and maintaining a service system while an alternative system providing the same type of service already exists.

A Profit Maximization Scheme in Cloud Computing With Deadline Constraints

Cloud computing has attracting more and more attention for its flexibility and economic benefits. To maintain the supply-demand relationship among different participants in cloud computing environment, the exchange of value is the inner drive. From the perspective of cloud service provider, its primary concern is to earn the profit, which can be obtained by finishing the tasks published from customers. In this paper, we consider each task consists of numbers of sub-tasks in the logical order, each sub-task corresponds to a type of service requests, which can be served in unique multi-server system. On this basis, we propose a profit maximization problem in the multistage multi-server queue systems, in which customers are served at more than one stage, arranged in a series structure. Moreover, a deadline constraint is taken into consideration, which demonstrates the maximum tolerance degree that the customers can wait. Therefore, how to configure the parameters in multistage multi-server queue systems to maximize profit on the premise of reducing the waiting times of customers is a critical issue for cloud service provider. To address this problem, we first discuss the probability distribution function of the waiting time for single multi-server system and multistage multi-server queue systems respectively, and then propose a profit maximization model under the deadline constraint. Due to the complexity of this model, the analytical solution can hardly be obtained, we study a heuristic method to search for the optimal solution. At last, a series of numerical simulations are implemented to describe the performance of the proposed profit maximization scheme, the results show that not only the profit can be maximized, but also the waiting time of customers have been reduced effectively.

Optimum cost analysis for an Geo/Geo/c/N feedback queue under synchronous working vacations and impatient customers

This paper concerns the cost optimisation analysis of a discrete-time finite-capacity multiserver queueing system with Bernoulli feedback, synchronous multiple and single working vacations, balking, and reneging during both busy and working vacation periods. A reneged customer can be retained in the system by employing certain persuasive mechanism for completion of service. Using recursive method, the explicit expressions for the stationary state probabilities are obtained. Various system performance measures are presented. Further, a cost model is formulated. Then, the optimization of the model is carried out using quadratic fit search method (QFSM). Finally, the impact of various system parameters on the performance measures of the queueing system is shown numerically.

Capacity choice game in a multiserver queue: Existence of a Nash equilibrium

AbstractIn many congestion‐prone services, front‐line employees have discretion over the rate at which they serve customers. To evaluate the impact of queue pooling on their decisions, we model the situation as a two‐server, single‐queue symmetric capacity choice game. Gopalakrishnan et al. (2016) characterize the existence of a Nash equilibrium in this game under a requirement on the servers' capacity cost functions, that is, where servers have limited discretion. Without that requirement and when servers are free to choose any service rate, the servers' cost function is ill‐behaved and standard tools for establishing the existence of an equilibrium cannot be applied. We consider a general power capacity cost function with no restriction on the servers' choice of capacity, and rely on a lesser‐known result, namely Tarski's intersection theorem, to establish the existence of a symmetric pure‐strategy Nash equilibrium. Comparing settings where queue stability is enforceable versus not, we show that there always exists a Nash equilibrium in the former case, unlike in the latter, and that some of the capacity choices that are equilibria in the former case are no longer equilibria in the latter. Our analysis highlights the criticality of the enforceability of system stability on equilibrium outcomes.

Multi-server queueing system with reserve servers

In this paper, we investigate a multi-server queueing system with an unlimited buffer, which can be used in the design of energy consumption schemes and as a mathematical model of unreliable real stochastic systems. Customers arrive to the system in a batch Markovian arrival process, the service times are distributed according to the phase law. If the service time of the customer by the server exceeds a certain random value distributed according to the phase law, this server receives assistance from the reserve server from a finite set of reserve servers. In the paper, we calculate the stationary distribution and performance characteristics of the system.

Comparison of fluid approximations for service systems with state-dependent service rates and return probabilities

We compare two models of a multi-server queueing system with state-dependent service rates and return probabilities. In both models, upon completing service, customers are delayed prior to possibly returning to service. In one model, the determination of whether a customer will return occurs immediately upon service completion, at the beginning of the delay. In the other, that determination is made at the end of the delay, capturing the idea that it takes time for the customer’s condition and needs to evolve or assess, before it becomes known whether a return to service is needed. Our comparison focuses on fluid approximations of the two models. The fluid approximation for the first model, which has been studied previously, consists of a system of two ordinary differential equations. The fluid approximation for the second model, which is new, consists of a delay differential equation. We find that the two fluid approximations have the same set of equilibrium points, but their transient behavior can differ markedly. Both fluid approximations can exhibit bistability for certain parameter values. We use discrete event simulation to illustrate the extent to which the findings from the fluid approximations carry over to the underlying stochastic models.

Exact sampling for some multi-dimensional queueing models with renewal input

AbstractUsing a result of Blanchet and Wallwater (2015) for exactly simulating the maximum of a negative drift random walk queue endowed with independent and identically distributed (i.i.d.) increments, we extend it to a multi-dimensional setting and then we give a new algorithm for simulating exactly the stationary distribution of a first-in–first-out (FIFO) multi-server queue in which the arrival process is a general renewal process and the service times are i.i.d.: the FIFO GI/GI/c queue with $ 2 \leq c \lt \infty$ . Our method utilizes dominated coupling from the past (DCFP) as well as the random assignment (RA) discipline, and complements the earlier work in which Poisson arrivals were assumed, such as the recent work of Connor and Kendall (2015). We also consider the models in continuous time, and show that with mild further assumptions, the exact simulation of those stationary distributions can also be achieved. We also give, using our FIFO algorithm, a new exact simulation algorithm for the stationary distribution of the infinite server case, the GI/GI/ $\infty$ model. Finally, we even show how to handle fork–join queues, in which each arriving customer brings c jobs, one for each server.

Control charts for traffic intensity monitoring of Markovian multiserver queues

AbstractA number of recent research studies have applied queueing theory as an approximate modeling tool to mathematically describe industrial systems, which include manufacturing, distribution, and service, for instance. Among the main observable characteristics in queues, the number of users in the system can be controlled to keep waiting times as minimal as possible. The design of efficient control charts is an attempt to monitor and control such systems. Control charts are proposed to monitor infinite queues with Markovian arrivals, exponential service times, and s identical parallel servers. The proposed charts monitor traffic intensities, which are the ratio between the arrival rate and the service rate, estimated through the number of users in the queueing system at random epochs. The effectiveness and efficiency of the proposed approaches in terms of the average run lengths are established by a comprehensive set of Monte Carlo simulations.

On the Analysis of Unreliable Markovian Multiserver Queue with Retrials and Impatience

This paper concerns an approximate analysis of a Markovian multiserver infinite source retrial queuing with impatience, in which all the servers are subject to breakdown and repairs. Customer who find the total number of busy and failed servers equal to $s$,i.e, he is given to choice to enter a retrial orbit for an random amount of time before attempting to reccess an available server or enter the queue of size $q$. Customer waiting in the queue start being served as an idle or repaired server assigned to them, they can also leave the queue and enter orbit due to impatience. Customers whose service is interrupted by a failure may have the option of leaving the system entirely or returning to the orbit to repeat or resume service. We assume that each server has its own dedicated repair person, and repairs begin immediately following a failure and all process are assumed to be mutually independent. The simultaneous effect of customer balking, impatience and retrials is analyzed. We try to approximate the steady-state joint distribution of the number of customers in orbit and the number of customers in the service area using a phase-merging Algorithm.

A novel scheduling algorithm to improve SUPT for multi-queue multi-server system

This study improves the Quality of Experience (QoE) of the multi-queue multi-server queueing system by solving the scheduling problem. The QoE is evaluated by a novel indicator named system user-perceived throughput (SUPT). According to the property of the traffic, the stochastic optimization problem for SUPT can be transformed into utility maximization under the constraint of queue stability. We then propose a drift-plus-penalty scheduling algorithm named max modified weight (MMW) to balance delay and utility. A Nike function for queue length replaces the queue length as the weight. Furthermore, we prove the stability of the queues based on the Foster–Lyapunov theorem and analyze the delay boundary under the proposed MMW scheduling algorithm. Finally, compared with several classical scheduling policies, the effectiveness of the MMW is verified by evaluating the average system throughput, SUPT, the average system backlog, and user-perceived throughput of the queues in three different scenarios. The simulation results show MMW policy achieves more efficient trade-off between SUPT and system delay, and is capable of maintaining system stability as max weight regardless of the system load.

Late-rejection, a strategy to perform an overflow policy

Motivated by overflow policies implemented in service systems, we consider a multi-server queue with customers’ abandonment where rejection control is exercised on customers currently waiting in the queue. Our aim is to find a good balance between conflicting goals, namely, the rate of rejected customers and a cost function which may involve wait and abandonment metrics like percentiles of the waiting time or rate of abandonment. We develop a Markov decision process approach where the waiting time of the first customer in line is used in a discretized form to define the system state. We show that a time-based threshold policy is optimal, and develop a procedure to compute the optimal threshold. Our analysis explains some known behaviors in practice. For instance, if the cost function is constant in the system state like with wait percentiles, then the optimal threshold is one of the time limits defining the percentiles. Also, abandonment is shown to have beneficial or detrimental effect depending on the system manager’s objective.

Analysis of a 2-state discrete-time queue with stochastic state-period lengths and state-dependent server availability and arrivals

In this work we study a discrete-time multiserver queueing system with an infinite storage capacity and deterministic service times equal to 1 slot. Specific to the model under study is that the system is assumed to be in one of two different states (state-1 or state-2) and that both the distribution of the number of available servers and the arrival process depend on the system state. State changes can only occur at slot boundaries and mark the beginning and end of state-1-periods and state-2-periods. The lengths of these state-1-periods and state-2-periods, expressed as a number of slots, are assumed to be two independent sets of independent and identically distributed stochastic variables. The number of available servers during a slot is a stochastic variable with a distribution that is completely determined by the system state during that slot. Likewise, the distribution of the number of arrivals during a slot only depends on the system state during that slot. The only restrictions we put on the distributions of the state-1-periods, state-2-periods and number of available servers is that they have rational probability generating functions (pgfs), and that during each slot at least one server is available. For the considered queueing system we present a method to determine the pgf of the steady-state system content at various observation instants. Several numerical examples demonstrate the possibilities of this model.

Optimal server assignment in multi-server queueing systems with random connectivities

In this paper, we provide complementary results on delay-optimal server allocation in multi-queue multi-server (MQMS) systems with random connectivities. More specifically, we consider an MQMS system where each queue is limited to get service by at most one server during each time slot. It is known that maximum weighted matching (MWM) is a throughput-optimal server assignment policy for such a system. In this paper, using dynamic coupling argument we prove that for a system with i.i.d. Bernoulli arrivals and connectivities, MWM minimizes, in stochastic ordering sense, a range of cost functions of the queue lengths such as total queue occupancy (which implies minimization of average queueing delay). Finally, we propose a low complexity heuristic server assignment policy for MQMS systems namely least connected server first/longest connected queue (LCSF/LCQ) and through simulations we show that it performs very closely compared with the optimal policy in terms of average queueing delay.

Stability Analysis of a Multi-server Model with Simultaneous Service and a Regenerative Input Flow

We study the stability conditions of a multi-server queueing system in which each customer requires a random number of servers simultaneously. The input flow is assumed to be a regenerative one and random service times are identical for all occupied servers. The service time has a hypoexponential distribution which belongs to the class of phase-type distributions. We introduce an auxiliary queueing system in which there are always customers in the queue and define an auxiliary service process as the number of served customers in this system. Then we construct the sequence of common regeneration points for the regenerative input flow and the auxiliary service process. Based on the relationship between the real and the auxiliary service processes we obtain upper and lower estimates for the mean of the number of actually served customers during the common regeneration period. It allows us to deduce the stability criterion of the model under consideration. It turns out that the stability condition does not depend on the structure of the input flow. It only depends on the rate of this process.

Priority, Capacity Rationing, and Ambulance Diversion in Emergency Departments

We evaluate the practices of priority, capacity rationing, and ambulance diversion in emergency department management. We consider a two-class non-preemptive priority M/M/c queue where high- and low-priority customers correspond to acute and non-acute patients, respectively; the two classes have heterogeneous service time requirements. We model capacity rationing by reserving k servers (beds) to high priority customers, and ambulance diversion by balking acute customers from entering the system when the number of waiting acute patients is higher than m. We provide asymptotic result indicating that for a system with no ambulance diversion and a large number of servers, c, the non-degenerative capacity rationing level is of order O(√c). We also derive exact solutions for different performance measures of interest for this system using queueing Markov chain decomposition (QMCD). These performance measures include expected number of patients, distribution of waiting time, and the rate of ambulance diversion. These are the first exact results for a multi-server queue with non-preemptive priorities and heterogeneous service rates, and thus are of independent interest. We demonstrate numerically how system parameters impact these performance measures and provide insights on the control of capacity rationing and ambulance diversion in emergency departments. Specifically, we show that, as predicted by the asymptotic analysis, a low level of capacity rationing can significantly reduce the waits of high-priority customers with little effect on the waiting of low priority customers.

Optimal strategies for managing complex authentication systems

We study an authentication system that receives requests from different types of users. A centralized controller must assign an authentication method to each request, considering the type, the state of the system and the characteristics of several available methods. Each authentication method has different capacity, service rate, level of security, level of usability and operating cost. We seek to optimize security, usability and operating cost, simultaneously by assigning authentication methods dynamically, in real time. To do this, we model the system as a network of parallel multi-server queues, where each queue represents an authentication method and each customer represents a request. We use two different approaches to handle the multiple objectives: a weighted total cost function, and treating security and latency as constraints while minimizing operating cost. We employ constrained and unconstrained Markov decision processes to determine the structure of policies that effectively balance these three objectives. We conclude that if there are infinitely many servers for each authentication method, then the optimal policy is static. We also show that if one method has finite capacity, then the optimal policy is of trunk reservation form. Our results regarding the structure of the optimal policy are consistent for both modeling approaches. Our work shows that optimal policies have intuitive, easy-to-implement structures that are useful in practice. Under certain assumptions, we provide a straightforward way to obtain an optimal policy. We also offer strategies to use our models to explore non-dominated solutions over the three objective functions.

Delay performance of data-center queue with setup policy and abandonment

This paper considers a finite capacity multiserver queue for performance modeling of data centers. One of the most important issues in data centers is to save energy of servers. To this end, a natural policy is to turn off a server immediately once it has no job to process. In this policy, which is called ON–OFF policy, the server must be setup upon the arrival of a new job. During the setup time, the server cannot process jobs but consumes energy. To mitigate the drawback, this paper considers a setup policy, where the number of setup servers at a time is limited. We also consider an extension of the setup policy in which some of servers, but not all of them, are allowed to remain idle for a random amount of time. The main purpose of this paper is to analyze the delay distribution of the multiserver queue with the setup policies. We assume that jobs may abandon the queue without receiving their services. We formulate the queue length process using a two-dimensional continuous-time Markov chain. It can be shown that we cannot apply the distributional Little’s law to obtain the waiting time distribution via the queue length distribution. Therefore, we construct a three-dimensional absorbing Markov chain which describes the virtual waiting time process. We then obtain a phase-type expression of the stationary waiting time distribution. We evaluate the performance of the multiserver queue with the setup policy, and then discuss the optimal control parameters of the policy based on the delay performance as well as the mean power consumption.

Stationary Characteristics of an Unreliable Multi-Server Queueing System with Losses and Time Redundancy

This paper considers an unreliable restorable multi-server queueing system with losses in which failures may occur during requests service; in case of such failures, requests are served using a random time redundancy. The incoming flow is assumed to be elementary and all other random variables in the system’s description are assumed to obey a generalform distribution. A semi-Markov model of system’s evolution over time is developed and a stationary distribution of the embedded Markov chain is found. Explicit-form expressions for determining the stationary probabilities and mean stationary sojourn times in different physical states of the system are obtained, which can be used to estimate the time redundancy effect for the stationary characteristics of the system.

On the Optimal Design of a Bipartite Matching Queueing System

We consider a multi-class multi-server queueing system and study the problem of designing an optimal matching topology (or service compatibility structure) between customer classes and servers under a FCFS-ALIS service discipline. Specifically, we are interested in finding matching topologies that optimize --in a Pareto efficiency-- sense the trade-off between two competing objectives: (i) minimizing customers' waiting time delays and (ii) maximizing matching rewards generated by pairing customers and servers. Our analysis of the problem is divided in three main parts. First, under heavy-traffic conditions, we show that any bipartite matching system can be partitioned into a collection of complete resource pooling (CRP) subsystems, which are interconnected by means of a direct acyclic graph (DAG). We show that this DAG together with the aggregate service capacity on each CRP component fully determine the vector of steady-state waiting times. In particular, we show that the average (scaled) steady-state delay across all customer classes is asymptotically equal to the number of CRP components divided by the total system capacity. Second, since computing matching rewards under a FCFS-ALIS service discipline is computationally infeasible as the number of customer classes and servers grow large, we propose a quadratic programming (QP) formulation to approximate matching rewards. We show that the QP formulation is exact for a number of instances of the problem and provides a very good approximation in general. Extensive numerical experiments show that in over 98% of problem instances the relative error between the exact rewards and the QP approximate rewards is less than 2%. Lastly, combining our characterization of average delays in terms of the number of CRP components and the quadratic programming formulation to compute matching rewards, we propose a mixed-integer linear program (MILP) that can be used to find the set of matching topologies that define the Pareto frontier of reward-delay pairs.