General Service Time Distributions Research Articles

Randomized load balancing with general service time distributions

Randomized load balancing greatly improves the sharing of resources in a number of applications while being simple to implement. One model that has been extensively used to study randomized load balancing schemes is the supermarket model. In this model, jobs arrive according to a rate-nλ Poisson process at a bank of n rate-1 exponential server queues. A notable result, due to Vvedenskaya et.al. (1996), showed that when each arriving job is assigned to the shortest of d ≥ 2 randomly chosen queues, the equilibrium queue sizes decay doubly exponentially in the limit as n to ∞. This is a substantial improvement over the case d=1, where queue sizes decay exponentially. The method of analysis used in the above paper and in the subsequent literature applies to jobs with exponential service time distributions and does not easily generalize. It is desirable to study load balancing models with more general, especially heavy-tailed, service time distributions since such service times occur widely in practice. This paper describes a modularized program for treating randomized load balancing problems with general service time distributions and service disciplines. The program relies on an ansatz which asserts that any finite set of queues in a randomized load balancing scheme becomes independent as n to ∞. This allows one to derive queue size distributions and other performance measures of interest. We establish the ansatz when the service discipline is FIFO and the service time distribution has a decreasing hazard rate (this includes heavy-tailed service times). Assuming the ansatz , we also obtain the following results: (i) as n to ∞, the process of job arrivals at any fixed queue tends to a Poisson process whose rate depends on the size of the queue, (ii) when the service discipline at each server is processor sharing or LIFO with preemptive resume, the distribution of the number of jobs is insensitive to the service distribution, and (iii) the tail behavior of the queue-size distribution in terms of the service distribution for the FIFO service discipline.

Sequential clinical scheduling with patient no-shows and general service time distributions

A sequential clinical scheduling method for patients with general service time distributions is developed in this paper. Patients call a medical clinic to request an appointment with their physician. During the call, the scheduler assigns the patient to an available slot in the physician's schedule. This is communicated to the patient before the call terminates and, thus, the schedule is constructed sequentially. In practice, there is very limited opportunity to adjust the schedule once the complete set of patients is known. Scheduled patients might not attend, that is, they might “no-show,” and the service times of those attending are random. A myopic scheduling algorithm with an optimal stopping criteria for this problem assuming exponential service times already exists in the literature. This work relaxes this assumption and develops numerical techniques for general service time distributions. A special case in which service times are gamma distributed is considered and it is shown that computation is significantly reduced. Finally, exhaustive experimental results are provided along with discussions that provide insights into the practical aspects of the scheduling approach.

Bernoulli schedule vacation queue with batch arrivals and random system breakdowns having general repair time distribution

We analyse a single server queue with general service time distribution, random system breakdowns and Bernoulli schedule server vacations where after a service completion, the server may decide to leave the system with probability p, or to continue serving customers with probability 1−p. It is assumed that the customers arrive to the system in batches of variable size, but served one by one. If the system breaks down, it enters a repair process immediately. It is assumed that the repair time has general distribution, while the vacation time has exponential distribution. The purpose is to find the steady-state results in explicit and closed form in terms of the probability-generating functions for the number of customers in the queue, the average number of customers and the average waiting time in the queue. Some special cases of interest are discussed and a numerical illustration is provided.

Importance sampling algorithms for first passage time probabilities in the infinite server queue

Abstract This paper applies importance sampling simulation for estimating rare event probabilities of the first passage time in the infinite server queue with renewal arrivals and general service time distributions. We consider importance sampling algorithms which are based on large deviations results of the infinite server queue, and we consider an algorithm based on the cross-entropy method, where we allow light-tailed and heavy-tailed distributions for the interarrival times and the service times. Efficiency of the algorithms is discussed by simulation experiments.

Higher-order distributional properties in closed queueing networks

In many real-life computer and networking applications, the distributions of service times, or times between arrivals of requests, or both, can deviate significantly from the memoryless negative exponential distribution that underpins the product-form solution for queueing networks. Frequently, the coefficient of variation of the distributions encountered is well in excess of one, which would be its value for the exponential. For closed queueing networks with non-exponential servers there is no known general exact solution, and most, if not all, approximation methods attempt to account for the general service time distributions through their first two moments. We consider two simple closed queueing networks which we solve exactly using semi-numerical methods. These networks depart from the structure leading to a product-form solution only to the extent that the service time at a single node is non-exponential. We show that not only the coefficients of variation but also higher-order distributional properties can have an important effect on such customary steady-state performance measures as the mean number of customers at a resource or the resource utilization level in a closed network. Additionally, we examine the state that a request finds upon its arrival at a server, which is directly tied to the resulting quality of service. Although the well-known Arrival Theorem holds exactly only for product-form networks of queues, some approximation methods assume that it can be applied to a reasonable degree also in other closed queueing networks. We investigate the validity of this assumption in the two closed queueing models considered. Our results show that, even in the case when there is a single non-exponential server in the network, the state found upon arrival may be highly sensitive to higher-order properties of the service time distribution, beyond its mean and coefficient of variation. This dependence of mean numbers of customers at a server on higher-order distributional properties is in stark contrast with the situation in the familiar open M / G / 1 queue. Thus, our results put into question virtually all traditional approximate solutions, which concentrate on the first two moments of service time distributions.

A diffusion approximation for batch queues of type [formula omitted

Queues with bulk arrivals and batch service arise in a variety of application areas, in particular in the performance analysis of manufacturing lines, like semiconductor fabrications. In the course of this paper, a diffusion approximation for queueing systems with bulk arrivals and batch service for general and independent interarrival time distributions, as well as general service time distributions, is presented. The approximation uses a modified absorbing barrier and instantaneous return approach. The accuracy of the gained approximation is shown by a subsequent comparison with the corresponding discrete event simulation results.

An approximation for multi-server queues with deterministic reneging times

This work was motivated by the timeout mechanism used in managing application servers in transaction processing environments. In such systems, a customer who stays in the queue longer than the timeout period is lost. We modeled a server node with a timeout threshold as a multi-server queue with Poisson arrivals, general service time distribution and deterministic reneging times. We proposed a scaling approach, and a fast and accurate approximation for the expected waiting time in the queue.

Response time reduction in make-to-order and assemble-to-order supply chain design

Make-to-order and assemble-to-order systems are successful business strategies in managing responsive supply chains, characterized by high product variety, highly variable customer demand and short product life cycles. These systems usually spell long customer response times due to congestion. Motivated by the strategic importance of response time reduction, this paper presents models for designing make-to-order and assemble-to-order supply chains under Poisson customer demand arrivals and general service time distributions. The make-to-order supply chain design model seeks to simultaneously determine the location and the capacity of distribution centers (DCs) and allocate stochastic customer demand to DCs by minimizing response time in addition to the fixed cost of opening DCs and equipping them with sufficient assembly capacity and the variable cost of serving customers. The problem is setup as a network of spatially distributed M/G/1 queues, modeled as a non-linear mixed-integer program, and linearized using a simple transformation and a piecewise linear approximation. An exact solution approach is presented that is based on the cutting plane method. Then, the problem of designing a two-echelon assemble-to-order supply chain comprising of plants and DCs serving a set of customers is considered. A Lagrangean heuristic is proposed that exploits the echelon structure of the problem and uses the solution methodology for the make-to-order problem. Computational results and managerial insights are provided. It is empirically shown that substantial reduction in response times can be achieved with minimal increase in total costs in the design of responsive supply chains. Furthermore, a supply chain configuration that considers congestion is proposed and its effect on the response time can be very different from the traditional configuration that ignores congestion.

Diffusion Approximation Model of Multiserver Stations with Losses

The article presents a diffusion approximation model of a G / G / N / N station – N parallel servers without queueing. Diffusion approximation allows us to include in queueing models fairly general assumptions. First of all it gives us a tool to consider in a natural way transient states of queues, which is very rare in classical queueing models. Then we may consider input streams with general interarrival time distributions and servers with general service time distributions. Single server models may be easily incorporated into a network of queues. Here, we apply the diffusion approximation formalism to study transient behaviour of G / G / N / N station and use it to construct a model of a typical call centre and to study the sliding window mechanism, a popular Call Admission Control (CAC) algorithm.

Probability masses fitting in the analysis of manufacturing flow lines

The analysis of manufacturing systems with finite capacity and with general service time distributions is made of two steps: the distributions have first to be transformed into tractable phase-type distributions, and then the modified system can be analytically modelled. In this paper, we propose a new alternative in order to build tractable phase-type distributions, and study its effects on the global modelling process. Called “probability masses fitting” (PMF), the approach is quite simple: the probability masses on regular intervals are computed and aggregated on a single value in the corresponding interval, leading to a discrete distribution. PMF shows some interesting properties: it is bounding, monotonic, refinable, it approximates distributions with finite support and it conserves the shape of the distribution. With the resulting discrete distributions, the evolution of the system is then exactly modelled by a Markov chain. Here, we focus on flow lines and show that the method allows us to compute upper and lower bounds on the throughput as well as good approximations of the cycle time distributions. Finally, the global modelling method is shown, by numerical experiments, to compute accurate estimations of the throughput and of various performance measures, reaching accuracy levels of a few tenths of a percent.

Optimal management policy for a single and bulk service queue under Bernoulli vacation schedules

This paper generalises the single server queue with single and bulk service characterised by the bilevel service discipline introduced by Baburaj (1999). In this system if the size (n) is not more than r, then the server takes a single customer for service according to FCFS rule and if the size is more than r then he serves min{n,R} (R > r) units in a batch according to FCFS rule. While Baburaj assumes exponential service times we consider general service time distributions. We also consider that the server implements a Bernoulli vacation schedule, so that a vacation is taken with some positive probability when a service is completed. The first objective is to derive the probability generating function of the number of customers in the system at a service completion epoch and at an arbitrary instant of time, as well as the performance characteristics of the system. The second objective is to develop an optimal management policy through a quick search algorithm.

POLLING SYSTEMS WITH TWO-PHASE GATED SERVICE

We study an asymmetric cyclic polling system with Poisson arrivals, general service-time and switch-over time distributions, and so-called two-phase gated service at each queue, an interleaving scheme that aims to enforce some level of “fairness” among the different customer classes. For this model, we use the classical theory of multitype branching processes to derive closed-form expressions for the Laplace–Stieltjes transform of the waiting-time distributions when the load tends to 1, in a general parameter setting and under proper heavy-traffic scalings. This result is strikingly simple and provides new insights in the behavior of two-phase polling systems. In particular, the result provides insight in the waiting-time performance and the trade-off between efficiency and fairness of two-phase gated polling compared to the classical one-phase gated service policy.

The Conditional Distribution of the Residual Service Time in the M n /G/1 Queue

We study a single server queueing system with general service time distribution and memoryless inter-arrival times. The arrival rate is not constant but varies with the number of customers in the system. A recursive formula for the conditional distribution of the remaining service time given the queue length is derived for an arbitrary epoch. It is also shown that this conditional distribution holds for arrival epochs. The recursion for the corresponding Laplace–Stieltjes transforms and expected values is given in a simple formula.

A Unifying Conservation Law for Single-Server Queues

We develop a conservation law for a multi-class GI/GI/1 queue operating under a general work-conserving scheduling discipline. For single-class single-server queues, conservation laws have been obtained for both nonanticipating and anticipating disciplines with general service time distributions. For multi-class single-server queues, conservation laws have been obtained for (i) nonanticipating disciplines with exponential service time distributions and (ii) nonpreemptive nonanticipating disciplines with general service time distributions. The unifying conservation law we develop generalizes already existing conservation laws. In addition, it covers popular nonanticipating multi-class time-sharing disciplines such as discriminatory processor sharing (DPS) and generalized processor sharing (GPS) with general service time distributions. As an application, we show that the unifying conservation law can be used to compare the expected unconditional response time under two scheduling disciplines.

A Unifying Conservation Law for Single-Server Queues

We develop a conservation law for a multi-class GI/GI/1 queue operating under a general work-conserving scheduling discipline. For single-class single-server queues, conservation laws have been obtained for both nonanticipating and anticipating disciplines with general service time distributions. For multi-class single-server queues, conservation laws have been obtained for (i) nonanticipating disciplines with exponential service time distributions and (ii) nonpreemptive nonanticipating disciplines with general service time distributions. The unifying conservation law we develop generalizes already existing conservation laws. In addition, it covers popular nonanticipating multi-class time-sharing disciplines such as discriminatory processor sharing (DPS) and generalized processor sharing (GPS) with general service time distributions. As an application, we show that the unifying conservation law can be used to compare the expected unconditional response time under two scheduling disciplines.

Polling models with renewal arrivals: A new method to derive heavy-traffic asymptotics

We consider asymmetric cyclic polling systems with an arbitrary number of queues, general service-time distributions, zero switch-over times, gated service at each queue, and with general renewal arrival processes at each of the queues. For this classical model, we propose a new method to derive closed-form expressions for the expected delay at each of the queues when the load tends to 1, under proper heavy-traffic (HT) scalings. In the literature on polling models, rigorous proofs of HT limits have only been obtained for polling models with Poisson-type arrival processes, whereas for renewal arrivals HT limits are based on conjectures [E.G. Coffman, A.A. Puhalskii, M.I. Reiman, Polling systems with zero switch-over times: A heavy-traffic principle, Ann. Appl. Probab. 5 (1995) 681–719; E.G. Coffman, A.A. Puhalskii, M.I. Reiman, Polling systems in heavy-traffic: A Bessel process limit, Math. Oper. Res. 23 (1998) 257–304; T.L. Olsen, R.D. van der Mei, Periodic polling systems in heavy-traffic: Renewal arrivals, OR Lett. 33 (2005) 17–25]. Therefore, the main contribution of this paper lies in the fact that we propose a new method to rigorously prove HT limits for a class of non-Poisson-type arrivals. The results are remarkably simple and provide new fundamental insight and reveal explicitly how the expected delay at each of the queues depends on the system parameters, and in particular on the interarrival-time distributions at each of the queues. The results also suggest simple approximations for the expected delay in stable polling systems. Numerical results show that the approximations are highly accurate when the system load is roughly 90% or more.

Time to Reach Buffer Capacity in a BMAP Queue

In this paper a detailed study is presented on the first time to reach buffer capacity in a queue with batch arrivals and general service time distribution. A flexible analytical model of the input stream, which is the Batch Markovian Arrival Process (BMAP), is assumed. The results include the explicit formula for the Laplace transform of the distribution of the first buffer overflow time and discussion of its computational aspects. In addition, the popular special case of the BMAP queue, which is the batch Poisson arrival queue, is studied. Theoretical results are illustrated via numerical calculations based on IP traffic data.

Estimation of the mean waiting time of a customer subject to balking: a simulation study

To a customer, the waiting time for order processing for a product or service is important information for order placement. If the time foreseen for order fulfillment is long, the order might be lost to a competitor. In particular, modern principles of supply chain management highly suggest information sharing between entities in the chain and information technology has enabled customers to conveniently consider the waiting time for a potential balking decision. To help determine the design and operation of a manufacturing or service system in which a customer may balk based on the foreseen waiting time, this paper develops procedures to estimate the average waiting time of an order. Either the procedures allow the maximum waiting time for a balking decision to be random or do not require knowledge of the arrival process of customers before balking if the balking limit is known. For generality of the model, this paper considers general inter-arrival and service time distributions, and uses the simulation and regression approach.

Estimation of the mean waiting time of a customer subject to balking: a simulation study

To a customer, the waiting time for order processing for a product or service is important information for order placement. If the time foreseen for order fulfillment is long, the order might be lost to a competitor. In particular, modern principles of supply chain management highly suggest information sharing between entities in the chain and information technology has enabled customers to conveniently consider the waiting time for a potential balking decision. To help determine the design and operation of a manufacturing or service system in which a customer may balk based on the foreseen waiting time, this paper develops procedures to estimate the average waiting time of an order. Either the procedures allow the maximum waiting time for a balking decision to be random or do not require knowledge of the arrival process of customers before balking if the balking limit is known. For generality of the model, this paper considers general inter-arrival and service time distributions, and uses the simulation and regression approach.

Buffer Overflow Period in a MAP Queue

The buffer overflow period in a queue with Markovian arrival process (MAP) and general service time distribution is investigated. The results include distribution of the overflow period in transient and stationary regimes and the distribution of the number of cells lost during the overflow interval. All theorems are illustrated via numerical calculations.