General Service Time Distributions Research Articles

Sampling for data freshness optimization: Non-linear age functions

In this paper, we study how to take samples at a data source for improving the freshness of received data samples at a remote receiver. We use non-linear functions of the age of information to measure data freshness, and provide a survey of non-linear age functions and their applications. The sampler design problem is studied to optimize these data freshness metrics, even when there is a sampling rate constraint. This sampling problem is formulated as a constrained Markov decision process (MDP) with a possibly uncountable state space. We present a complete characterization of the optimal solution to this MDP: The optimal sampling policy is a deterministic or randomized threshold policy, where the threshold and the randomization probabilities are characterized based on the optimal objective value of the MDP and the sampling rate constraint. The optimal sampling policy can be computed by bisection search, and the curse of dimensionality is circumvented. These age optimality results hold for (i) general data freshness metrics represented by monotonic functions of the age of information, (ii) general service time distributions of the queueing server, (iii) both continuoustime and discrete-time sampling problems, and (iv) sampling problems both with and without the sampling rate constraint. Numerical results suggest that the optimal sampling policies can be much better than zero-wait sampling and the classic uniform sampling.

Analysis of Redundancy(d) with Identical Replicas

Queueing systems with redundancy have received considerable attention recently. The idea of redundancy is to reduce latency by replicating each incoming job a number of times and to assign these replicas to a set of randomly selected servers. As soon as one replica completes service the remaining replicas are cancelled. Most prior work on queueing systems with redundancy assumes that the job durations of the different replicas are i.i.d., which yields insights that can be misleading for computer system design. In this paper we develop a differential equation, using the cavity method, to assess the workload and response time distribution in a large homogeneous system with redundancy without the need to rely on this independence assumption. More specifically, we assume that the duration of each replica of a single job is identical across the servers and follows a general service time distribution. Simulation results suggest that the differential equation yields exact results as the system size tends to infinity and can be used to study the stability of the system.

The mean-field behavior of processor sharing systems with general job lengths under the SQ([formula omitted]) policy

This paper addresses the mean-field behavior of large-scale systems of parallel servers with a processor sharing service discipline when arrivals are Poisson and jobs have general service time distributions when an SQ(d) routing policy is used. Under this policy, an arrival is routed to the server with the least number of progressing jobs among d randomly chosen servers. The limit of the empirical distribution is then used to study the statistical properties of the system. In particular, this shows that in the limit as N grows, individual servers are statistically independent of others (propagation of chaos) and more importantly, the equilibrium point of the mean-field is insensitive to the job length distributions that has important engineering relevance for the robustness of such routing policies used in web server farms. We use a framework of measure-valued processes and martingale techniques to obtain our results. We also provide numerical results to support our analysis.

Design heuristic for parallel many server systems

We study a parallel queueing system with multiple types of servers and customers. A bipartite graph describes which pairs of customer-server types are compatible. We consider the service policy that always assigns servers to the first, longest waiting compatible customer, and that always assigns customers to the longest idle compatible server if on arrival multiple compatible servers are available. For a general renewal stream of arriving customers, general service time distributions that depend both on customer and on server types, and general customer patience distributions, the behavior of such systems is very complicated. Key quantities for their performance are the matching rates, the fraction of services for each pair of compatible customer-server. Calculation of these matching rates in general is intractable, it depends on the entire shape of service time distributions. We suggest through a heuristic argument that if the number of servers becomes large, the matching rates are well approximated by matching rates calculated from the tractable bipartite infinite matching model. We present simulation evidence to support this heuristic argument, and show how this can be used to design systems with desired performance requirements.

Socially optimal design of a service network with location-aware multi-services under different delivery policies

Abstract This paper considers an immobile service network design/redesign problem with multiple location-aware service types and general service-time distributions where customers are assigned to appropriate service centers based on their geographical locations and requested service types. The problem seeks to design the service system so as to maximize a performance metric considering the system’s profitability and social metrics of service accessibility and quality under different service-delivery policies. The problem is formulated as a discrete optimization model, and different managerial insights are provided.

The PDE Method for the Analysis of Randomized Load Balancing Networks

We introduce a new framework for the analysis of large-scale load balancing networks with general service time distributions, motivated by applications in server farms, distributed memory machines, cloud computing and communication systems. For a parallel server network using the so-called $SQ(d)$ load balancing routing policy, we use a novel representation for the state of the system and identify its fluid limit, when the number of servers goes to infinity and the arrival rate per server tends to a constant. The fluid limit is characterized as the unique solution to a countable system of coupled partial differential equations (PDE), which serve to approximate transient Quality of Service parameters such as the expected virtual waiting time and queue length distribution. In the special case when the service time distribution is exponential, our method recovers the well-known ordinary differential equation characterization of the fluid limit. Furthermore, we develop a numerical scheme to solve the PDE, and demonstrate the efficacy of the PDE approximation by comparing it with Monte Carlo simulations. We also illustrate how the PDE can be used to gain insight into the performance of large networks in practical scenarios by analyzing relaxation times in a backlogged network. In particular, our numerical approximation of the PDE uncovers two interesting properties of relaxation times under the SQ(2) algorithm. Firstly, when the service time distribution is Pareto with unit mean, the relaxation time decreases as the tail becomes heavier. This is a priori counterintuitive given that for the Pareto distribution, heavier tails have been shown to lead to worse tail behavior in equilibrium. Secondly, for unit mean light-tailed service distributions such as the Weibull and lognormal, the relaxation time decreases as the variance increases. This is in contrast to the behavior observed under random routing, where the relaxation time increases with increase in variance.

Perfect sampling of GI/GI/c queues

We introduce the first class of perfect sampling algorithms for the steady-state distribution of multi-server queues with general interarrival time and service time distributions. Our algorithm is built on the classical dominated coupling from the past protocol. In particular, we use a coupled multi-server vacation system as the upper bound process and develop an algorithm to simulate the vacation system backwards in time from stationarity at time zero. The algorithm has finite expected termination time with mild moment assumptions on the interarrival time and service time distributions.

Approximating service-time distributions by phase-type distributions in single-server queues: a strong stability approach

Phase-type queueing systems are used to approximate queues with general service-time distributions. In this work, we provide by means of the strong stability method, the mathematical justification of the approximation method by phase-type distributions that is already used in several works. We consider the approximation of M/G/1 queueing system by a M/PH/1 system, where PH refers to a hyperexponential H2 or a hypoexponential HOE2 distribution depending on the value of the coefficient of variation of the original distribution. We prove the robustness of the underlying Markov chain in each case and estimate an upper bound of the deviation of the stationary vector, resulting from the perturbation of the service-time distribution. We provide numerical examples and compare the perturbation bounds obtained in this paper with the estimates of the real deviation of the stationary vector obtained by simulation.

The PDE Method for the Analysis of Randomized Load Balancing Networks

We introduce a new framework for the analysis of large-scale load balancing networks with general service time distributions, motivated by applications in server farms, distributed memory machines, cloud computing and communication systems. For a parallel server network using the so-called $SQ(d)$ load balancing routing policy, we use a novel representation for the state of the system and identify its fluid limit, when the number of servers goes to infinity and the arrival rate per server tends to a constant. The fluid limit is characterized as the unique solution to a countable system of coupled partial differential equations (PDE), which serve to approximate transient Quality of Service parameters such as the expected virtual waiting time and queue length distribution. In the special case when the service time distribution is exponential, our method recovers the well-known ordinary differential equation characterization of the fluid limit. Furthermore, we develop a numerical scheme to solve the PDE, and demonstrate the efficacy of the PDE approximation by comparing it with Monte Carlo simulations. We also illustrate how the PDE can be used to gain insight into the performance of large networks in practical scenarios by analyzing relaxation times in a backlogged network. In particular, our numerical approximation of the PDE uncovers two interesting properties of relaxation times under the SQ(2) algorithm. Firstly, when the service time distribution is Pareto with unit mean, the relaxation time decreases as the tail becomes heavier. This is a priori counterintuitive given that for the Pareto distribution, heavier tails have been shown to lead to worse tail behavior in equilibrium. Secondly, for unit mean light-tailed service distributions such as the Weibull and lognormal, the relaxation time decreases as the variance increases. This is in contrast to the behavior observed under random routing, where the relaxation time increases with increase in variance.

IMPROVING THE PERFORMANCE OF POLLING MODELS USING FORCED IDLE TIMES

We consider polling models in the sense of Takagi [19]. In our case, the feature of the server is that it may be forced to wait idly for new messages at an empty queue instead of switching to the next station. We propose four different wait-and-see strategies that govern these waiting periods. We assume Poisson arrivals for new messages and allow general service and switchover time distributions. The results are formulas for the mean average queueing delay and characterizations of the cases where the wait-and-see strategies yield a lower delay compared with the exhaustive strategy.

Infinite Divisibility of Some Steady State Queue Characteristics

In this paper we consider various continuous/discrete time queueing models with general service time distribution. With an aim to identify queueing models we examine the structural properties like infinite divisibility of the stationary distribution of various characteristics of different queueing models and report some new results. Our investigation is further extended to bulk arrival models and multiple vacation queueing models, revealing the structural aspects of their steady state distributions.

Multi-server preemptive priority queue with general arrivals and service times

We present a simple approximate solution for preemptive-resume queues with multiple servers, general (phase-type) service and general (phase-type) interarrival time distributions. In our solution, priority levels are solved one at a time in the order of decreasing priorities. Each priority level is solved approximately using a reduced state description. The complexity of our approximate solution in terms of the number of equations solved grows linearly with the number of servers and priority levels.We studied a large number of numerical examples with a range of values for mean service times and offered loads across priority levels, varying the number of servers from 8 to 48. Discrete-event simulation was used to assess the accuracy of our approximate solution. Overall, in the case of Poisson and quasi-Poisson arrivals, expected relative error for the mean number of customers in the system was below 2% while the corresponding median relative error was below 0.25%. The good accuracy of our approximation appears to extend to the case of phase-type times between arrivals, with expected relative errors for the mean number in system below 5% even for a Pareto-like distribution of interarrival times with a large coefficient of variation. Our numerical results indicate that the proposed approximation provides a relatively simple and generally accurate approach to preemptive-resume queues with larger numbers of servers and general distributions of service and interarrival times.

Analysis of a two-node closed queueing cyclic network with one non-exponential node

Consider a two-node closed cyclic queueing network with a total of M customers. The first node consists of N⩽M parallel identical servers having exponential service times. Node 2 consists of a single server having a general service time distribution. Waiting space at each node is sufficient to accommodate all M customers.We derive an exact solution for the steady state system size probabilities. Our approach is based on developing and solving an imbedded Markov chain at Node 2 service completion epochs coupled with a birth process over Node 1 service times. In that sense, our methodology is intuitively appealing and easy to apply. We focus on analytically and numerically investigating the effect of Node 2 service time variability on system performance. We find that closed queueing networks, like the one considered here, are less vulnerable to variability than open networks. However, the effect of variability is still significant to merit considerable attention. Numerical results are presented to illustrate the viability of our method for many service time distributions. As by-products, our work yield new results on birth processes and combinatorial identities, which can be useful in other contexts.A primary application of this model is the well-known machine repair model where a set of identical machines are attended by a single repairman. Other applications include performance evaluation of manufacturing and computer networks, as well as reliability studies where our model can be easily used to compute system availability.

Efficient Redundancy Techniques for Latency Reduction in Cloud Systems

In cloud computing systems, assigning a task to multiple servers and waiting for the earliest copy to finish is an effective method to combat the variability in response time of individual servers and reduce latency. But adding redundancy may result in higher cost of computing resources, as well as an increase in queueing delay due to higher traffic load. This work helps in understanding when and how redundancy gives a cost-efficient reduction in latency. For a general task service time distribution, we compare different redundancy strategies in terms of the number of redundant tasks and the time when they are issued and canceled. We get the insight that the log-concavity of the task service time creates a dichotomy of when adding redundancy helps. If the service time distribution is log-convex (i.e., log of the tail probability is convex), then adding maximum redundancy reduces both latency and cost. And if it is log-concave (i.e., log of the tail probability is concave), then less redundancy, and early cancellation of redundant tasks is more effective. Using these insights, we design a general redundancy strategy that achieves a good latency-cost trade-off for an arbitrary service time distribution. This work also generalizes and extends some results in the analysis of fork-join queues.

Optional services in a non-Markovian queue

We study a single server queue with Poisson arrivals, two optional services following a general service time distributions. The first service is essential. Other two services are optional. Only some of the arriving customers demand the first optional service or the second optional service. We derive the system size distribution at random points and at departure points and other performance indices such as average number of customers and the average waiting time in the queue and the system by employing generating function and supplementary variable techniques. By taking numerical illustration, the sensitivity analysis is also conducted to determine the effects of the system parameters on various performance measures of system state are derived.

Estimation of the traffic intensity in a piecewise-stationary Mt/Gt/1 queue with probing

We use a probing strategy to estimate the time dependent traffic intensity in an Mt/Gt/1 queue, where the arrival rate and the general service-time distribution change from one time interval to another, and derive statistical properties of the proposed estimator. We present a method to detect a switch from a stationary interval to another using a sequence of probes to improve the estimation. At the end, we compare our results with two estimators proposed in the literature for the M/G/1 queue.

Two-parameter process limits for an infinite-server queue with arrival dependent service times

We study an infinite-server queue with a general arrival process and a large class of general time-varying service time distributions. Specifically, customers’ service times are conditionally independent given their arrival times, and each customer’s service time, conditional on her arrival time, has a general distribution function. We prove functional limit theorems for the two-parameter processes Xe(t,y) and Xr(t,y) that represent the numbers of customers in the system at time t that have received an amount of service less than or equal to y, and that have a residual amount of service strictly greater than y, respectively. When the arrival process and the initial content process both have continuous Gaussian limits, we show that the two-parameter limit processes are continuous Gaussian random fields. In the proofs, we introduce a new class of sequential empirical processes with conditionally independent variables of non-stationary distributions, and employ the moment bounds resulting from the method of chaining for the two-parameter stochastic processes.

Spectrum Handoffs Based on Preemptive Repeat Priority Queue in Cognitive Radio Networks.

Cognitive radio can significantly improve the spectrum efficiency, and spectrum handoff is considered as an important functionality to guarantee the quality of service (QoS) of primary users (PUs) and the continuity of data transmission of secondary users (SUs). In this paper, we propose an analytical framework based on a preemptive repeat identical (PRI) M/G/1 queuing network model to characterize spectrum handoff behaviors with general service time distribution of both primary and secondary connections, multiple interruptions and transmission delay resulting from the appearance of primary connections. Then, we derive the close-expression of the extended data delivery and the system sojourn time in both staying and changing scenarios. In addition, based on analysis of spectrum handoff behaviors resulting from multiple interruptions caused by the appearance of the primary connections, we investigate the traffic-adaptive policy, by which the considered SU will optimally adjust its handoff spectrum policy. Moreover, we investigate the admissible region and provide the reference for designing the admission control rule for the arriving secondary connection requests. Finally, simulation results verify that our proposed analytical framework is reasonable and can provide the reference for executing the optimal spectrum handoff strategy and designing the admission control rule for the SU in cognitive radio networks.

A discrete-time queueing network approach to performance evaluation of autonomous vehicle storage and retrieval systems

In this paper, we present a method for performance evaluation of autonomous vehicle storage and retrieval systems (AVS/RSs) with tier-captive single-aisle vehicles. A discrete-time open queueing network approach is applied. The data obtained from the evaluation of the lift and vehicle movements can be used directly as input for the general discrete service time distributions of the queueing network. Furthermore, the approach allows for the computation of the retrieval transaction time distribution as well as of the distribution of the number of transactions waiting to be stored. Consequently, not only expected values and variances but also quantiles of the performance measures can be obtained. Comparison to discrete-event simulation quantifies approximation errors resulting from the decomposition approach in the discrete-time domain. Moreover, the errors obtained by the discrete-time approach are compared to the errors obtained using a continuous-time open queueing network approach. Finally, it will be outlined how the model can be used for designing AVS/RSs according to given system requirements, such as storage capacity, throughput, height and length of the system as well as the 95% quantile of the retrieval transaction time.

Queueing network [formula omitted] with high-rate arrivals

An analysis of the open queueing network MAP−(GI/∞)K is presented in this paper. The MAP−(GI/∞)K network implements Markov routing, general service time distribution, and an infinite number of servers at each node. Analysis is performed under the condition of a growing fundamental rate for the Markovian arrival process. It is shown that the stationary probability distribution of the number of customers at the nodes can be approximated by multi-dimensional Gaussian distribution. Parameters of this distribution are presented in the paper. Numerical results validate the applicability of the obtained approximations under relevant conditions. The results of the approximations are applied to estimate the optimal number of servers for a network with finite-server nodes. In addition, an approximation of higher-order accuracy is derived.