Job Service Times Research Articles

A Foreground–Background queueing model with speed or capacity modulation

Two models involving a foreground and a background queue are studied in the steady state. Service is provided either by a single server whose speed depends on the total number of jobs present, or by several parallel servers whose number may be controlled dynamically. Job service times have a two-phase Coxian distribution. Incoming jobs join the foreground queue where they execute phase 1, and then possibly move to the background queue for the second phase at lower priority. The trade-offs between holding and energy consumption costs are examined by means of a suitable cost function. Two different two-dimensional Markov processes are solved exactly. The solutions are used in several numerical experiments, aimed at illustrating different aspects of system behaviour.

The M/M/k with Deterministic Setup Times

Capacity management, whether it involves servers in a data center, or human staff in a call center, or doctors in a hospital, is largely about balancing a resource-delay tradeoff. On the one hand, one would like to turn off servers when not in use (or send home staff that are idle) to save on resources. On the other hand, one wants to avoid the considerable setup time required to turn an ''off'' server back ''on.'' This paper aims to understand the delay component of this tradeoff, namely, what is the effect of setup time on average delay in a multi-server system? Surprisingly little is known about the effect of setup times on delay. While there has been some work on studying the M/M/k with Exponentially-distributed setup times, these works provide only iterative methods for computing mean delay, giving little insight as to how delay is affected by k , by load, and by the setup time. Furthermore, setup time in practice is much better modeled by a Deterministic random variable, and, as this paper shows, the scaling effect of a Deterministic setup time is nothing like that of an Exponentially-distributed setup time. This paper provides the first analysis of the M/M/k with Deterministic setup times. We prove a lower bound on the effect of setup on delay, where our bound is highly accurate for the common case where the setup time is much higher than the job service time. Our result is a relatively simple algebraic formula which provides insights on how delay scales with the input parameters. Our proof uses a combination of renewal theory, martingale arguments and novel probabilistic arguments, providing strong intuition on the transient behavior of a system that turns servers on and off.

Ergodicity of Two Class Priority Queues with Preemptive Priority

Well-known results on the ergodicity of queues with preemptive priority were obtained under the assumption that jobs arrive according to the Poisson process. This assumption, however, does not always hold true in practice. In our work we find sufficient ergodicity conditions for queues with two priority classes with a single server, where interarrival times of high-priority jobs have either Erlang or hyperexponential distribution and interarrival times of low-priority jobs and service times of jobs of both classes have arbitrary continuous distributions. We present results in disciplines with loss of an interrupted job and with retrial of an interrupted job.

TB/GI/1 queues with arrival traffic envelopes

Consider a queueing system where the job service times are not known upon arrival; e.g., a transmission server of a wireless channel where packet transmission times are random, or a virtual machine handling a stream of tasks whose execution times are not perfectly predictable. We give bounds on the tail of the workload distribution of a partially regulated, single-server queue whose arrival processes are arbitrarily distributed stationary random point processes on the integers that satisfy token-bucket constraints expressed via an arbitrary concave function f, and whose job service times are independent with a common distribution.

Dynamic Resource Allocation in Fork-Join Queues

<?tight?>Fork-join systems play a pivotal role in the analysis of distributed systems, telecommunication infrastructures, and storage systems. In this article, we consider a fork-join system consisting of K parallel servers, each of which works on one of the K tasks that form each job. The system allocates a fixed amount of computational resources among the K servers, hence determining their service speed. The goal of this article is that of studying the resource allocation policies among the servers. We assume that the queueing disciplines of the fork- and join-queues are First Come First Served. At each epoch, at most K tasks are in service while the others wait in the fork-queues. We propose an algorithm with a very simple implementation that allocates the computational resources in a way that aims at minimizing the join-queue lengths, and hence at reducing<?brk?> the expected job service time. We study its performance in saturation and under exponential service time. The model has an elegant closed-form stationary distribution. Moreover, we provide an algorithm to numerically or symbolically derive the marginal probabilities for the join-queue lengths. Therefore, the expressions for the expected join-queue length and the expected response time under immediate join can be derived. Finally, we compare the performance of the proposed resource allocation algorithm with that of other strategies.

Scheduling with Testing

We study a new class of scheduling problems that capture common settings in service environments, in which one has to serve a collection of jobs that have a priori uncertain attributes (e.g., processing times and priorities) and the service provider has to decide how to dynamically allocate resources (e.g., people, equipment, and time) between testing (diagnosing) jobs to learn more about their respective uncertain attributes and processing jobs. The former could inform future decisions, but could delay the service time for other jobs, while the latter directly advances the processing of the jobs but requires making decisions under uncertainty. Through novel analysis we obtain surprising structural results of optimal policies that provide operational managerial insights, efficient optimal and near-optimal algorithms, and quantification of the value of testing. We believe that our approach will lead to further research to explore this important practical trade-off. The online appendix is available at https://doi.org/10.1287/mnsc.2017.2973 . This paper was accepted by Yinyu Ye, optimization.

Optimal workload allocation in closed queueing networks with state dependent queues

The problem of optimal workload allocation in closed queueing network models with multi-server exponential infinite capacity workstations and finite capacity state dependent queueing models is examined. The processing rates (i.e. service times) of jobs in the queueing system are the main focus. State dependent queues are appropriate for modeling the transportation and material transfer of the customers and products within the system. By combining these two types of queues, we can model the processing of jobs or customers at service stations and also the material handling transfer of job and customers between these stations. Because the environment is a closed queueing network model, it allows for the dynamic interaction of the jobs and customers in the optimal workload allocation values. This combination of queues and optimal search process provides a broad range of potential applications in manufacturing and service type design operations where the travel time between workstations is important. Conveyors, automated-guided vehicle systems (AGVS), and fork lift trucks are utilized to show the full range of material handling component systems.

ASIdE: Using Autocorrelation-Based Size Estimation for Scheduling Bursty Workloads

Temporal dependence in workloads creates peak congestion that can make service unavailable and reduce system performance. To improve system performability under conditions of temporal dependence, a server should quickly process bursts of requests that may need large service demands. In this paper, we propose and evaluateASIdE, an Autocorrelation-based SIze Estimation, that selectively delays requests which contribute to the workload temporal dependence. ASIdE implicitly approximates the shortest job first (SJF) scheduling policy but without any prior knowledge of job service times. Extensive experiments show that (1) ASIdE achieves good service time estimates from the temporal dependence structure of the workload to implicitly approximate the behavior of SJF; and (2) ASIdE successfully counteracts peak congestion in the workload and improves system performability under a wide variety of settings. Specifically, we show that system capacity under ASIdE is largely increased compared to the first-come first-served (FCFS) scheduling policy and is highly-competitive with SJF.

State-dependent Control of a Single Stage Hybrid System with Poisson Arrivals

We consider a single-stage hybrid manufacturing system where jobs arrive according to a Poisson process. These jobs undergo a deterministic process which is controllable. We define a stochastic hybrid optimal control problem and decompose it hierarchically to a lower-level and a higher-level problem. The lower-level problem is a deterministic optimal control problem solved by means of calculus of variations. We concentrate on the stochastic discrete-event control problem at the higher level, where the objective is to determine the service times of jobs. Employing a cost structure composed of process costs that are decreasing and strictly convex in service times, and system-time costs that are linear in system times, we show that receding horizon controllers are state-dependent controllers, where state is defined as the system size. In order to improve upon receding horizon controllers, we search for better state-dependent control policies and present two methods to obtain them. These stochastic-approximation-type methods utilize gradient estimators based on Infinitesimal Perturbation Analysis or Imbedded Markov Chain techniques. A numerical example demonstrates the performance improvements due to the proposed methods.

Receding Horizon Control of Mixed Line Flow Shop Systems

We consider reliable mixed line flow shop systems that are composed of controllable and uncontrollable machines. These systems are assumed to receive arrivals at random instants and process jobs deterministically in the order of arrival so as to depart them before their deadlines that are revealed at the time of arrival. We model these flow shops as serial networks of queues operating under a non-preemptive first-come-first-served policy. Defining completion-time costs for jobs and process costs at controllable machines, a stochastic convex optimization problem is formulated where the control variables are the constrained service times of jobs at the controllable machines. As an on-line solution method to determine these service times, we propose a receding horizon controller, which solves a deterministic problem at each decision instant. We quantify the available future information by the look-ahead window size. Numerical examples demonstrate the value of information and that the no-waiting property of the full-information case is not observed in the partial-information case.

A clairvoyant site allocation policy based on service demands of jobs in a computational grid

In the present paper we evaluate the performance of three different site allocation policies in a 2-level computational grid with heterogeneous sites. We examine the case where service demands of jobs show high variability, and we consider that schedulers are aware of the service times. A simulation model is used to evaluate performance in terms of the average response time and slowdown, under medium and high load. The impact of the various input parameters on performance is also taken into account. Simulation results show that one policy outperforms the other two that are examined, especially at high load. Furthermore, no performance degradation is observed when estimations are used instead of exact knowledge about job service times.

Performance and energy aware cluster-level scheduling of compute-intensive jobs with unknown service times

In this paper we examine three local resource allocation policies, which are based on shortest queue, in a cluster with heterogeneous servers. Two of them are optimized for performance and the third one is optimized for energy conservation. We assume that there are two types of processors in the cluster, with different performance and energy characteristics. We consider that service times of jobs are unknown to the scheduler. A simulation model is used to evaluate the performance and energy behavior of the policies. Simulation results indicate that the differences among the policies depend on system load and there is a trade-off between performance and energy consumption.

A discrete-time retrial queueing model with one server

This paper presents a one-server queueing model with retrials in discrete-time. The number of primary jobs arriving in a time slot follows a general probability distribution and the different numbers of primary arrivals in consecutive time slots are mutually independent. Each job requires from the server a generally distributed number of slots for its service, and the service times of the different jobs are independent. Jobs arriving in a slot can start their service only at the beginning of the next slot. When upon arrival jobs find the server busy all incoming jobs are sent into orbit. When upon arrival in a slot jobs find the server idle, then one of the incoming jobs (randomly chosen) in that slot starts its service at the beginning of the next slot, whereas the other incoming jobs in that slot, if any, are sent into orbit. During each slot jobs in the orbit try to re-enter the system individually, independent of each other, with a given retrial probability. The ergodicity condition and the generating function of the joint equilibrium distribution of the number of jobs in orbit and the residual service time of the job in service are calculated. From the generating function several performance measures are deduced, like the average orbit size. Also the busy period and the number of jobs served during a busy period are discussed. To conclude, extensive numerical results are presented.

A two-class parallel system with general service times of the parallel class

A parallel system with two job classes is analyzed. Type-1 jobs require one server for their execution and have exponentially distributed service times while type-2 jobs need two servers and have general service times. The model consists of a single queue served by two servers that may work either independently or in parallel. It is assumed that all jobs are rigid and share the servers according to pure space sharing. We provide closed-form expressions, exact as well as approximate, for various performance measures of interest. The approximate formula is found to be extremely accurate for various distributions of the service times of type-2 jobs. Furthermore, the maximal occupancy of the servers as well as the maximal throughput are examined. Finally, numerical results to investigate the impact of each parameter on system performance are conducted.

Analysis of the M/G/1 processor-sharing queue with bulk arrivals

We analyze the single server processor-sharing queue for the case of bulk arrivals. We obtain an expression for the expected response time of a job as a function of its size, when the service times of jobs have a generalized hyperexponential distribution and more generally for distributions with rational Laplace transforms. Our analysis significantly extends the class of distributions for which processor-sharing queues with bulk arrivals were previously analyzed.

Performance of Adaptive Space Sharing Processor Allocation Policies for Distributed-Memory Multicomputers

Several space sharing policies have been proposed for distributed-memory multicomputer systems. We consider adaptive space sharing policies, as these policies provide a better performance than fixed and static policies by taking system load and user requirements into account. In this paper we propose an improved space sharing policy by suggesting a simple modification to a previously proposed policy. We study performance sensitivity of the original and modified policies to job structure and various other system and workload parameters like variances in inter-arrival times and job service times. The results presented here demonstrate that the modified policy performs substantially better than the original policy.

Modelling And Analyses Of Dynamic Load-Balancing Policies By State Aggregation

Exact performance analysis of dynamic load-balancing policies for distributed systems is known to be very difficult owing to the facts that the state space is multidimensional and that load-balancing decisions are state dependent. In this paper, a state-aggregation method is proposed to model and analyze dynamic load-balancing policies. Those states with the same number of jobs are aggregated into a single state. The number of jobs in the system is modelled by a birth-death Markov process. The state transition rates are estimated by an iterative procedure. The procedure is developed according to the operation of the load-balancing policy. The proposed method is used to analyze the performance of a dynamic load-balancing policy, namely, symmetric policy. Extensive simulations are performed to study the accuracy of the method. This method provides accurate performance estimates for the symmetric policy for systems of various sizes when the mean job transfer delay is small compared to the average job service time.

Waiting time distributions for closed M/M/N processor sharing queues

We consider a multiple server processor sharing model with a finite number of terminals (customers). Each terminal can submit at most one job for service at any time. The think times of the terminals and the service time demands are independently exponentially distributed. We focus our attention on the exact detailed analysis of the waiting time distribution of a tagged job. We give the Laplace-Stieltjes transform of the waiting time distribution conditioned on the job's service time demand and the state of the other terminals and show that these transforms can be efficiently evaluated and inverted. Further results include the representation of conditioned waiting times as mixtures of a constant and several exponentially distributed components. The numerical precision of our results is being compared with results from a discrete approximation of the waiting time distributions.

Optimal Multiserver Stochastic Scheduling of two Interconnected Priority Queues

A number of jobs on two interconnected queues are to be processed by m identical servers. The servers operate in parallel, so that every server can process any job. Jobs in queue i, i = 1, 2, incur an instantaneous holding cost Ci during the time they remain in the system. The service time for jobs in queue i, denoted by Xi , is a random variable with a general distribution. The interconnection process is independent of the service process. We establish sufficient conditions on the service times, the holding costs and the interconnection process under which the non-preemptive scheduling strategy that gives priority to queue 1 minimizes the total expected α -discounted cost. We call this strategy P1. We present counterexamples showing that if any of the sufficient conditions is not satisfied P1 may not be optimal, and that the optimal policy for the single-server problem is not necessarily optimal for the multiserver problem.

Optimal Multiserver Stochastic Scheduling of two Interconnected Priority Queues

A number of jobs on two interconnected queues are to be processed by m identical servers. The servers operate in parallel, so that every server can process any job. Jobs in queue i, i = 1, 2, incur an instantaneous holding cost Ci during the time they remain in the system. The service time for jobs in queue i, denoted by Xi, is a random variable with a general distribution. The interconnection process is independent of the service process. We establish sufficient conditions on the service times, the holding costs and the interconnection process under which the non-preemptive scheduling strategy that gives priority to queue 1 minimizes the total expected α -discounted cost. We call this strategy P1. We present counterexamples showing that if any of the sufficient conditions is not satisfied P1 may not be optimal, and that the optimal policy for the single-server problem is not necessarily optimal for the multiserver problem.