Sojourn Time Processes Research Articles

A Sojourn-Based Approach to Semi-Markov Reinforcement Learning

In this paper we introduce a new approach to discrete-time semi-Markov decision processes based on the sojourn time process. Different characterizations of discrete-time semi-Markov processes are exploited and decision processes are constructed by their means. With this new approach, the agent is allowed to consider different actions depending also on the sojourn time of the process in the current state. A numerical method based on Q-learning algorithms for finite horizon reinforcement learning and stochastic recursive relations is investigated. Finally, we consider two toy examples: one in which the reward depends on the sojourn-time, according to the gambler’s fallacy; the other in which the environment is semi-Markov even if the reward function does not depend on the sojourn time. These are used to carry on some numerical evaluations on the previously presented Q-learning algorithm and on a different naive method based on deep reinforcement learning.

A functional law of the iterated logarithm for multi-class queues with batch arrivals

A functional law of the iterated logarithm (LIL) and its corresponding LIL are established for a multiclass single-server queue with first come first served (FCFS) service discipline. The functional LIL and its LIL quantify the magnitude of asymptotic stochastic fluctuations of the stochastic processes compensated by their deterministic fluid limits. The functional LIL and LIL are established in three cases: underloaded, critically loaded and overloaded, for performance measures including the total workload, idle time, queue length, workload, busy time, departure and sojourn time processes. The proofs of the functional LIL and LIL are based on a strong approximation approach, which approximates discrete performance processes with reflected Brownian motions. Numerical examples are considered to provide insights on these limit results.

The (functional) law of the iterated logarithm of the sojourn time for a multiclass queue

Two types of the law of iterated logarithm (LIL) and one functional LIL (FLIL) are established for the sojourn time process for a multiclass queueing model, having a priority service discipline, one server and $K$ customer classes, with each class characterized by a batch renewal arrival process and independent and identically distributed (i.i.d.) service times. The LIL and FLIL limits quantify the magnitude of asymptotic stochastic fluctuations of the sojourn time process compensated by its deterministic fluid limits in two forms: the numerical and functional. The LIL and FLIL limits are established in three cases: underloaded, critically loaded and overloaded, defined by the traffic intensity. We prove the results by a approach based on strong approximation, which approximates discrete performance processes with reflected Brownian motions. We conduct numerical examples to provide insights on these LIL results.

A Law of the Iterated Logarithm for the Sojourn Time Process in Queues in Series

The object of this research in the sphere of queueing theory is the law of the iterated logarithm under the conditions of heavy traffic in queues in series. In this paper, the laws of the iterated logarithm are proved for the values of important probabilistic characteristics of the queueing system, like the sojourn time of a customer, and maximum of the sojourn time of a customer. Also, we prove that the sojourn time of a customer can be approximated by some recurrent functional. We also provide the results of statistical simulations for various system parameters and distributions.

In This Issue

In This Issue

Strong approximations for a Kumar-Seidman network under a priority service discipline

In this paper, we derive the strong approximations for a four-class two station multi-class queuing network (Kumar-Seidman network) under a priority service discipline. Within a group, jobs are served in the order of arrival, that is, a first-in-first-out disciple, and among groups, jobs are served under a pre-emptiveresume priority disciple. We show that if the input data (i.e., the arrival and service processe) satisfy an approximation (such as the functional law-of-iterated logarithm approximation or the strong approximation), the output data (the departure processes) and the performance measures (such as the queue length, the work load and the sojourn time process) satisfy a similar approximation.

Some Local Asymptotic Laws for the Cauchy Process on the Line

This paper investigates the lim inf behavior of the sojourn time process and the escape rate process associated with the Cauchy process on the line. The monotone functions associated with the lower asymptotic growth rate of the sojourn time are characterized and the asymptotic size of the large values of the escape rate process is developed.

Sample path large deviations for multiclass feedforward queueing networks in critical loading

We consider multiclass feedforward queueing networks with first in first out and priority service disciplines at the nodes, and class dependent de terministic routing between nodes. The random behavior of the network is constructed from cumulative arrival and service time processes which are as sumed to satisfy an appropriate sample path large deviation principle. We establish logarithmic asymptotics of large deviations for waiting time, idle time, queue length, departure and sojourn-time processes in critical loading. This transfers similar results from Puhalskii about single class queueing net works with feedback to multiclass feedforward queueing networks, and com plements diffusion approximation results from Peterson. An example with renewal inter arrival and service time processes yields the rate function of a reflected Brownian motion. The model directly captures stationary situations.

Fractional Brownian Heavy Traffic Approximations of Multiclass Feedforward Queueing Networks

We consider multiclass feedforward queueing networks under first in first out and priority service disciplines driven by long-range dependent arrival and service time processes. We show that in critical loading the normalized workload, queue length and sojourn time processes can converge to a multi-dimensional reflected fractional Brownian motion. This weak heavy traffic approximation is deduced from a deterministic pathwise approximation of the network behavior close to constant critical load in terms of the solution of a Skorokhod problem. Since we model the doubly infinite time interval, our results directly cover the stationary case.

Strong approximations for multiclass feedforward queueing networks

This paper derives the strong approximation for a multiclass queueing network,where jobs after service completion can only move to a downstream service station. Job classes are partitioned into groups. Within a group, jobs are served in the order of arrival; that is, a first-in first-out (FIFO) discipline is in force, and among groups, jobs are served under a preassigned preemptive priority discipline. We obtain the strong approximation for the network through an inductive application of an input–output analysis for a single-station queue. Specifically, we show that if the input data (i.e., the arrival and the service processes) satisfy an approximation (such as the functional law-of-iterated logarithm approximation or the strong approximation), then the output data (i.e., the departure processes) and the performance measures (such as the queue length, the workload and the sojourn time processes) satisfy a similar approximation. Based on the strong approximation, some procedures are proposed to approximate the stationary distribution of various performance measures of the queueing network. Our work extends and complements the existing work of Peterson and Harrison and Williams on the feedforward queueing network. The numeric examples show that strong approximation provides a better approximation than that suggested by a straightforward interpretation of the heavy traffic limit theorem.

Sojourn times in queues with instantaneous tri-route decision process

In this paper, we study the total sojourn time in a queueing system with an instantaneous tri-route decision process. Even though the computations are more difficult, we give here the structure of the sojourn time process for the M/ G/1 queue with tri-route decision process. A numerical study is carried out in this paper.

Palm calculus for a process with a stationary random measure and its applications to fluid queues

We consider a process associated with a stationary random measure, which may have infinitely many jumps in a finite interval. Such a process is a generalization of a process with a stationary embedded point process, and is applicable to fluid queues. Here, fluid queue means that customers are modeled as a continuous flow. Such models naturally arise in the study of high speed digital communication networks. We first derive the rate conservation law (RCL) for them, and then introduce a process indexed by the level of the accumulated input. This indexed process can be viewed as a continuous version of a customer characteristic of an ordinary queue, e.g., of the sojourn time. It is shown that the indexed process is stationary under a certain kind of Palm probability measure, called detailed Palm. By using this result, we consider the sojourn time processes in fluid queues. We derive the continuous version of Little's formula in our framework. We give a distributional relationship between the buffer content and the sojourn time in a fluid queue with a constant release rate.

Sojourn times in queues with a pair of instantaneous independent bernoulli feedback

In this paper we study the total sojourn time in a queueing system with a pair of instantaneous independent Bernoulli feedback processes. As the details of the computations are difficult, we give only the structure of the sojourn time process for the M/G2/l queue with a pair of Bernoulli feedback. We have also carried out a numerical study in this paper

A Heavy Traffic Limit Theorem for Networks of Queues with Multiple Customer Types

The central result of this paper is a heavy traffic limit theorem for the vector of total station workloads in an open network of queues with multiple customer types, under first-come-first-served and priority disciplines. The limit process is a regulated Brownian motion on the nonnegative orthant, with parameters specified from the first two moments of the interarrival and service time distributions and a matrix of reduced routing information. Through the phenomenon of state space collapse, associated limit results for queue length, workload and sojourn time processes by customer type are obtained jointly as simple transformations of the total workload limit process. Diffusion approximations based on the theorem are discussed.

A network of priority queues in heavy traffic: One bottleneck station

In this paper we consider an open queueing network having multiple classes, priorities, and general service time distributions. In the case where there is a single bottleneck station we conjecture that normalized queue length and sojourn time processes converge, in the heavy traffic limit, to one-dimensional reflected Brownian motion, and present expressions for its drift and variance. The conjecture is motivated by known heavy traffic limit theorems for some special cases of the general model, and some conjectured “Heavy Traffic Principles” derived from them. Using the known stationary distribution of one-dimensional reflected Brownian motion, we present expressions for the heavy traffic limit of stationary queue length and sojourn time distributions and moments. For systems with Markov routing we are able to explicitly calculate the limits.

A multiclass feedback queue in heavy traffic

We consider a single station queueing system with several customer classes. Each customer class has its own arrival process. The total service requirement of each customer is divided into a (possibly) random number of service quanta, where the distribution of each quantum may depend on the customer's class and the other quanta of that customer. The service discipline is round-robin, with random quanta.We prove a heavy traffic limit theorem for the above system which states that as the traffic intensity approaches unity, properly normalized sequences of queue length and sojourn time processes converge weakly to one-dimensional reflected Brownian motion.

A multiclass feedback queue in heavy traffic

We consider a single station queueing system with several customer classes. Each customer class has its own arrival process. The total service requirement of each customer is divided into a (possibly) random number of service quanta, where the distribution of each quantum may depend on the customer's class and the other quanta of that customer. The service discipline is round-robin, with random quanta. We prove a heavy traffic limit theorem for the above system which states that as the traffic intensity approaches unity, properly normalized sequences of queue length and sojourn time processes converge weakly to one-dimensional reflected Brownian motion.

Open Queueing Networks in Heavy Traffic

This paper presents heavy traffic limit theorems for the queue length and sojourn time processes associated with open queueing networks. These limit theorems state that properly normalized sequences of queue length and sojourn time processes converge weakly to a certain diffusion as the network traffic intensity converges to unity. The limit diffusion is reflected Brownian motion on the nonnegative orthant. This process behaves like Brownian motion on the interior of its state space, and reflects instantaneously on the boundaries. The reflection direction is a constant for each boundary hyperplane.

A note on sojourn times in M/G/1 queues with instantaneous, bernoulli feedback

AbstractQueueing systems which include the possibility for a customer to return to the same server for additional service are called queueing systems with feedback. Such systems occur in computer networks for example. In these systems a chosen customer will wait in the queue, be serviced and then, with probability p, return to wait again, be serviced again and continue this process until, with probability (1 – p) = q, it departs the system never to return. The time of waiting plus service time, the nth time the customer goes through, we will call his nth sojourn time. The (random) sum of these sojourn times we will call the total sojourn time (abbreviated, sojourn time when there is no confusion which sojourn time we are talking about). In this paper we study the total sojourn time in a queueing system with feedback. We give the details for M/G/1 queues in which the decision to feedback or not is a Bernoulli process. While the details of the computations can be more difficult, the structure of the sojourn time process is unchanged for the M/G/1 queue with a more general decision process as will be shown. We assume the reader is familiar with Disney, McNickle and Simon [1].