Type Markov Chains Research Articles

Stability of Queueing Systems with Impatience, Balking and Non-Persistence of Customers

The operation of many queueing systems is adequately described by the structured multidimensional continuous-time Markov chains. The most well-studied classes of such chains are level-independent Quasi-Birth-and-Death processes, GI/M/1 type and M/G/1 type Markov chains, generators of which have the block tri-diagonal, lower- and upper-Hessenberg structure, respectively. All these classes assume that the matrices of transition rates are quasi-Toeplitz. This property greatly simplifies their analysis but makes them inappropriate for the study of many important systems, e.g., retrial queues with a retrial rate depending on the number of customers in orbit, queues with impatient customers, etc. The importance of such systems attracts significant interest to their analysis. However, in the literature, there is a methodological gap relating to the ergodicity condition of the corresponding Markov chains. To fulfill this gap and facilitate the analysis of a wide range of such systems, we show that under non-restrictive assumptions, the following hold true: (i) if the customers can balk or are impatient or non-persistent, then the Markov chain describing the behavior of the system belongs to the class of asymptotically quasi-Toeplitz Markov chains; (ii) this chain is ergodic; (iii) known algorithms can be applied for the calculation of the stationary distribution of the corresponding queueing system.

A Markov-Dependent stochastic approach to modeling lactation curves in dairy cows

The modeling of lactation curves is an essential aspect of formulating farm managerial practices in dairy cows. In this study, we propose and examine a Markov-Dependent stochastic approach to modeling lactation curves in dairy cows, with the aim of developing a model that accurately fits lactation curves for a maximum number of lactations. Specifically, we develop a special type of Gamma Type Markov Chain Model that considers the first-order linear regressive property, which makes the model more realistic and reliable. We compared the proposed model with three other models - quadratic model, mixed log function, and wood model - using various goodness of fit measures such as adjusted R2, root mean square error (RMSE), and Bayesian Information Criteria (BIC). Our results showed that lactation curve modeling using the proposed model could help set management strategies at the farm level. However, it is important to optimize the modeling process regularly before implementing these strategies to enhance productivity in dairy cows. Our study contributes to the existing literature by proposing a novel approach that accounts for Markov dependence and linear regression in modeling lactation curves, which can lead to more accurate and reliable predictions. This modeling approach has practical implications for dairy farmers who seek to maximize productivity and efficiency while minimizing costs.

A Markov Chain Model for Determining the Optimal Time to Move Pregnant Cows to Individual Calving Pens.

The use of individual calving pens in modern farming is widely recognized as a good practice for promoting good animal welfare during parturition. However, determining the optimal time to move a pregnant cow to a calving pen can be a management challenge. Moving cows too early may result in prolonged occupancy of the pen, while moving them too late may increase the risk of calving complications and production-related diseases. In this paper, a simple random walk type Markov Chain Model to predict the optimal time for moving periparturient cows to individual calving pens was proposed. Behavior changes such as lying time, standing time, and rumination time were analyzed using a video monitoring system, and we formulated these changes as the states of a Markov Chain with an absorbing barrier. The model showed that the first time entering an absorbing state was the optimal time for a pregnant cow to be moved to a calving pen. The proposed method was validated through a series of experiments in a real-life dairy farm, showing promising results with high accuracy.

Analysis of the axial stability for an assembly of optical modes with stochastic fluctuations type Markov chain

We describe the engineering of optical modes whose axial structure follows fluctuations of Markov-chain-type.

Moran-type bounds for the fixation probability in a frequency-dependent Wright-Fisher model.

We study stochastic evolutionary game dynamics in a population of finite size. Individuals in the population are divided into two dynamically evolving groups. The structure of the population is formally described by a Wright-Fisher type Markov chain with a frequency dependent fitness. In a strong selection regime that favors one of the two groups, we obtain qualitatively matching lower and upper bounds for the fixation probability of the advantageous population. In the infinite population limit we obtain an exact result showing that a single advantageous mutant can invade an infinite population with a positive probability. We also give asymptotically sharp bounds for the fixation time distribution.

Denker–Sato type Markov chains and Harnack inequality

In ([DS1], [DS2], [DS3]), Denker and Sato studied a Markov chain on the finite words space of the Sierpinski gasket (SG). They showed that the Martin boundary is homeomorphic to the SG. Recently, Lau and Wang (2015 Math. Z. 280 401–20) showed that the homeomorphism holds for an iterated function system with the open set condition provided that the transition probability on the finite words space is of DS-type. In this work, we continue studying this kind of transition probability on the unit interval. Using matrix expressions, we obtain a formula to calculate the Green function. By the ergodic arguments for non-negative matrices, we find that the Martin boundary is homeomorphic to the unit interval or the union of the unit interval and a countable set. This gives a good illustration for the results in Lau and Wang (2015 Math. Z. 280 401–20).

Construction of Markov chains for discrete time MAP/PH/K queues

Two GI/M/1 type Markov chains associated with the queue length are often used in analyzing the discrete time MAP/PH/K queue. The first Markov chain is introduced by tracking service phases for servers; a method we call TPFS. The transition probability matrix of the Markov chain can be constructed in a straightforward manner. The second Markov chain is introduced by counting servers for phases; which we call CSFP. An algorithm is developed for the construction of the transition probability matrix of the second Markov chain, which is the main contribution of this paper. Whereas the construction of the matrices for the case of continuous time is available in the literature, it is not available for the discrete time case. The effort in constructing the matrices for the discrete time case is extensively more involved than for the continuous time case. Some basic properties of the constructed transition blocks are shown. We demonstrate that for queueing systems with a large number of servers and many service phases, there is a considerable saving in the matrix sizes. For example, when those values are 30 and 2, respectively, the block size for TPFS is more than 3×107 times that of CSFP; a major saving.

Tail probability and singularity of Laplace-Stieltjes transform of a Pareto type random variable

We give a sufficient condition for a non-negative random variable X to be of Pareto type by investigating the Laplace-Stieltjes transform of the cumulative distribution function. We focus on the relation between the singularity at the real point of the axis of convergence and the asymptotic decay of the tail probability. For the proof of our theorems, we apply Graham-Vaaler’s complex Tauberian theorem. As an application of our theorems, we consider the asymptotic decay of the stationary distribution of an M/G/1 type Markov chain.

Denker–Sato type Markov chains on self-similar sets

In Denker and Sato (Potential Anal 14: 211–232, 2001; Publ RIMS 35: 769–794, 1999; Math Nachr 241: 32–55, 2002) studied certain Markov chain on the symbolic spaces of the Sierpinski gasket (SG). They showed that the Martin boundary is homeomorphic to SG, and used the potential theory on the Martin boundary to induce a harmonic structure on SG. In this paper, we consider a more general Denker–Sato type Markov chain associated with self-similar sets $$K$$ with the open set condition. The chain is defined on the augmented tree of the symbolic space. Such tree was introduced by Kaimanovich, it is hyperbolic in the sense of Gromov (Kaimanovich in Random walks on Sierpinski graphs: hyperbolicity and stochastic homogenization, Birhauser, Basel, 2003; Lau and Wang in Indiana Univ Math J 58:1777–1795, 2009). We show that the Martin boundary, the hyperbolic boundary and the self-similar set $$K$$ are homeomorphic. The hitting distribution of the chain is also obtained.

Logarithmic Asymptotics for the GI/G/1-type Markov Chains and their Applications to the BMAP/G/1 Queue with Vacations

We study tail asymptotics of the stationary distribution for the -type Markov chain with finitely many background states. Decay rate in the logarithmic sense is identified under a number of conditions on the transition probabilities. The results are applied to the queue with vacations. The relationship between vacation time and the decay rate of the queue length distribution is investigated.

SMCSolver and Q-MAM

Matrix-analytic methods have advanced considerably since the pioneering work of Marcel Neuts [6, 5] on Quasi-Birth-Death (QBD), GI/M/1- and M/G/1- type Markov chains (MCs). Especially the algorithms involved to (iteratively) solve these structured Markov chains have matured a lot, which has resulted in more efficient, but also more complex algorithms [4, 1]. While the first algorithms were straightforward to implement---as they were based on simple functional iterations---more advanced algorithms/features like cyclic-reduction, the Newton iteration or the shift technique (to accelerate convergence), require more effort; in particular for GI/M/1- and M/G/1-type Markov chains. This has motivated us to develop the Structured Markov Chain Solver (SMCSolver) tool [2], which implements a large number of basic and more advanced algorithms for solving QBD, GI/M/1- and M/G/1-type MCs1 (as well as the more general Non-Skip-Free M/G/1-type MCs). The MATLAB version of the tool consists of a collection of MATLAB functions, while the Fortran version is accompanied by a graphical user-interface (GUI). Apart from making these more advanced algorithms accessible to non-specialists, the tool is also useful as a platform for the development and study of new algorithms and acceleration techniques. Since its initial release in 2006, various extensions have been made. In [3] different transformation techniques and shift strategies are incorporated in order to speed up and optimize the algorithms, while even more recently an efficient Newton iteration for GI/M/1- and M/G/1-type Markov chains was included [8]. Matrix-analytic methods have also been very effective in the analysis of many queueing systems in both discrete- and continuous-time. The Q-MAM tool [7] is a collection of MATLAB functions that allows one to compute the queue length, waiting time and delay distribution of various queueing systems of infinite size. It includes amongst others implementations of the PH/PH/1, MAP/MAP/1, MAP/M/c, MAP/D/c, RAP/RAP/1, MMAP[K]/PH[K]/1, MMAP[K]/SM[K]/1, SM[K]/PH[K]/1 (many in both discrete- and continuous-time), where state-of-the-art solution techniques are used to solve these models efficiently. The Matlab version of the SMCSolver and Q-MAM tool is available at http://win.ua.ac.be/?vanhoudt/ while the Fortran 90 version of the SMCSolver tool with the GUI can be downloaded from http://bezout.dm.unipi.it/SMCSolver.

A compressed cyclic reduction for QBDs with low rank upper and lower transitions

Consider a quasi-birth-and-death (QBD) Markov chain [6], having probability transition matrix where Bi, Ai, i = ?1, 0, 1, are m x m matrices. In the numerical solution of QBD Markov chains a crucial step is the efficient computation of the minimal nonnegative solution R of the quadratic matrix equation X = X 2 A?1 + XA 0 + A 1 . (1) To this purpose, many numerical methods, with different properties, have been designed in the last years (see for instance [1, 2, 3, 4]). However, many of these numericalmethods are defined for general block coefficients A?1, A0 and A1, and do not exploit the possible structure of these blocks. Recently, some attention has been addressed to the case where A ?1 has only few non-null columns, or A1 has only few non-null rows. These properties are satisfied when the QBD has restricted transitions to higher (or lower) levels. In particular, in [7] the authors exploit these properties of the matrix A ?1 , or A 1 , to formulate the QBD in terms of an M/G/1 type Markov chain, where the block matrices have size smaller than m; in particular, when both A?1 and A1 have the desired property, the latter M/G/1 type Markov chain reduces to a QBD. In [5] the structure of A ?1 is used in order to reduce the computational cost of some algorithms for computing R. Here we assume that both the matrices A ?1 and A 1 have small rank with respect to their size m. In particular, if A?1 and A1 have only few non-null columns and rows, respectively, they have small rank. We show that, under this assumption, the matrix R can be computed by using the cyclic reduction algorithm, where the matrices A(k) i , i = ?1, 0, 1, generated at the kth step of the algorithm, can be represented by small rank matrices. In particular, if r ?1 is the rank of A ?1 , and if r 1 is the rank of A 1 , then each step of cyclic reduction can be performed by means of O((r ?1+r1 ) 3 ) arithmetic operations. This cost estimate must be compared with the cost of O(m3) arithmetic operations, needed without exploiting the structure of A ?1 and A 1 . Therefore, if r 1 and r 1 /are much smaller than m, the advantage is evident. It remains an open issue to understand how the structure can be exploited in the case where only one between A ?1 and A 1 has low rank.

A note on the subexponential asymptotics of the stationary distribution of M/G/1 type Markov chains

Masuyama (2011) obtained the subexponential asymptotics of the stationary distribution of an M/G/1 type Markov chain under the assumption related to the periodic structure of G-matrix. In this note, we improve Masuyama’s result by showing that the subexponential asymptotics holds without the assumption related to the periodic structure of G-matrix.

Note on functional iteration technique for [formula omitted] type Markov chains

A thorough theoretical explanation of the numerical behaviour of functional iteration methods for the computation of the minimal nonnegative solution G of the matrix equation X = ∑ i = 0 ∞ X i A i , arising in the numerical solution of M / G / 1 type Markov chains, is given in Meini (1997) [2]. In this note we add some more results. In particular, we show that an upper bound of the mean asymptotic convergence rate of the best functional iteration method is given in terms of the second largest modulus eigenvalue of G .

Computational Procedures for a Class of GI/D/kSystems in Discrete Time

A class of discrete time GI/D/ksystems is considered for which the interarrival times have finite support and customers are served in first-in first-out (FIFO) order. The system is formulated as a single server queue with new general independent interarrival times and constant service duration by assuming cyclic assignment of customers to the identical servers. Then the queue length is set up as a quasi-birth-death (QBD) type Markov chain. It is shown that this transformed GI/D/1 system has special structures which make the computation of the matrixRsimple and efficient, thereby reducing the number of multiplications in each iteration significantly. As a result we were able to keep the computation time very low. Moreover, use of the resulting structural properties makes the computation of the distribution of queue length of the transformed system efficient. The computation of the distribution of waiting time is also shown to be simple by exploiting the special structures.

A probabilistic interpretation of cyclic reduction and its relationships with logarithmic reduction

We provide a simple convergence proof for the cyclic reduction algorithm for M/G/1 type Markov chains together with a probabilistic interpretation which helps to understand better the relationships between logarithmic reduction and cyclic reduction.

Algorithmic analysis of the discrete time GIX/ GY/1 queueing system

In this paper, we study the discrete time G I X / G Y / 1 queueing system. First, some general results are obtained for the stability condition, stationary distributions of the queue lengths and waiting times. Then we show that, for some practical situations, the queueing system of interest has the properties of both the M / G / 1 and G I / M / 1 non-skip-free Markov chains described by [H.R. Gail, S.L. Hantler, B.A. Taylor, Non-skip-free M / G / 1 and G I / M / 1 types of Markov chains, Advances in Applied Probability, 29 (3) (1997) 733–758]. For such cases, the queueing system can be analyzed as a QBD process, a M / G / 1 type Markov chain, or a G I / M / 1 type Markov chain with finite blocks after re-blocking. Computational procedures are developed for computing a number of performance measures for the queueing system of interest. In addition, we study a G I / M / 1 type Markov chain associated with the age process of the customers in service.

Rational Generating Functions and Matrix-Geometric Invariant Measures

The paper studies the connection between rational generating matrix-valued functions and not necessarily nonnegative matrix-geometric invariant measures for M/G/1 and G/M/1 type Markov chains. It characterizes the situation of the existence, and shows that in the transient QBD case not every positive invariant measure is matrix-geometric. A method for the determination of all invariant measures in the QBD case is presented. The questions of summability and generalized notions of matrix-geometricity in the spirit of system theory are studied. Main conclusion: invariant measures are not always matrix-geometric, but are very often generalized matrix-geometric.

Geometric Decay in a QBD Process with Countable Background States with Applications to a Join-the-Shortest-Queue Model

A geometric tail decay of the stationary distribution has been recently studied for the GI/G/1 type Markov chain with both countable level and background states. This method is essentially the matrix analytic approach, and simplicity is an obvious advantage of this method. However, so far it can be only applied to the α-positive case (or the jittered case, as referred to in the literature). In this paper, we specialize the GI/G/1 type to a quasi-birth-and-death process. This not only refines some expressions because of the matrix geometric form for the stationary distribution, but also allows us to extend the study, in terms of the matrix analytic method, to non-α-positive cases. We apply the result to a generalized join-the-shortest-queue model, which only requires elementary computations. The obtained results enable us to discuss when the two queues are balanced in the generalized join-the-shortest-queue model, and establish the geometric tail asymptotics along the direction of the difference between the two queues.

Age Process, Workload Process, Sojourn Times, and Waiting Times in a Discrete Time SM[K]/PH[K]/1/FCFS Queue

In this paper, we study a discrete time queueing system with multiple types of customers and a first-come-first-served (FCFS) service discipline. Customers arrive according to a semi-Markov arrival process and the service times of individual customers have PH-distributions. A GI/M/1 type Markov chain for a generalized age process of batches of customers is introduced. The steady state distribution of the GI/M/1 type Markov chain is found explicitly and, consequently, the steady state distributions of the age of the batch in service, the total workload in the system, waiting times, and sojourn times of different batches and different types of customers are obtained. We show that the generalized age process and a generalized total workload process have the same steady state distribution. We prove that the waiting times and sojourn times have PH-distributions and find matrix representations of those PH-distributions. When the arrival process is a Markov arrival process with marked transitions, we construct a QBD process for the age process and the total workload process. The steady state distributions of the waiting times and the sojourn times, both at the batch level and the customer level, are obtained from the steady state distribution of the QBD process. A number of numerical examples are presented to gain insight into the waiting processes of different types of customers.