Stationary Probability Vector Research Articles

Product form solution for some queueing-inventory supply chain problem

In this paper we analyze a single server supply chain model in which stocks are kept in both the manufacturer warehouse (production centre) and the retail shop (distribution centre). Arrival of customers to the retail shop form a Poisson process and their service time are exponentially distributed. The maximum stock of the distribution centre is limited to s + Q(=S). When the inventory level depletes to s due to services, it demands Q units at a time from the production centre. The lead time follows an exponential distribution. If the production centre has the required stock on-hand, the items are supplied. Supply of items from the production centre to the distribution centre is done only as a packet of Q units at a time. So if a packet of size Q is not available the distribution centre has to wait till Q units accumulates in the production centre. The production inventory system adopts a (rQ, KQ) policy where the processing of inventory requires a positive random amount of time. Production time for unit item is exponentially distributed. Also we assume that no customer joins the queue when the inventory level in the distribution centre is zero. This assumption leads to an explicit product form solution for the steady state probability vector.

M/C2/c/K queuing model and optimization for a geriatric care center

Queuing systems with finite buffers are reasonable models for many manufacturing, telecommunication, and healthcare systems. Although some approximations exist, the exact analysis of multi‐server and finite‐buffer queues with general service time distribution is unknown. However, the phase‐type assumption for service time is a frequently used approach. Because the Cox distribution, a kind of phase‐type distribution, provides a good representation of data with great variability, it has a vast area of application in modeling service times.The research focus is twofold. First, a theoretical structure of a multi‐server and finite‐buffer queuing system in which the service time is modeled by the two‐phase Cox distribution is studied. It is focused on finding an efficient solution to the stationary probabilities using the matrix‐geometric method. It is shown that the stationary probability vector can be obtained with the matrix‐geometric method by using level‐dependent rate matrices, and the mean queue length is computed. Second, an empirical analysis of the model is presented. The proposed methodology is applied in a case study concerning the geriatric patients. Some numerical calculations and optimizations are performed by using geriatric data. Copyright © 2015 John Wiley & Sons, Ltd.

Large Deviations for Stationary Probabilities of a Family of Continuous Time Markov Chains via Aubry–Mather Theory

In the present paper, we consider a family of continuous time symmetric random walks indexed by \(k\in \mathbb {N}\), \(\{X_k(t),\,t\ge 0\}\). For each \(k\in \mathbb {N}\) the matching random walk take values in the finite set of states \(\Gamma _k=\frac{1}{k}(\mathbb {Z}/k\mathbb {Z})\); notice that \(\Gamma _k\) is a subset of \(\mathbb {S}^1\), where \(\mathbb {S}^1\) is the unitary circle. The infinitesimal generator of such chain is denoted by \(L_k\). The stationary probability for such process converges to the uniform distribution on the circle, when \(k\rightarrow \infty \). Here we want to study other natural measures, obtained via a limit on \(k\rightarrow \infty \), that are concentrated on some points of \(\mathbb {S}^1\). We will disturb this process by a potential and study for each \(k\) the perturbed stationary measures of this new process when \(k\rightarrow \infty \). We disturb the system considering a fixed \(C^2\) potential \(V: \mathbb {S}^1 \rightarrow \mathbb {R}\) and we will denote by \(V_k\) the restriction of \(V\) to \(\Gamma _k\). Then, we define a non-stochastic semigroup generated by the matrix \(k\,\, L_k + k\,\, V_k\), where \(k\,\, L_k \) is the infinifesimal generator of \(\{X_k(t),\,t\ge 0\}\). From the continuous time Perron’s Theorem one can normalized such semigroup, and, then we get another stochastic semigroup which generates a continuous time Markov Chain taking values on \(\Gamma _k\). This new chain is called the continuous time Gibbs state associated to the potential \(k\,V_k\), see (Lopes et al. in J Stat Phys 152:894–933, 2013). The stationary probability vector for such Markov Chain is denoted by \(\pi _{k,V}\). We assume that the maximum of \(V\) is attained in a unique point \(x_0\) of \(\mathbb {S}^1\), and from this will follow that \(\pi _{k,V}\rightarrow \delta _{x_0}\). Thus, here, our main goal is to analyze the large deviation principle for the family \(\pi _{k,V}\), when \(k \rightarrow \infty \). The deviation function \(I^V\), which is defined on \( \mathbb {S}^1\), will be obtained from a procedure based on fixed points of the Lax–Oleinik operator and Aubry–Mather theory. In order to obtain the associated Lax–Oleinik operator we use the Varadhan’s Lemma for the process \(\{X_k(t),\,t\ge 0\}\). For a careful analysis of the problem we present full details of the proof of the Large Deviation Principle, in the Skorohod space, for such family of Markov Chains, when \(k\rightarrow \infty \). Finally, we compute the entropy of the invariant probabilities on the Skorohod space associated to the Markov Chains we analyze.

An Accelerated Two-level Multigrid Method for Markov Chains

Abstract Based on the quadratic extrapolation method and its generalization, this paper presents an accelerated two-level multigrid method for speeding up the numerical computation of the stationary probability vector of an irreducible Markov chain. It shows how to combine these vector extrapolation methods with the two-level multigrid method on the coarse level in detail. Numerical results on two Markov chain problems are provided to illustrate the effectiveness of our proposed method in terms of reducing the iteration counts and computing time.

Queues with two units connected in series with a multiserver bulk service with accessible and non accessible batch in unit II

This paper analyses a queueing model consisting of two units I and II connected in series, separated by a finite buffer of size N. Unit I has only one exponential server capable of serving customers one at a time. Unit II consists of c parallel exponential servers and they serve customers in groups according to the bulk service rule. This rule admits each batch served to have not less than ‘a’ and not more than ‘b’ customers such that the arriving customers can enter service station without affecting the service time if the size of the batch being served is less than ‘d’ ( a ≤ d ≤ b ). The steady stateprobability vector of the number of customers waiting and receiving service in unit I and waiting in the buffer is obtained using the modified matrix-geometric method. Numerical results are also presented. AMS Subject Classification number: 60k25 and 65k30

A Bayesian Based Method to Generate a Synergetic Land-Cover Map from Existing Land-Cover Products

Global land cover is an important parameter of the land surface and has been derived by various researchers based on remote sensing images. Each land cover product has its own disadvantages and limitations. Data fusion technology is becoming a notable method to fully integrate existing land cover information. In this paper, we developed a method to generate a synergetic global land cover map (synGLC) based on Bayes theorem. A state probability vector was defined to precisely and quantitatively describe the land cover classification of every pixel and reduce the errors caused by legends harmonization and spatial resampling. Simple axiomatic approaches were used to generate the prior land cover map, in which pixels with high consistency were regarded to be correct and then used as benchmark to obtain posterior land cover map. Validation results show that our hybrid land cover map (synGLC, the dataset is available on request) has the best overall performance compared with the existing global land cover products. Closed shrub-lands and permanent wetlands have the highest uncertainty in our fused land cover map. This novel method can be extensively applied to fusion of land cover maps with different legends, spatial resolutions or geographic ranges.

Stationary probability vectors of higher-order Markov chains

We consider the higher-order Markov chain, and characterize the second order Markov chains admitting every probability distribution vector as a stationary vector. The result is used to construct Markov chains of higher-order with the same property. We also study conditions under which the set of stationary vectors of the Markov chain has a certain affine dimension.

Last-Position Elimination-Based Learning Automata

An update scheme of the state probability vector of actions is critical for learning automata (LA). The most popular is the pursuit scheme that pursues the estimated optimal action and penalizes others. This paper proposes a reverse philosophy that leads to last-position elimination-based learning automata (LELA). The action graded last in terms of the estimated performance is penalized by decreasing its state probability and is eliminated when its state probability becomes zero. All active actions, that is, actions with nonzero state probability, equally share the penalized state probability from the last-position action at each iteration. The proposed LELA is characterized by the relaxed convergence condition for the optimal action, the accelerated step size of the state probability update scheme for the estimated optimal action, and the enriched sampling for the estimated nonoptimal actions. The proof of the ϵ-optimal property for the proposed algorithm is presented. Last-position elimination is a widespread philosophy in the real world and has proved to be also helpful for the update scheme of the learning automaton via the simulations of well-known benchmark environments. In the simulations, two versions of the LELA, using different selection strategies of the last action, are compared with the classical pursuit algorithms Discretized Pursuit Reward-Inaction (DP(RI)) and Discretized Generalized Pursuit Algorithm (DGPA). Simulation results show that the proposed schemes achieve significantly faster convergence and higher accuracy than the classical ones. Specifically, the proposed schemes reduce the interval to find the best parameter for a specific environment in the classical pursuit algorithms. Thus, they can have their parameter tuning easier to perform and can save much more time when applied to a practical case. Furthermore, the convergence curves and the corresponding variance coefficient curves of the contenders are illustrated to characterize their essential differences and verify the analysis results of the proposed algorithms.

Lean Blow-Out Prediction in Gas Turbine Combustors Using Symbolic Time Series Analysis

This paper develops a novel strategy for prediction of lean blowout in gas turbine combustors based on symbolic analysis of time series data from optical sensors, where the range of instantaneous data is partitioned into a finite number of cells and a symbol is assigned to each cell. Depending on the cell to which a data point belongs, a symbolic value is assigned to the data point. Thus, the set of time series data is converted to a symbol string. The (estimated) state probability vector is computed based on the number of occurrence of each symbol over a given time span. For the purpose of detecting lean blowout in gas turbine combustors, the state probability vector obtained at a condition sufficiently away from lean blowout (reference state) is considered as the reference vector. The deviation of the current state vector from the reference state vector is used as an anomaly measure for early detection of lean blowout. The results showed that the rate of change of the anomaly measure with equivalence ratio changed significantly as the system approached lean blowout. This change in slope of the curve was observed approximately at a similar proximity to lean blowout for different operating conditions and, hence, could be used as an early lean blowout precursor. The actual location of the change of slope depended primarily on the choice of reference state. This technique performed satisfactorily over a wide range of premixing.

Stochastic decomposition in production inventory with service time

We study an (s,S) production inventory system where the processing of inventory requires a positive random amount of time. As a consequence a queue of demands is formed. Demand process is assumed to be Poisson, duration of each service and time required to add an item to the inventory when the production is on, are independent, non-identically distributed exponential random variables. We assume that no customer joins the queue when the inventory level is zero. This assumption leads to an explicit product form solution for the steady state probability vector, using a simple approach. This is despite the fact that there is a strong correlation between the lead-time (the time required to add an item into the inventory) and the number of customers waiting in the system. The technique is: combine the steady state vector of the classical M/M/1 queue and the steady state vector of a production inventory system where the service is instantaneous and no backlogs are allowed. Using a similar technique, the expected length of a production cycle is also obtained explicitly. The optimal values of S and the production switching on level s have been studied for a cost function involving the steady state system performance measures. Since we have obtained explicit expressions for the performance measures, analytic expressions have been derived for calculating the optimal values of S and s.The technique developed here could be applied to a few other problems in inventory. To substantiate this claim we analyze in detail a variant (which is discussed in Schwarz et al. (2006)) of the above problem. For that model, we assume that in a production run, production occurs only once in a cycle and the amount produced (in bulk) is sufficient to take the inventory level back to S. A brief discussion on the application of our method to inventory system with lead-time for replenishment has also been provided.

The Extrapolation-Accelerated Multilevel Aggregation Method in PageRank Computation

An accelerated multilevel aggregation method is presented for calculating the stationary probability vector of an irreducible stochastic matrix in PageRank computation, where the vector extrapolation method is its accelerator. We show how to periodically combine the extrapolation method together with the multilevel aggregation method on the finest level for speeding up the PageRank computation. Detailed numerical results are given to illustrate the behavior of this method, and comparisons with the typical methods are also made.

A New GMRES(m) Method for Markov Chains

This paper presents a class of new accelerated restarted GMRES method for calculating the stationary probability vector of an irreducible Markov chain. We focus on the mechanism of this new hybrid method by showing how to periodically combine the GMRES and vector extrapolation method into a much efficient one for improving the convergence rate in Markov chain problems. Numerical experiments are carried out to demonstrate the efficiency of our new algorithm on several typical Markov chain problems.

Stochastic decomposition in production inventory with service time (abstract only)

We study an (s, S) inventory system with positive service time (for an overview of the work reported so far in inventory with positive service time one may refer to Krishnamoorthy, Lakshmi and Manikandan: A survey on inventory models with positive service time, OPSEARCH, DOI 10.1007/s12597-010-0032-z). This leads to a queue of demands being formed. The process of demand arrival constitutes a Poisson process. The duration of each service is exponentially distributed. Our model is a supply chain where items are added to the inventory through a production process. This starts each time the inventory level goes down to s and continues to be on until inventory level reaches S with the time required to add one unit of the item into the inventory when the production is on, are independent, identically distributed exponential random variables. Further all distributions involved in this paper are assumed to be mutually independent. We assume that no customer joins the queue when the inventory level is 0. This assumption leads us to an explicit product form solution for the steady state probability vector, using a simple approach. This is despite the fact that there is a strong correlation between lead time (the time required to add an item into the inventory) and the number of customers joining the queue during the lead time (except when the inventory level is zero during which time no customer joins the queue). The technique is to combine the steady state probability vector of the classical M/M/1 queue and that of the production inventory system where each service requires negligible time and no backlogs are allowed. Using a similar technique, the expected length of a production cycle is also obtained explicitly. The optimality of the highest inventory level S and the production switching on level s has been studied using a cost function constructed using the steady state system performance measures. Since we have obtained explicit expressions for these measures, analytic expressions have been derived for the optimal values of S and s. To show that our method can be applied to other similar problems, we analyze in detail a variant of the above problem (discussed in Schwarz M, Sauer. C, Daduna, H., Kulik, R and Szekli, R: M/M/1 Queueing systems with inventory, Queueing Systems, 54,55-78, 2006). For that model, we assume that in a production run, production occurs only once in a cycle and the amount produced is sufficient to take the inventory level back to S. A brief discussion on the application of our method to inventory system with lead time for replenishment has also been provided.

A continuous-time Markov chain model and analysis for cognitive radio networks

Cognitive radio concept has been widely researched to improve the spectrum usage efficiency. Appropriate modelling of the spectrum occupancy by both licensed and unlicensed users is necessary to do clear system analysis in a cognitive framework. In this paper, a continuous-time Markov chain model is developed to better describe the radio spectrum usage. The state space vector and the transition rate matrix that completely describe the system are obtained; a steady-state analysis is performed and the stationary state probability (SSP) vector is derived. In addition, we take into account the inaccuracy of the existing spectrum sensing model (missed opportunities), and derive an improved expression for the maximum throughput of secondary users as a function of the primary user traffic parameters and the achieved opportunity ratio (AOR). The optimum sensing period that maximises AOR is also analytically obtained. The proposed model and the derived expressions were examined through numerical analysis and compared with the existing models. This model is very general and applicable to systems with N secondary users in the vicinity of the primary user.

Triangular and skew-symmetric splitting method for numerical solutions of Markov chains

In this paper, a theorem is presented to indicate that there exists a nonnegative constant ϵ ≥ 0 such that the matrix A = Q T + ϵ I is a positive-definite matrix, where I ∈ R n × n is an identity matrix and Q T ∈ R n × n is a matrix with positive diagonal and nonpositive off-diagonal elements. Then a class of triangular and skew-symmetric splitting (TSS) iteration method is applied to solve the positive-definite linear system A x = b for obtaining the stationary probability vector of an irreducible Markov chain. Theoretical analyses show that the TSS iteration method converges unconditionally to the unique solution of the linear system, with the upper bound of its contraction factor dependent only on the spectrum of the triangular part and independent of the eigenvectors of the matrices involved. Moreover, the inexact triangular and skew-symmetric splitting (ITSS) iteration method, which employs certain Krylov subspace methods as the inner iteration processes at each step of the outer TSS iteration method, is proposed to accelerate the convergence of the TSS iteration method. Numerical experiments are used to illustrate the effectiveness of the TSS and ITSS iteration methods.

Fast multilevel methods for Markov chains

SUMMARYThis paper describes multilevel methods for the calculation of the stationary probability vector of large, sparse, irreducible Markov chains. In particular, several recently proposed significant improvements to the multilevel aggregation method of Horton and Leutenegger are described and compared. Furthermore, we propose a very simple improvement of that method using an over‐correction mechanism. We also compare with more traditional iterative methods for Markov chains such as weighted Jacobi, two‐level aggregation/disaggregation, and preconditioned stabilized biconjugate gradient and generalized minimal residual method. Numerical experiments confirm that our improvements lead to significant speedup, and result in multilevel methods that are competitive with leading iterative solvers for Markov chains. Copyright © 2011 John Wiley & Sons, Ltd.

Convergence of multi-level iterative aggregation–disaggregation methods

This paper introduces an error propagation formula of a certain class of multi-level iterative aggregation–disaggregation (IAD) methods for numerical solutions of stationary probability vectors of discrete finite Markov chains. The formula can be used to investigate convergence by computing the spectral radius of the error propagation matrix for specific Markov chains. Numerical experiments indicate that the same type of the formula could be used for a wider class of the multi-level IAD methods. Using the formula we show that for given data there is no relation between convergence of two-level and of multi-level IAD methods.

THE M/G/1-TYPE MARKOV CHAIN WITH RESTRICTED TRANSITIONS AND ITS APPLICATION TO QUEUES WITH BATCH ARRIVALS

We consider M/G/1-type Markov chains where a transition that decreases the value of the level triggers the phase to a small subset of the phase space. We show how this structure—referred to as restricted downward transitions—can be exploited to speed up the computation of the stationary probability vector of the chain. To this end we define a new M/G/1-type Markov chain with a smaller block size, the G matrix of which is used to find the original chain's G matrix. This approach is then used to analyze the BMAP/PH/1 queue and the BMAP[2]/PH[2]/1 preemptive priority queue, yielding significant reductions in computation time.

Two shock and wear systems under repair standing a finite number of shocks

A shock and wear system standing a finite number of shocks and subject to two types of repairs is considered. The failure of the system can be due to wear or to a fatal shock. Associated to these failures there are two repair types: normal and severe. Repairs are as good as new. The shocks arrive following a Markovian arrival process, and the lifetime of the system follows a continuous phase-type distribution. The repair times follow different continuous phase-type distributions, depending on the type of failure. Under these assumptions, two systems are studied, depending on the finite number of shocks that the system can stand before a fatal failure that can be random or fixed. In the first case, the number of shocks is governed by a discrete phase-type distribution. After a finite (random or fixed) number of non-fatal shocks the system is repaired (severe repair). The repair due to wear is a normal repair. For these systems, general Markov models are constructed and the following elements are studied: the stationary probability vector; the transient rate of occurrence of failures; the renewal process associated to the repairs, including the distribution of the period between replacements and the number of non-fatal shocks in this period. Special cases of the model with random number of shocks are presented. An application illustrating the numerical calculations is given. The systems are studied in such a way that several particular cases can be deduced from the general ones straightaway. We apply the matrix-analytic methods for studying these models showing their versatility.

Tail asymptotics for M/M/c retrial queues with non-persistent customers

In this paper, we consider a variant of the classical M/M/c retrial queue, in which we allow non-persistent customers. When c > 1, this system does not have an explicit closed form solution for the joint stationary distribution of the number of retrial customers in the orbit and the number of busy servers. Our main focus is on the tail asymptotics for the joint probabilities. We first present a matrix-product solution for the joint stationary probability vectors, which is further simplified to a scalar-product form, according to matrix-analytic theory. We then apply the censoring technique, which has been proven an efficient approach for analyzing queueing systems including retrial queues, to obtain the censored equations and the Key Lemma. In terms of these results, we finally prove an exact tail asymptotic result for the stationary probabilities.