Abstract
This paper deals with the computation of invariant measures and stationary expectations for discrete-time Markov chains governed by a block-structured one-step transition probability matrix. The method generalizes in some respect Neuts’ matrix-geometric approach to vector-state Markov chains. The method reveals a strong relationship between Markov chains and matrix continued fractions which can provide valuable information for mastering the growing complexity of real-world applications of large-scale grid systems and multidimensional level-dependent Markov models. The results obtained are extended to continuous-time Markov chains.
Highlights
This paper is concerned with the computation of invariant measures of discrete-time Markov chains X = ðXkÞk≥0 having a block-partitioned one-step transition matrix P of the form P00 P01 0 ⋯ ⋯ ⋯⋯ P = BBBBBBBBBBB@ P10 ⋮ Pn0 0 P11 ⋮ Pn1 Pn+1,1 P12 ⋱ ⋯ ⋯ 0 ⋱ Pnn Pn+1,n
This paper deals with the computation of invariant measures and stationary expectations for discrete-time Markov chains governed by a block-structured one-step transition probability matrix
Irreducibility, and recurrence, the solution to yQ = 0 is unique up to constant multiples and strictly positive. These assumptions allow constructing the embedded jump chain which is a discretetime Markov chain X = ðXnÞ∞ n=0 with transition probability matrix with structure (1) and Pij = D−i 1Qij + δijI where Di is the diagonal matrix with entries −qði,jÞ,ði,jÞ at position ðj, jÞ
Summary
Since ðNÞP is not stochastic anymore, the last step of the above general method has to be interpreted appropriately: xðNÞ shall solve xðNÞ·ðNÞP = xðNÞ up to the first scalar equation This simplified matrix-analytic algorithm becomes even more powerful when only stationary expectations have to be computed since the memory-efficient variant suggested in [7] can be applied. Since Markov chains with general block structure are discussed, transition matrices of the form (1) or its continuous analogue ((20) below) can be interpreted as special cases. In some way, we combine the ideas of Neuts’ original matrix-geometric approach with considerations for level-dependent quasi-birth-death processes ( the resulting transition structures are not as general as in [8]), and with memory-efficient algorithms as developed in [7] for quasi-birth-death processes.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
