Computing the stationary distribution for infinite Markov chains

Abstract

In a situation where the unique stationary distribution vector of an infinite irreducible positive-recurrent stochastic matrix P is not analytically determinable, numerical approximations are needed. This paper partially synthesizes and extends work on finite-vector approximative solutions obtained from n x n northwest corner truncations ( n ) P of P , from the standpoints of (pointwise convergence) algorithms as n →∞, and the manner of their computer implementation with a view to numerical stability and conditioning. The problem for finite n is connected with that of finding the unique stationary distribution of the finite stochastic matrix ( n ) P̃ obtained from ( n ) P by augmenting a column.