A Simple Numerical Approach for Infinite-State Markov Chains

Abstract

This paper presents a simple and practical approach to solving the equilibrium equations for a class of Markov chains with an infinite number of states. Markov chains arising in queueing and inventory applications often have the property that the state probabilities exhibit a geometric tail behavior. The basic idea of the approach is to reduce the infinite system of linear equations to a finite system using the geometric tail behavior of the equilibrium probabilities. The reduction typically leads to a remarkably small system of linear equations that can be routinely solved by a Gaussian elimination method. An application is given to the single-server queue with scheduled arrivals.