Discrete-Time Finite Markov Chains

Abstract

In this chapter, we examine some of the properties of discrete-time finite Markov chains. These processes are discrete valued, and to keep the analysis as elementary as possible, we restrict our attention to homogeneous, i.e., time-invariant, and finite chains, i.e., chains where the process X(t) can take only a finite number of values. Under these simplifying assumptions, Markov chains can be analyzed by using tools of linear algebra. Yet, the class of finite homogeneous Markov chains covers a wide range of engineering systems, since it is in essence identical to the class of synchronous stochastic automata.