CHAPTER 8 - FINITE MARKOV CHAINS

Abstract

This chapter discusses the n-state homogeneous Markov chains. The study of such chains provides the applications of the theory of nonnegative matrices. A Markov chain is a special type of stochastic process, and a stochastic process is concerned with events that change in a random way with time. A typical stochastic process might predict the motion of an object that is constrained to be at any one time in exactly one of a number of possible states. It would then be a scheme for determining the probability of the object's being in a specific state at a specific time. The probability would generally depend upon a large number of factor, such as the state, the time, some or all of the previous states the object has been in, and the states other objects are in or have been in. If the probability that an object will move to another state depends only on the two states involved and not on earlier states, time, or other factors, then the stochastic process is called a homogeneous Markov process. The theory of Markov processes comprises the largest and the most important part of the theory of stochastic processes. The theory of Markov processes has found many applications in the physical, biological, and social sciences and in engineering and commerce.