Chapter 6 - Continuous Time Markov Chains

Abstract

This chapter presents several important examples of continuous time, discrete state Markov processes. Birth and death processes form a powerful tool available to the stochastic modeler. The richness of the birth and death parameters facilitates modeling a variety of phenomena. At the same time, standard methods of analysis are available for determining numerous important quantities such as stationary distributions and mean first passage times. The traditional procedure for constructing birth and death processes is to prescribe the birth and death parameters {λi, μi}i=∞ and build the path structure by utilizing the preceding description concerning the waiting times and the conditional transition probabilities of the various states. Birth and death processes in which λ0 = 0 arise frequently and are correspondingly important. For these processes, the zero state is an absorbing state. In physics, engineering, sociology, and biology, Markov processes arise whose values are subsets of a given finite set, while in sociology, the process may track the set of people-pairs having a specified relation. These processes are finite state Markov chains.