Examples of Continuous-Time Markov Chains

Abstract

In this case we have a finite state space E which we can take to be \(\left\{ {1,2,3,...,n} \right\}\). Given any q-matrix Q, which need not be conservative, there is a unique Q-function, and we can solve either the backward or forward equations to find it. Let us consider, for example, the backward equations $$P'\left( t \right) = QP\left( t \right),P\left( 0 \right) = I.$$ (1.1) Suppose that Q is similar to the n x n matrix J; that is, that $$Q = MJ{M^{ - 1}}$$ (1.2) for some n x n invertible matrix M. Then \(T\left( t \right) = {M^{ - 1}}P\left( t \right)\) satisfies $$Y'\left( t \right) = JY\left( t \right),Y\left( 0 \right) = {M^{ - 1}}$$ (1.3) The point is J can in general be a simpler matrix than Q, with the result that the equation in (1.3) is straightforward to solve. The extreme case is when Q has n independent eigenvectors and thus J can be taken to be the diagonal matrix whose diagonal components are the eigenvalues of Q, and M is the matrix whose columns are the corresponding eigenvectors.