A Convexity Property in the Theory of Random Variables Defined on a Finite Markov Chain

Abstract

Let $P = (p_{jk})$ be the transition matrix of an ergodic, finite Markov chain with no cyclically moving sub-classes. For each possible transition $(j, k)$, let $H_{jk}(x)$ be a distribution function admitting a moment generating function $f_{jk}(t)$ in an interval surrounding $t = 0$. The matrix $P(t) = \{p_{jk}f_{jk}(t)\}$ is of interest in the study of the random variable $S_n = X_1 + \cdots + X_n$, where $X_m$ has the distribution $H_{jk}(x)$ if the $m$th transition takes the chain from state $j$ to state $k$. The matrix $P(t)$ is non-negative and therefore possesses a maximal positive eigenvalue $\alpha_1(t)$, which is shown to be a convex function of $t$. As an application of the convexity property, we obtain an asymptotic expression for the probability of tail values of the sum $S_n$, in the case where the $X_m$ are integral random variables. The results are related to those of Blackwell and Hodges [1], whose methods are followed closely in Section 5, and Volkov [4], [5], who treats in detail the case of integer-valued functions of the state of the chain, i.e., the case $f_{jk}(t) = \exp(\beta_kt) (\beta_k$ integral).