A computational approach to extreme values and related hitting probabilities in level-dependent quasi-birth–death processes

Abstract

This paper analyzes the dynamics of a level-dependent quasi-birth–death process X={(I(t),J(t)):t≥0}, i.e., a bi-variate Markov chain defined on the countable state space ∪i=0∞l(i) with l(i)={(i,j):j∈{0,…,Mi}}, for integers Mi∈N0 and i∈N0, which has the special property that its q-matrix has a block-tridiagonal form. Under the assumption that the first passage to the subset l(0) occurs in a finite time with certainty, we characterize the probability law of (τmax,Imax,J(τmax)), where Imax is the running maximum level attained by process X before its first visit to states in l(0), τmax is the first time that the level process {I(t):t≥0} reaches the running maximum Imax, and J(τmax) is the phase at time τmax. Our methods rely on the use of restricted Laplace–Stieltjes transforms of τmax on the set of sample paths {Imax=i,J(τmax)=j}, and related processes under taboo of certain subsets of states. The utility of the resulting computational algorithms is demonstrated in two epidemic models: the SIS model for horizontally and vertically transmitted diseases; and the SIR model with constant population size.