Approximation of the exit probability of a stable Markov modulated constrained random walk

Abstract

Let X be the constrained random walk on \({\mathbb {Z}}_+^2\) having increments (1, 0), \((-\,1,1)\), \((0,-\,1)\) with jump probabilities \(\lambda (M_k)\), \(\mu _1(M_k)\), and \(\mu _2(M_k)\) where M is an irreducible aperiodic finite state Markov chain. The process X represents the lengths of two tandem queues with arrival rate \(\lambda (M_k)\), and service rates \(\mu _1(M_k)\), and \(\mu _2(M_k)\); the process M represents the random environment within which the system operates. We assume that the average arrival rate with respect to the stationary measure of M is less than the average service rates, i.e., X is assumed stable. Let \(\tau _n\) be the first time when the sum of the components of X equals n for the first time. Let Y be the random walk on \({{\mathbb {Z}}} \times {{\mathbb {Z}}}_+\) having increments \((-\,1,0)\), (1, 1), \((0,-\,1)\) with probabilities \(\lambda (M_k)\), \(\mu _1(M_k)\), and \(\mu _2(M_k)\). Supposing that the queues share a joint buffer of size n, \(p_n =P_{(x_n,m)}(\tau _n < \tau _0)\) is the probability that this buffer overflows during a busy cycle of the system. To the best of our knowledge, the only methods currently available for the approximation of \(p_n\) are classical large deviations analysis giving the exponential decay rate of \(p_n\) and rare event simulation. Let \(\tau \) be the first time the components of Y are equal. For \(x \in {{\mathbb {R}}}_+^2\), \(x(1) + x(2) < 1\), \(x(1) > 0\), and \(x_n = \lfloor nx \rfloor \), we show that \(P_{(n-x_n(1),x_n(2),m)}( \tau < \infty )\) approximates \(P_{(x_n,m)}(\tau _n < \tau _0)\) with exponentially vanishing relative error as \(n\rightarrow \infty \). For the analysis we define a characteristic matrix in terms of the jump probabilities of (X, M). The 0-level set of the characteristic polynomial of this matrix defines the characteristic surface; conjugate points on this surface and the associated eigenvectors of the characteristic matrix are used to define (sub/super) harmonic functions which play a fundamental role both in our analysis and the computation/approximation of \(P_{(y,m)}(\tau < \infty )\).