Countable-state-space Markov chains with two time scales and applications to queueing systems

Abstract

Motivated by various applications in queueing systems, this work is devoted to continuous-time Markov chains with countable state spaces that involve both fast-time scale and slow-time scale with the aim of approximating the time-varying queueing systems by their quasistationary counterparts. Under smoothness conditions on the generators, asymptotic expansions of probability vectors and transition probability matrices are constructed. Uniform error bounds are obtained, and then sequences of occupation measures and their functionals are examined. Mean square error estimates of a sequence of occupation measures are obtained; a scaled sequence of functionals of occupation measures is shown to converge to a Gaussian process with zero mean. The representation of the variance of the limit process is also explicitly given. The results obtained are then applied to treat Mt/Mt/1 queues and Markov-modulated fluid buffer models.