Abstract
We study the asymptotic behaviour of Markov chains (Xn,ηn) on Z+×S, where Z+ is the non-negative integers and S is a finite set. Neither coordinate is assumed to be Markov. We assume a moments bound on the jumps of Xn, and that, roughly speaking, ηn is close to being Markov when Xn is large. This departure from much of the literature, which assumes that ηn is itself a Markov chain, enables us to probe precisely the recurrence phase transitions by assuming asymptotically zero drift for Xn given ηn. We give a recurrence classification in terms of increment moment parameters for Xn and the stationary distribution for the large- X limit of ηn. In the null case we also provide a weak convergence result, which demonstrates a form of asymptotic independence between Xn (rescaled) and ηn. Our results can be seen as generalizations of Lamperti’s results for non-homogeneous random walks on Z+ (the case where S is a singleton). Motivation arises from modulated queues or processes with hidden variables where ηn tracks an internal state of the system.
Highlights
There are many applications that naturally give rise to Markov processes on a product statespace X × S where S describes some operating regime or internal state of the system, which influences the motion of the process in the primary space X
The queueing models become Markov-modulated [26,10], while other examples fit into the class of Markov random walks [14]
This case includes processes that can be represented as additive functionals of Markov chains [27]
Summary
There are many applications that naturally give rise to Markov processes on a product statespace X × S where S describes some operating regime or internal state of the system, which influences the motion of the process in the primary space X. In the most classical setting, the projection of the process onto S is itself Markovian In this case, the queueing models become Markov-modulated [26,10], while other examples fit into the class of Markov random walks [14]. Much less is known when the process projected onto S is not Markovian: the main focus of the present work is to replace the Markovian assumption by a weaker (asymptotic) condition that provides sufficient structure This relaxation is necessary to probe more intimately the recurrence–transience phase transition for these models, since the natural setting (parallelling the classical work of Lamperti) is to suppose that the law of the process is non-homogeneous in X, in particular, the mean drift of the X-component of the process will be asymptotically zero.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
