Random Markov processes for countable and uncountable alphabets

Abstract

In this paper, we explore two notions of stationary processes — uniform martingales and random Markov processes. We show that every stationary process on a countable alphabet that is a uniform martingale and is dominated by a finite measure on that alphabet is also a random Markov process, and that the random look-back times and associated coupling can be chosen so that the distribution on the present given both the n-past and a look-back time of n is ‘deterministic’: all probabilities are in {0, 1}. In the case of finite alphabets, those random Markov processes for which the look-back time can be chosen with finite expected value are characterized. For stationary processes on an uncountable alphabet, a stronger condition is also considered which is sufficient to imply that a process is a random Markov processes. In addition, a number of examples are given throughout to show the sharpness of the results.