Ergodicity of Markov Processes via Nonstandard Analysis

Abstract

The Markov chain ergodic theorem is well-understood if either the time-line or the state space is discrete. However, there does not exist a very clear result for general state space continuous-time Markov processes. Using methods from mathematical logic and nonstandard analysis, we introduce a class of hyperfinite Markov processes-namely, general Markov processes which behave like finite state space discrete-time Markov processes. We show that, under moderate conditions, the transition probability of hyperfinite Markov processes align with the transition probability of standard Markov processes. The Markov chain ergodic theorem for hyperfinite Markov processes will then imply the Markov chain ergodic theorem for general state space continuous-time Markov processes.