Explicit Causal Recursive Estimators for Continuous-Time Bivariate Markov Chains

Abstract

A bivariate Markov chain comprises a pair of random processes which are jointly Markov. In this paper, both processes are assumed to be continuous-time with finite state space. One of the two processes is observable, while the other is an underlying process which affects the statistical properties of the observable process. Neither the observable, nor the underlying process , is required to be a Markov chain. Examples of bivariate Markov chains include the Markov modulated Markov process (MMMP), the Markov modulated Poisson process (MMPP), and the batch Markovian arrival process (BMAP). We develop explicit causal recursions for estimating the number of jumps from one state to another, and the total sojourn time in each state, of a general bivariate Markov chain. Explicit causal recursions of these statistics were previously developed for the MMMP and the MMPP using the transformation of measure approach. We argue that this approach cannot be extended to a general bivariate Markov chain. Instead, we modify the approach developed by Ryden for noncausal estimation of the same statistics of an MMPP, and use the state augmentation approach of Zeitouni and Dembo and a matrix recursion from Stiller and Radons, to derive the causal recursions. The new recursions do not require any numerical integration or sampling scheme of the continuous-time bivariate Markov chain.