Explicit Forward Recursive Estimators for Markov Modulated Markov Processes

Abstract

A Markov modulated Markov process is a doubly stochastic finite-alphabet continuous-time random process with an underlying hidden Markov chain and an observable conditionally Markov chain. Explicit forward recursions are developed for the conditional mean estimators of the state, number of jumps, and total sojourn time of the underlying chain, and for the conditional number of jumps and total sojourn time of the observable process given any state of the underlying chain. The recursions are derived using the transformation of measure approach, and their explicit forms follow from vectorization and Van Loan's formula for integration of matrix exponentials. The recursions are applicable to Markov modulated Poisson processes. Numerical results from applications of these recursions are also provided.