Sampling and filtering with Markov chains

Abstract

A new continuous-time Markov chain rate change formula is proven. This theorem is used to derive existence and uniqueness of novel filtering equations akin to the Duncan–Mortensen–Zakai equation and the Fujisaki–Kallianpur–Kunita equation but for Markov signals with general continuous-time Markov chain observations. The equations in this second theorem have the unique feature of being driven by both the observations and the process counting the observation transitions. A direct method of solving these filtering equations is also derived. Most results apply as special cases to the continuous-time Hidden Markov Models (CTHMM), which have become important in applications like disease progression tracking, The corresponding CTHMM results are stated as corollaries. Finally, application of our general theorems to Markov chain importance sampling, rejection sampling and branching particle filtering algorithms is also explained and these are illustrated by way of disease tracking simulations.