Tracking the output of a poorly known markov process

Abstract

Tracking the output of an unknown Markov process with unknown generator and unknown output function is considered. It is assumed the unknown quantities have a known prior probability distribution. It is shown that the optimal control is a linear feedback in the tracking error plus the conditional expectation of a quantity involving the unknown generator and output function of the Markov process. The results also have application to Bayesian identification of hidden Markov models