Discretized maximum likelihood estimates and almost self-optimizing controls for ergodic Markov models

Abstract

In this paper, three unknown ergodic Markov models are considered. The models are a discrete time Markov process with complete observations, a diffusion-process with complete observations and a discrete time Markov process with partial observations. The partial observations have the special form of complete observations in one subset and noisy observations in its complement. A finite discretization of the parameter set is used to construct the maximum likelihood estimates. Randomized certainty equivalence controls using these maximum likelihood estimates and finite families of almost optimal ergodic controls are shown to yield almost optimal adaptive controls. A continuity property of the information of the model for one parameter value with respect to another is used to establish this almost optimality property.