Maximum likelihood estimation on mismatch for stochastic nearly optimal control

Abstract

In a robot image processing system along with the visual feedback or in a TV telecommunication system along with signal processing, pattern recognition is most often confronted with a large state space representation requirement due to the complexity of unexpected shape, color and motion as well as environmental disturbance. It is inevitable that raw signals are affected by Gaussian noises. This problem in the presence of random noises can be modeled as a LQG model modulated by a finite state Markov chain. The optimal solution is achieved by dynamic programming and associated HJB equations. For large-scale systems, averaging approach is necessary to obtain consistent solutions to Riccati equations, which is the nearly optimal control scheme. The mismatch from time scale separation should be minimized. As a result, maximum likelihood estimation is proposed to optimize the total mismatch, which is a generally consistent and asymptotic Gaussian. In this article, the total mismatch and the convergence property within stochastic nearly optimal control problem are illustrated by a set of multi-dimensional numerical simulations and then maximum likelihood estimation scheme is derived and investigated on a basis of the multi-dimensional state space.