Two extensions of asymptotic methods in controlled Markov chains

Abstract

The application of the Skorokhod representation of martingales and of the local asymptotic normality to derive limit inequalities for the cost in controlled finite state Markov chains is reviewed. The inequalities are usable in self-optimizing control. The methods are taken from the references listed but, with the exception of proposition 4, the results are formulated for Markov chains for the first time.