Certainty equivalence control with forcing: revisited

Abstract

Certainty equivalence control with forcing has been shown to be optimal for several stochastic adaptive control problems with the average cost per unit time criterion. Recently researchers have started looking at stochastic adaptive control problems with a view to minimizing the rate of increase of the learning loss. This criterion is stronger than the average cost per unit time criterion. Certainty equivalence control with forcing does not usually suffice for the learning loss criterion and one has to develop fairly complicated schemes in order to achieve optimality. The objective of this paper is to see how well one might be able to do with a certainty-equivalence-control-with-forcing type of scheme. In particular we construct a class of such schemes whose learning loss is O(( log n) 1+δ) for δ > 0, whereas optimal schemes typically have a O( log n) learning loss.