Stochastic $\varepsilon$-Optimal Linear Quadratic Adaptation: An Alternating Controls Policy

Abstract

Listen

Translate article

This paper presents a continuous time stochastic linear quadratic (LQ) adaptive control algorithm for completely observed linear stochastic systems with unknown parameters. Based on a certainty equivalence approach, we propose to utilize an alternating controls policy, whereby the linear feedback matrix is switched between two $\sqrt{\varepsilon}$-apart distinct matrices $K_i$, $i=1,2$. The associated adaptive estimation algorithm is designed so that it drives the maximum likelihood based estimate into the sets ${\cal I}_i$, $i=1,2$, and consequently into ${\cal I}_1 \cap {\cal I}_2$, with ${\cal I}_i$ corresponding to the true closed loop dynamics under the $i$th control. A mild geometric assumption is shown to guarantee that ${\cal I}_1 \cap{\cal I}_2 =\theta^*$, the true parameter. This strongly consistent estimation, coupled with the alternating controls policy, then yields $\varepsilon$-optimal long-run LQ closed loop performance.