Optimal adaptive control of linear systems

Abstract

For the quadratic cost, nonlinear, adaptive stochastic control problem with linear discrete plant and measurement models excited by white gaussian noise, and unknown time-invariant model parameters, the optimal stochastic control is obtained and shown to separate ("Nonlinear Separation Theorem") into a bank of model-conditional deterministic control gains and a corresponding bank of known nonlinear functionals of the model-conditional, causal, mean-square state-vector estimates. This separation may also be viewed as a decomposition of the optimal, nonlinear adaptive control into a bank of model-conditional optimal, non-adaptive linear controls, one for each admissible value of the unknown parameter θ and a nonlinear part, namely, the bank of a-posteriori model probabilities, which incorporate the adaptive nature, of the optimal adaptive control. Results are given for several special cases of the above problem that exhibit drastically reduced computational requirements. These are the cases of (a) uncertainty in the measurement matrix only; and (b) the case of completely known models, but with nongaussian initial state-vector. In both special cases, we have explicit separation between control and estimation. Moreover, in both cases only one deterministic controller is required to be used with the nonlinear, adaptive mean-square state-vector estimate. Several illustrative examples are included to demonstrate the adaptive control algorithm developed in this paper.