Stochastic Adaptive Control

Abstract

The objective of this paper is to present some identification problems and adaptive control problems for continuous time linear and nonlinear stochastic systems that are completely or partially observed. For continuous time linear of stochastic systems the consistency of a family of least squares estimates of some unknown parameters is verified. The unknown parameters appear in the linear transformations of the state and the control. An approach to the verification of the consistency associates a family of control problems to the identification problem and the asymptotic behavior of the solutions of a family of algebraic Riccati equations from the control problems implies a persistent excitation property for the identification problem. The theorem of locally asymptotically normal experiment is used to test hypotheses about the parameters of a controlled linear stochastic system. The tests are formulated for both continuous and sampled observations of the input and the output.An adaptive control problem will be described and solved for continuous time linear stochastic systems using a diminishing excitation control to show a strong consistency of a family of least squares estimates and using switchings to show self-optimizing property.We shall investigate the ergodic control of a multidimensional diffusion process described by a nonlinear stochastic differential equation that has unknown parameters appearing in the drift. For ∈ > 0 it is required to find an adaptive control such that the ergodic cost for this control is within ∈ of the optimal ergodic likelihood estimation procedure that was used by Kumar and Becker. An adaptive control is constructed from a discretization of the range of this family of estimates using the certainty equivalence principle and this control is verified to be almost self-optimizing.We shall also consider adaptive control problem of a discrete time Markov process that is completely observed in a fixed recurrent domain and partially observed elsewhere. An almost self-optimal strategy is constructed for this problem. Finally, some numerical examples and simulation results will be presented.