On sampled-data extremum seeking control via stochastic approximation methods

Abstract

This note establishes a link between stochastic approximation and extremum seeking of dynamical nonlinear systems. In particular, it is shown that by applying classes of stochastic approximation methods to dynamical systems via periodic sampled-data control, convergence analysis can be performed using standard tools in stochastic approximation. A tuning parameter within this framework is the period of the synchronised sampler and hold device, which is also the waiting time during which the system dynamics settle to within a controllable neighbourhood of the steady-state input-output behaviour. Semiglobal convergence with probability one is demonstrated for three basic classes of stochastic approximation methods: finite-difference, random directions, and simultaneous perturbation. The tradeoff between the speed of convergence and accuracy is also discussed within the context of asymptotic normality of the outputs of these optimisation algorithms.