A dynamical systems framework for stochastic iterative optimization

Abstract

Stochastic optimization typically are solved via stochastic iterative algorithms. Examples are stochastic gradient descent, and other first-order methods for stochastic optimization. In such algorithms, exact computation of the gradient of the objective function is generally not possible. Hence, empirical estimates are used instead. We view such algorithms as a linear dynamical system but with noisy non-linear feedback. We give a general framework for probabilistic stability analysis of such dynamical systems. This is done via a novel stochastic dominance argument that was developed for convergence analysis of iterated random operators. We are also able to give a non-asymptotic rate of convergence within this framework.