Local minima escape transients by stochastic gradient descent algorithms in blind adaptive equalizers

Abstract

Many adaptive algorithms perform stochastic gradient descent on performance surfaces that are not guaranteed to be unimodal. In some examples, it is possible to show that not only is there more than one stationary point on this performance surface, but also that there is at least one incorrect local minimum. In the past, many authors have noted the existence of these incorrect stable equilibria, and noted that transitions between the regions of attraction of these local equilibria are possible. However, very little work has been done to determine the escape times, beyond observing that if the valleys surrounding these undesirable equilibria are very small and shallow, the escape time should not be too large. In this paper, we begin with a general discussion of the escape behaviour of adaptive algorithms, and follow this with an analysis, using diffusion approximations and large deviations theory, of the escape behaviour of the Godard class of blind equalizers. From this analysis, we obtain asymptotic estimates for the expected value of the escape time when leaving the region of attraction of local equilibria. Some observations are made also on the trajectories followed during such escapes. The basis for the computation of escape time estimates is the connection between large deviations and optimal control theory. For this interesting class of adaptive estimation problems, possessing multiple equilibria, the construction and solution of the optimal control problem is approximated, and shown to yield reasonable quantifications.