Adaptive algorithms with nonlinear data and error functions

Abstract

The tools of nonlinear system theory are used to examine several common nonlinear variants of the LMS algorithm and derive a persistence of excitation criterion for local exponential stability. The condition is tight when the inputs are periodic, and a generic counterexample is demonstrated which gives (local) instability for a large class of such nonlinear versions of LMS, specifically, those which utilize a nonlinear data function. The presence of a nonlinear error function is found to be relatively benign in that it does not affect the stability of the error system. Rather, it defines the cost function the algorithm tends to minimize. Specific examples include the dead zone modification, the cubed data nonlinearity, the cubed error nonlinearity, the signed regressor algorithm, and a single-layer version of the backpropagation algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>