Normalized data nonlinearities for LMS adaptation

Abstract

Properly designed nonlinearly-modified LMS algorithms, in which various quantities in the stochastic gradient estimate are operated upon by memoryless nonlinearities, have been shown to perform better than the LMS algorithm in system identification-type problems. The authors investigate one such algorithm given by W/sub k+l/=W/sub k/+/spl mu/(d/sub k/-W/sub k//sup t/X/sub k/)X/sub k/f(X/sub k/) in which the function f(X/sub k/) is a scalar function of the sum of the squares of the N elements of the input data vector X/sub k/. This form of algorithm generalizes the so-called normalized LMS (NLMS) algorithm. They evaluate the expected behavior of this nonlinear algorithm for both independent input vectors and correlated Gaussian input vectors assuming the system identification model. By comparing the nonlinear algorithm's behavior with that of the LMS algorithm, they then provide a method of optimizing the form of the nonlinearity for the given input statistics. In the independent input case, they show that the optimum nonlinearity is a single-parameter version of the NLMS algorithm with an additional constant in the denominator and show that this algorithm achieves a lower excess mean-square error (MSE) than the LMS algorithm with an equivalent convergence rate. Additionally, they examine the optimum step size sequence for the optimum nonlinear algorithm and show that the resulting algorithm performs better and is less complex to implement than the optimum step size algorithm derived for another form of the NLMS algorithm. Simulations verify the theory and the predicted performance improvements of the optimum normalized data nonlinearity algorithm. >