A fast method for regularized adaptive filtering

Abstract

Regression models are used in many areas of signal processing, e.g., spectral analysis and speech LPC, where block processing methods have typically been used to estimate the unknown coefficients. Iterative methods for adaptive estimation fall into two categories: the least-mean-square (LMS) algorithm and the recursive-least-squares (RLS) algorithm. The LMS algorithm offers low complexity and stable operation at the expense of convergence speed. The RLS algorithm offers improved convergence performance at the expense of possible stability problems. Note that low complexity is used here to denote that the computational burden is proportional to the number of adaptive coefficients. The LMS algorithm is the standard algorithm for applications in which low complexity is tantamount. The LMS algorithm produces an approximation to the minimum mean square error estimate; that is, the expected value of the output error approaches zero. The major drawback of the LMS algorithm is that its rate of convergence is dependent upon the eigenvalue spread of the input data correlation matrix, usually excluding its use in high-speed, real-time signal processing applications [ 11. However, it is widely used where the convergence speed is not a problem [ 2,3]. The convergence rate of the LMS algorithm can be increased by introducing methods that attempt to orthogonalize the input data and reduce the eigenvalue spread of the correlation matrix [ 4,5]. The convergence rate of any adaptive algorithm can be specified by either the coefficient error or the estimation error, or both. It is well known that, for the LMS algorithm, the mean square error in the output converges faster than the mean square error in the coefficients [ 6,7]. Thus, the LMS algorithm is useful for situations in which the primary function of the adaptive system is