A Direction Set Based Algorithm for Adaptive Least Squares Problems in Signal Processing

Abstract

A fast algorithm, called the direction set based algorithm, has been developed recently for solving a class of adaptive least squares problems arising in signal processing. The algorithm is based on the direction set method developed by Powell and Zangwill for solving unconstrained minimization problems without using derivatives. It is designed so as to fully take advantage of the special structure of the adaptive least squares problems. It is a fast algorithm because it requires only O(N) multiplications for each system update where N is the number of parameters used in the system design. The algorithm has been implemented and applied to applications in signal processing. Computer simulation results have shown that the algorithm is stable and often converges fast with a rate which is comparable to that of the known Conjugate Gradient algorithm and Recursive Least Squares algorithm. The algorithm has been shown to converge linearly for adaptive least squares problems in general and its rate of convergence can be faster than linear for some applications. The algorithm can be simplified and its rate of convergence can be improved with the use of some direction sets. The direction set based algorithm has also been modified to solve constrained adaptive least squares problems arising in spectral estimation. Computer simulations illustrate that the algorithm is effective for spectral estimation.