Family Of Adaptive Filtering Algorithms Research Articles

A family of adaptive filtering algorithms based on the variable step size

In this paper, we present the most used adaptive filtering algorithms such as Least Mean Square (LMS) and its normalized version NLMS with their advantages and drawbacks, and then show how the Variable Step Size (VSS) algorithms have been proposed to slove problems coming from the fixed step size. Series of simulations have been carried out under different effects such as: the size of the adaptive filter and different values of step size to validate the good behaviour of the four presented VSS based algorithms over the classical adaptive filtering algorithms with fixed step size. Also, results have confirmed the superiority of VSS based algorithms in terms of convergence speed with almost identical computational complexity.

Robust adaptive filtering algorithms based on (inverse)hyperbolic sine function.

Recently, adaptive filtering algorithms were designed using hyperbolic functions, such as hyperbolic cosine and tangent function. However, most of those algorithms have few parameters that need to be set, and the adaptive estimation accuracy and convergence performance can be improved further. More importantly, the hyperbolic sine function has not been discussed. In this paper, a family of adaptive filtering algorithms is proposed using hyperbolic sine function (HSF) and inverse hyperbolic sine function (IHSF) function. Specifically, development of a robust adaptive filtering algorithm based on HSF, and extend the HSF algorithm to another novel adaptive filtering algorithm based on IHSF; then continue to analyze the computational complexity for HSF and IHSF; finally, validation of the analyses and superiority of the proposed algorithm via simulations. The HSF and IHSF algorithms can attain superior steady-state performance and stronger robustness in impulsive interference than several existing algorithms for different system identification scenarios, under Gaussian noise and impulsive interference, demonstrate the superior performance achieved by HSF and IHSF over existing adaptive filtering algorithms with different hyperbolic functions.

Feature Adaptive Filtering: Exploiting Hidden Sparsity

We have been witnessed a growing research activity to advance new strategies to detect and exploit underlying sparsity in the parameters of physical models. In many cases, the sparsity is not explicit in the relations among the parameter coefficients requiring some suitable tools to reveal the potential sparsity. This work proposes a family of adaptive filtering algorithms, aimed at exposing some hidden features of the unknown parameters. Although the basic idea applies to any algorithm, we will concentrate the work in the LMS-type algorithms, giving rise to a family collectively named as Feature LMS (F-LMS) algorithms. These algorithms increase the convergence speed and reduce the steady-state mean-squared error, in comparison with the classical LMS solution. The main idea is to apply linear transformations, through the so-called feature matrices, to reveal the sparsity hidden in the coefficient vector, followed by a sparsity-promoting penalty function to exploit the exposed sparsity. For illustration, a few F-LMS algorithms for lowpass, bandpass, and highpass systems are introduced by using simple feature matrices that require either only simple operations or can learn the features. Simulations and real-life experiments demonstrate that the F-LMS algorithms bring about several performance improvements whenever the unknown sparsity of parameters is exposed.

Nonlinear acoustic echo cancellation with kernelized adaptive filters

A well-known problem in speech communication is the occurrence of acoustic echo in hands-free telephony. There are several classical adaptive filters that have been used as a stand alone approach for linear acoustic echo cancellation(AEC). A common assumption in most AEC techniques is that the echo path is linear. However, in real scenarios, owing to distortions introduced by other acoustic artefacts in handheld devices, the echo path is nonlinear. Therefore, there is a need to employ a nonlinear echo canceller. In this paper, a kernel expansion on a family of adaptive filtering algorithms based on least mean square filter is proposed. Kernel methods help model the nonlinear echo path and attain global minimum easily by finding the optimal set of filter weights. The simulations are carried out for speech signal with and without noise under different signal-to-noise ratio values. The results show that the proposed method achieves a significant improvement in a nonlinear acoustic echo cancellation environment in terms of echo return loss enhancement.

Novel Proportionate Adaptive Filters with Coefficient Vector Reusing

In this paper, a new family of adaptive filtering algorithms is presented, which aims to combine the small misalignment resulting from the reuse of past weight vectors with the fast convergence arising from the proportionate adaptation and logarithmic cost functions. This family of algorithms is obtained as a solution to a deterministic constrained optimization problem, by using the Lagrange multipliers technique, which differs from the traditionally employed stochastic gradient technique. Two special cases are proposed, namely the improved mu-law proportionate least mean logarithmic square with reuse of coefficients (IMPLMLS-RC) algorithm and the improved mu-law proportionate least logarithmic absolute difference with reuse of coefficients (IMPLLAD-RC) algorithm. An energy conservation relationship is established, which can be employed to perform stochastic transient analyses of the proposed algorithms. Simulations in system identification and active noise control applications show the advantages of the IMPLMLS-RC and IMPLLAD-RC algorithms over the traditional LMS and LAD, and the recently proposed LMLS and LLAD, with respect to both steady-state performance and robustness against impulsive noise.

Tracking Analysis of High Order Stochastic Gradient Adaptive Filtering Algorithms

This paper analyzes the tracking performance of the family of adaptive filtering algorithms based on minimizing the ${{2}}L$ -th moment of the estimation error, with $L$ being an integer greater than 1. The analysis is done for a Markov plant without the usually used assumption of small degree of non-stationarity (DNS). General $L$ and DNS are considered in the analysis. Dependence of the minimum steady-state mean square weight deviation on the values of $L$ , DNS and type of the noise distribution is studied. A measure for evaluating the steady-state stability of the algorithm is introduced. Dependence of the steady-state stability on the values of $L$ , DNS, and type of the noise distribution is studied. Some results are surprising. Theoretical results are supported by simulations.

Algorithm and VLSI Architecture Design of Proportionate-Type LMS Adaptive Filters for Sparse System Identification

Proportionate-type normalized LMS (Pt-NLMS) family of adaptive filtering algorithms for sparse system identification pose significant implementation challenges due to their high computational complexity especially for real-time applications like network echo cancelation. In this paper, we make the first attempt to implement Pt-NLMS algorithms in hardware. Several reformulations are proposed to simplify the original Pt-NLMS algorithms, thereby making them amenable to real-time VLSI implementations and the reformulated algorithms referred as delayed $\mu $ -law proportionate LMS (DMPLMS) algorithm for white input and delayed wavelet MPLMS (DWMPLMS) for colored input are then implemented in hardware. Simulation studies demonstrate that the performance loss is very small for the proposed reformulations. We implemented the proposed designs considering 16-bit fixed point representation in hardware, and synthesis results show that the DMPLMS architecture with ${\approx } 30\%$ increase in hardware over the state-of-the-art conventional delayed LMS architecture achieves $3\times $ improvement in convergence rate for white input and the DWMPLMS architecture with ${\approx } 70\%$ increase in hardware achieves $10\times $ improvement in convergence rate for correlated input conditions.

Stabilization of High-Order Stochastic Gradient Adaptive Filtering Algorithms

The paper is concerned with stabilizing the family of adaptive filtering algorithms based on minimizing the 2Lth moment of the estimation error, with L being an integer greater than 1. Stabilization is attained via a proposed normalization of the algorithm. Mean square stability of the normalized algorithm is proved for a Markov plant for all L > 1. Transient and steady-state performances of the algorithm are analyzed for a time-invariant plant. Expressions are derived for the steady-state misadjustment and convergence time. Tradeoff between the transient and steady-state performances is evaluated. Dependence of this tradeoffon the value of L is studied. This tradeoff is compared with the tradeoff of the NLMS algorithm. The proposed algorithm is dramatically superior to the NLMS algorithm for sub-Gaussian noise. The superiority increases with L, initial mean-square deviation and signal-to-noise ratio. Analytical results are supported by simulations.

A Novel Family of Adaptive Filtering Algorithms Based on the Logarithmic Cost

We introduce a novel family of adaptive filtering algorithms based on a relative logarithmic cost inspired by the “competitive methods” from the online learning literature. The competitive or regret based approaches stabilize or improve the convergence performance of adaptive algorithms through relative cost functions. The new family elegantly and gradually adjusts the conventional cost functions in its optimization based on the error amount. We introduce important members of this family of algorithms such as the least mean logarithmic square (LMLS) and least logarithmic absolute difference (LLAD) algorithms. However, our approach and analysis are generic such that they cover other well-known cost functions as described in the paper. The LMLS algorithm achieves comparable convergence performance with the least mean fourth (LMF) algorithm and enhances the stability performance significantly. The LLAD and least mean square (LMS) algorithms demonstrate similar convergence performance in impulse-free noise environments while the LLAD algorithm is robust against impulsive interferences and outperforms the sign algorithm (SA). We analyze the transient, steady-state and tracking performance of the introduced algorithms and demonstrate the match of the theoretical analyses and simulation results. We show the enhanced stability performance of the LMLS algorithm and analyze the robustness of the LLAD algorithm against impulsive interferences. Finally, we demonstrate the performance of our algorithms in different scenarios through numerical examples.

A Family of Shrinkage Adaptive-Filtering Algorithms

A family of adaptive-filtering algorithms that uses a variable step size is proposed. A variable step size is obtained by minimizing the energy of the noise-free a posteriori error signal which is obtained by using a known $L_1{\hbox{-}}L_2$ minimization formulation. Based on this methodology, a shrinkage affine projection (SHAP) algorithm, a shrinkage least-mean-squares (SHLMS) algorithm, and a shrinkage normalized least-mean-squares (SHNLMS) algorithm are proposed. The SHAP algorithm yields a significantly reduced steady-state misalignment as compared to the conventional affine projection (AP), variable-step-size AP, and set-membership AP algorithms for the same convergence speed although the improvement is achieved at the cost of an increase in the average computational effort per iteration in the amount of 11% to 14%. The SHLMS algorithm yields a significantly reduced steady-state misalignment and faster convergence as compared to the conventional LMS and variable-step-size LMS algorithms. Similarly, the SHNLMS algorithm yields a significantly reduced steady-state misalignment and faster convergence as compared to the conventional normalized least-mean-squares (NLMS) and set-membership NLMS algorithms.

A family of variable step-size affine projection adaptive filter algorithms using statistics of channel impulse response

Abstract This paper extends the recently introduced variable step-size (VSS) approach to the family of adaptive filter algorithms. This method uses prior knowledge of the channel impulse response statistic. Accordingly, optimal step-size vector is obtained by minimizing the mean-square deviation (MSD). The presented algorithms are the VSS affine projection algorithm (VSS-APA), the VSS selective partial update NLMS (VSS-SPU-NLMS), the VSS-SPU-APA, and the VSS selective regressor APA (VSS-SR-APA). In VSS-SPU adaptive algorithms the filter coefficients are partially updated which reduce the computational complexity. In VSS-SR-APA, the optimal selection of input regressors is performed during the adaptation. The presented algorithms have good convergence speed, low steady state mean square error (MSE), and low computational complexity features. We demonstrate the good performance of the proposed algorithms through several simulations in system identification scenario.

A Family of Adaptive Filter Algorithms in Noise Cancellation for Speech Enhancement

In many application of noise cancellation, the changes in signal characteristics could be quite fast. This requires the utilization of adaptive algorithms, which converge rapidly. Least Mean Squares (LMS) and Normalized Least Mean Squares (NLMS) adaptive filters have been used in a wide range of signal processing application because of its simplicity in computation and implementation. The Recursive Least Squares (RLS) algorithm has established itself as the ultimate adaptive filtering algorithm in the sense that it is the adaptive filter exhibiting the best convergence behavior. Unfortunately, practical implementations of the algorithm are often associated with high computational complexity and/or poor numerical properties. Recently adaptive filtering was presented, have a nice tradeoff between complexity and the convergence speed. This paper describes a new approach for noise cancellation in speech enhancement using the two new adaptive filtering algorithms named fast affine projection algorithm and fast Euclidean direction search algorithms for attenuating noise in speech signals. The simulation results demonstrate the good performance of the two new algorithms in attenuating the noise.

H∞ bounds for quasi-Newton adaptive algorithm

The upper and lower bounds for the H ∞ norm of the quasi-Newton (QN) family of adaptive filtering algorithms are obtained. A simulation study of the behaviour of the bounds with respect to the iteration order and the weight-vector dimension for a practical input data model reveals that the QN family is quite robust against modelling errors and errors due to the uncertainty in the initial weight-vector assumption.

Mean-square performance of the family of adaptive filters with selective partial updates

In this paper we present a general formalism for the family of adaptive filter algorithms with selective partial updates. Based on this, the mean-square performance analysis of this family of adaptive filters is presented in a unified way. This analysis is based on energy conservation arguments and does not need to assume a Gaussian or white distribution for the regressors. We demonstrate through simulations that the results are useful in predicting the performance of this family of adaptive filters.

A family of adaptive filter algorithms with decorrelating properties

Although the normalized least mean square (NLMS) algorithm is robust, it suffers from low convergence speed if driven by highly correlated input signals. One method presented to overcome this problem is the Ozeki/Umeda (1984) affine projection (AP) algorithm. The algorithm applies update directions that are orthogonal to the last P input vectors and thus allows decorrelation of an AR(P) input process, speeding up the convergence. This article presents a simple approach to show this property, which furthermore leads to the construction of new algorithms that can handle other kinds of correlations such as MA and ARMA processes. A statistical analysis is presented for this family of algorithms. Similar to the AP algorithm, these algorithms also suffer a possible increase in the noise energy caused by their pre-whitening filters.

Stochastic gradient adaptation under general error criteria

Examines a family of adaptive filter algorithms of the form W/sub k+1/=W/sub k/+/spl mu/f(d/sub k/-W/sub k//sup t/X/sub k/)X/sub k/ in which f(/spl middot/) is a memoryless odd-symmetric nonlinearity acting upon the error. Such algorithms are a generalization of the least-mean-square (LMS) adaptive filtering algorithm for even-symmetric error criteria. For this algorithm family, the authors derive general expressions for the mean and mean-square convergence of the filter coefficients For both arbitrary stochastic input data and Gaussian input data. They then provide methods for optimizing the nonlinearity to minimize the algorithm misadjustment for a given convergence rate. Using the calculus of variations, it is shown that the optimum nonlinearity to minimize misadjustment near convergence under slow adaptation conditions is independent of the statistics of the input data and can be expressed as -p'(x)/p(x), where p(x) is the probability density function of the uncorrelated plant noise. For faster adaptation under the white Gaussian input and noise assumptions, the nonlinearity is shown to be x/{1+/spl mu//spl lambda/x/sup 2///spl sigma//sub k//sup 2/}, where /spl lambda/ is the input signal power and /spl sigma//sub k//sup 2/ is the conditional error power. Thus, the optimum stochastic gradient error criterion for Gaussian noise is not mean-square. It is shown that the equations governing the convergence of the nonlinear algorithm are exactly those which describe the behavior of the optimum scalar data nonlinear adaptive algorithm for white Gaussian input. Simulations verify the results for a host of noise interferences and indicate the improvement using non-mean-square error criteria. >