Steady-state Excess Mean Square Error Research Articles

Steady-State Performance of Non-Negative Least-Mean-Square Algorithm and Its Variants

Non-negative least-mean-square (NNLMS) algorithm and its variants have been proposed for online estimation under non-negativity constraints. The transient behavior of the NNLMS, Normalized NNLMS, Exponential NNLMS and Sign-Sign NNLMS algorithms have been studied in our previous work. In this technical report, we derive closed-form expressions for the steady-state excess mean-square error (EMSE) for the four algorithms. Simulations results illustrate the accuracy of the theoretical results. This is a complementary material to our previous work.

A variable step-size selective partial update LMS algorithm

Selective partial update of the adaptive filter coefficients has been a popular method for reducing the computational complexity of least mean-square (LMS)-type adaptive algorithms. These algorithms use a fixed step-size that forces a performance compromise between fast convergence speed and small steady state misadjustment. This paper proposes a variable step-size (VSS) selective partial update LMS algorithm, where the VSS is an approximation of an optimal derived one. The VSS equations are controlled by only one parameter, and do not require any a priori information about the statistics of the system environment. Mean-square performance analysis will be provided for independent and identically distributed (i.i.d.) input signals, and an expression for the algorithm steady state excess mean-square error (MSE) will be presented. Simulation experiments are conducted to compare the proposed algorithm with existing full-update VSS LMS algorithms, which indicate that the proposed algorithm performs as well as these algorithms while requiring less computational complexity.

A regional multimodulus algorithm for blind equalization of QAM signals: Introduction and steady-state analysis

It is well known that constant-modulus-based algorithms present a large mean-square error for high-order quadrature amplitude modulation (QAM) signals, which may damage the switching to decision-directed-based algorithms. In this paper, we introduce a regional multimodulus algorithm for blind equalization of QAM signals that performs similar to the supervised normalized least-mean-squares (NLMS) algorithm, independently of the QAM order. We find a theoretical relation between the coefficient vector of the proposed algorithm and the Wiener solution and also provide theoretical models for the steady-state excess mean-square error in a nonstationary environment. The proposed algorithm in conjunction with strategies to speed up its convergence and to avoid divergence can bypass the switching mechanism between the blind mode and the decision-directed mode.

A Novel GSC Beamformer Using a Combination of Two Adaptive Filters for Smart Antenna Array

A novel affine combination of two adaptive filters is proposed to enhance the performance of a generalized sidelobe canceller (GSC) in smart antenna array processing. The proposed method exploits a mixing vector, rather than a scalar as considered in current studies, to weight the coefficients of each component filter in such a way that the overall performances, such as convergence speed and the steady-state performance, of conventional GSC are improved. Attributing to more degrees of freedom (DOFs) offered by the mixing vector, the combined GSC presents significant performance improvement over current approaches. The optimal design as well as adaptive implementation of the overall combined GSC filter is developed with detailed theoretical analysis on the convergence rate and the steady-state excess mean-square error. The effectiveness of the proposed method is demonstrated by simulation studies.

On Error-Saturation Nonlinearities in NLMS Adaptation

The effect of a saturation-type error nonlinearity in the weight update equation in normalized least mean-square (NLMS) adaptation is investigated for system identification for a white Gaussian data model. Nonlinear recursions are derived for the weight mean error and mean-square deviation (MSD) that include the effect of an error function (erf) saturation-type nonlinearity on the error sequence driving the algorithm. The nonlinear recursion for the MSD is solved numerically and shown in excellent agreement with Monte Carlo simulations, supporting the theoretical model assumptions. The theory is extended to tracking a Markov channel and accurately predicts the tracking behavior as well. The saturation behavior of the algorithm is easily studied by varying a single parameter in the error function, varying from a linear device to a hard limiter. For the white data case, the excess mean square-error (EMSE) is simply related to the MSD. The tradeoff between the extent of error saturation, steady-state EMSE, and algorithm convergence rate is studied using these results.

Convergence and tracking analysis of a variable normalised LMF (XE-NLMF) algorithm

The least-mean-fourth (LMF) algorithm is known for its fast convergence and lower steady state error, especially in sub-Gaussian noise environments. Recent work on normalised versions of the LMF algorithm has further enhanced its stability and performance in both Gaussian and sub-Gaussian noise environments. For example, the recently developed normalised LMF (XE-NLMF) algorithm is normalised by the mixed signal and error powers, and weighted by a fixed mixed-power parameter. Unfortunately, this algorithm depends on the selection of this mixing parameter. In this work, a time-varying mixed-power parameter technique is introduced to overcome this dependency. A convergence analysis, transient analysis, and steady-state behaviour of the proposed algorithm are derived and verified through simulations. An enhancement in performance is obtained through the use of this technique in two different scenarios. Moreover, the tracking analysis of the proposed algorithm is carried out in the presence of two sources of nonstationarities: (1) carrier frequency offset between transmitter and receiver and (2) random variations in the environment. Close agreement between analysis and simulation results is obtained. The results show that, unlike in the stationary case, the steady-state excess mean-square error is not a monotonically increasing function of the step size.

Efficient design of block adaptive equalization and diversity combining for space-time block-coded single-carrier systems

In this paper, a weighted space-time block-coding (STBC)-like matrix-wise block adaptive frequency-domain equalization (WSB-FDE), which has a flexibility over the STBC-like matrix-wise block length as compared to a normalized least-mean-square (NLMS)-FDE and a recursive-least-square (RLS)-FDE, is proposed for the cyclic-prefixed STBC single-carrier systems equipped with a diversity combining. In the WSB-FDE, the STBC-like matrix-wise block is first formulated from received blocks. The correction terms of the equalizer coefficients in each block are then updated so as to minimize a weighted squared-norm of the a posteriori error vector for the formulated block. Moreover, a square-root adaptive algorithm based on an unitary Givens rotation is proposed to achieve the numerical stability and computational efficiency in update procedure of the WSB-FDE. The theoretic convergence and the steady-state excess mean-square error (EMSE) of the WSB-FDE are analyzed, where it is observed that the EMSE decreases as the STBC-like matrix-wise block length and the number of antenna increase. Finally, the performance evaluation shows in a typical urban (TU) channel that the bit-error-rate (BER) performance of the WSB-FDE is superior to that of the RLS-FDE as the STBC-like matrix-wise block length increases.

Improving the Tracking Capability of Adaptive Filters via Convex Combination

As is well known, Hessian-based adaptive filters (such as the recursive-least squares algorithm (RLS) for supervised adaptive filtering, or the Shalvi-Weinstein algorithm (SWA) for blind equalization) converge much faster than gradient-based algorithms [such as the least-mean-squares algorithm (LMS) or the constant-modulus algorithm (CMA)]. However, when the problem is tracking a time-variant filter, the issue is not so clear-cut: there are environments for which each family presents better performance. Given this, we propose the use of a convex combination of algorithms of different families to obtain an algorithm with superior tracking capability. We show the potential of this combination and provide a unified theoretical model for the steady-state excess mean-square error for convex combinations of gradient- and Hessian-based algorithms, assuming a random-walk model for the parameter variations. The proposed model is valid for algorithms of the same or different families, and for supervised (LMS and RLS) or blind (CMA and SWA) algorithms.

The Excess Mean-Square Error Analyses for Bussgang Algorithm

Without appealing to the circularity assumption of the a priori estimation error in , two closed-form analytical expressions for the steady-state excess mean-square error (EMSE) in a noise-free environment are derived again based on Taylor series expansion for real and complex Bussgang algorithms, respectively; and the restrictive conditions of these two expressions for the steady-state EMSE are also given.

A noise resilient variable step-size LMS algorithm

This work presents a modified version of the variable step size (VSS) least mean square (LMS) algorithm originally proposed by Kwong and Johnston [IEEE Trans. Signal Process. 40(7) (July 1992) 1633–1642]. The performance of the new algorithm, called noise resilient variable step size (NRVSS), is less sensitive than VSS to the power of the measurement noise. Its implementation requires only a very small increase in the computational complexity. Analytical models are derived for both NRVSS and VSS algorithms for Gaussian signals and small step-size fluctuations. Simulation results show that the NRVSS algorithm has approximately the same transient behavior as VSS but leads to lower steady-state excess mean-square errors as the signal-to-noise ratio (SNR) decreases. The NRVSS algorithm is specially indicated for adaptive interference reduction in biomedical applications.

Rapid identification of a sparse impulse response using an adaptive algorithm in the Haar domain

This paper proposes a fast convergence adaptive algorithm for identifying a sparse impulse response that is rich in spectral content. A sparse impulse response is referred here as a discrete time impulse response that has a large number of zero or near zero coefficients. The basic idea for rapid identification is to automatically determine the locations of the nonzero impulse response coefficients for their adaptation and eliminate the unnecessary adaptation of zero coefficients. The proposed method, which is called the Haar-Basis algorithm, employs a transform approach by modeling the sparse impulse response in the Haar domain. The Haar transform has many basis sets and each of them contains basis vectors that span the entire time domain range. This special nature of the Haar transform allows for the selection of one small subset of adaptive filter coefficients whose basis vectors span the entire range of the impulse response. These coefficients are adapted at the beginning and are then used subsequently to identify, from the hierarchical structure of the Haar transform, the rest of the filter coefficients that must be adapted to correctly model the unknown sparse impulse response. The consequence is avoiding adaptation of many zero coefficients, leading to a dramatic improvement in either convergence speed or steady state excess mean-square error (EASE), while requiring no a priori knowledge such as the number of nonzero coefficients in the unknown sparse impulse response. The proposed algorithm has been tested with a variety of unknown sparse systems using both white noise input and colored input whose spectrum closely resembles that of speech. Simulation results show that the new approach provides promising results.

Rapid identification of a sparse impulse response using an adaptive algorithm in the Haar domain

This paper proposes a fast convergence adaptive algorithm for identifying a sparse impulse response that is rich in spectral content. A sparse impulse response is referred here as a discrete time impulse response that has a large number of zero or near zero coefficients. The basic idea for rapid identification is to automatically determine the locations of the nonzero impulse response coefficients for their adaptation and eliminate the unnecessary adaptation of zero coefficients. The proposed method, which is called the Haar-Basis algorithm, employs a transform approach by modeling the sparse impulse response in the Haar domain. The Haar transform has many basis sets and each of them contains basis vectors that span the entire time domain range. This special nature of the Haar transform allows for the selection of one small subset of adaptive filter coefficients whose basis vectors span the entire range of the impulse response. These coefficients are adapted at the beginning and are then used subsequently to identify, from the hierarchical structure of the Haar transform, the rest of the filter coefficients that must be adapted to correctly model the unknown sparse impulse response. The consequence is avoiding adaptation of many zero coefficients, leading to a dramatic improvement in either convergence speed or steady state excess mean-square error (EASE), while requiring no a priori knowledge such as the number of nonzero coefficients in the unknown sparse impulse response. The proposed algorithm has been tested with a variety of unknown sparse systems using both white noise input and colored input whose spectrum closely resembles that of speech. Simulation results show that the new approach provides promising results.

A class of adaptive step-size control algorithms for adaptive filters

A class of new adaptive step-size control algorithms, which is applicable to most of the LMS-derived tap weight adaptation algorithms, is proposed. Analysis yields a set of difference equations for theoretically calculating the transient behavior of the filter convergence and derives an explicit formula for the steady-state excess mean-square error (EMSE). Experiments for some examples prove that the proposed algorithm is highly effective in improving the convergence rate in both transient and tracking phases. The theoretically calculated convergence is shown to be in good agreement with that obtained through simulations. Alternative formulae of the step-size adaptation for specific tap weight adaptation algorithms are also proposed.

Degradation of the tracking performance of adaptive filtering algorithms with data correlation

Degradation of the tracking performance of adaptive filtering algorithms with data correlation is analyzed. The analysis is done in the context of the identification of a randomly time varying plant. Five algorithms are considered. They are least mean square, recursive least squares, sign, signed regressor, and sign-sign algorithms. The analysis is done in terms of the steady-state excess mean-square error and the steady-state mean-square weight misalignment. The degradation measure adopted is the ratio of the performance index for correlated data to that for white data.

Coherent LMS algorithms

Pilot symbol-assisted adaptive algorithms provide coherent detection for communication systems when the filtering coefficients, such as beamforming weights or equalizer coefficients, are converged. This property can be exploited to speed up the convergence of adaptive algorithms used. In this letter, two new adaptive algorithms, coherent least mean square (C-LMS) and coherent normalized LMS (CN-LMS), are proposed by constraining the desired signal component of the filtering output to be always coherent in phase with the reference signal at each iteration. For same adaption step size, these new algorithms provide faster convergence, while yield same steady-state excess mean-square error as compared with their standard LMS counterparts.

An LMS adaptive second-order Volterra filter with a zeroth-order term: steady-state performance analysis in a time-varying environment

This article studies the steady-state performance of the least mean square (LMS) adaptive second-order Volterra filter (SOVF) with a zeroth-order term for Gaussian inputs. The mean-square-error (MSE) criterion is evaluated first. Then, SOV LMS algorithm-based updating equations are derived. Next, the steady-state performance of the recursions is analyzed for a random walk model for the unknown system parameters, and the steady-state excess MSE is evaluated. Finally, the theoretical performance predictions are shown to be in good agreement with simulation results, especially for small step sizes.

Maximum and minimum tracking performances of adaptive filtering algorithms over target weight cross correlations

This paper is concerned with studying the dependence of the tracking performance of the LMS, RLS, sign, signed regressor, and sign-sign algorithms on the cross correlations among the fluctuations of individual target weights. In practical applications, these cross correlations are usually unknown. Therefore, it is useful for design purposes to find the extreme values of the performance measures over all possible cross correlations. The paper derives, for each one of the above five algorithms, the conditions of target weight cross correlations that maximize and the ones that minimize the steady-state excess mean-square error /spl xi/ and the steady-state mean-square weight misalignment /spl eta/. The relationship between the step sizes /spl mu//sub /spl xi// and /spl mu//sub /spl eta// that minimize /spl xi/ and /spl eta/, respectively, for given target weight cross correlations is studied. Maxima and minima of /spl mu//sub /spl xi// and /spl mu//sub /spl eta// over all target weight cross correlations are found. The necessary and sufficient condition of equality of /spl mu//sub /spl xi// and /spl mu//sub /spl eta// for all target weight cross correlations is derived. A rule that maps the tracking performance measures of the LMS algorithm to the ones of the RLS algorithm is found. The necessary and sufficient condition of equality of the tracking capabilities of the RLS and LMS algorithms for all target weight cross correlations is derived. A measure of the degree of ambiguity of the tracking performance, due to ignorance of target weight cross correlations, is defined. It is found that all of the above algorithms share the same degree of ambiguity, and that this degree increases with the eigenvalue spread of the input covariance matrix.

Leaky LMS algorithm: MSE analysis for Gaussian data

Despite the widespread usage of the leaky LMS algorithm, there has been no detailed study of its performance. This paper presents an analytical treatment of the mean-square error (MSE) performance for the leaky LMS adaptive algorithm for Gaussian input data. The common independence assumption regarding W(n) and X(n) is also used. Exact expressions that completely characterize the second moment of the coefficient vector and algorithm steady-state excess MSE are developed. Rigorous conditions for MSE convergence are also established. Analytical results are compared with simulation and are shown to agree well.

Comparison of RLS, LMS, and sign algorithms for tracking randomly time-varying channels

The performance of adaptive FIR filters governed by the recursive least-squares (RLS) algorithm, the least mean square (LMS) algorithm, and the sign algorithm (SA), are compared when the optimal filtering vector is randomly time-varying. The comparison is done in terms of the steady-state excess mean-square estimation error /spl xi/ and the steady-state mean-square weight deviation, /spl eta/. It is shown that /spl xi/ does not depend on the spread of eigenvalues of the input covariance matrix, R, in the cases of the LMS algorithm and the SA, while it does in the case of the RLS algorithm. In the three algorithms, /spl eta/ is found to be increasing with the eigenvalue spread. The value of the adaptation parameter that minimizes /spl xi/ is different from the one that minimizes /spl eta/. It is shown that the minimum values of /spl xi/ and /spl eta/ attained by the RLS algorithm are equal to the ones attained by the LMS algorithm in any one of the three following cases: (1) if R has equal eigenvalues, (2) if the fluctuations of the individual elements of the optimal vector are mutually uncorrelated and have the same mean-square value, or (3) if R is diagonal and the fluctuations of the individual elements of the optimal vector have the same mean-square value. Conditions that make the values of /spl xi/ and /spl eta/ of the LMS algorithm smaller (or greater) than the ones of the RLS algorithm are derived. For Gaussian input data, the minimum values of /spl xi/ and /spl eta/ attained by the SA are found to exceed the ones attained by the LMS algorithm by 1 dB independently of R and the mutual correlation between the elements of the optimal vector.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

On error-saturation nonlinearities in LMS adaptation

The effect of a saturation-type error nonlinearity in the weight update equation in least-mean-squares (LMS) adaptation is investigated for a white Gaussian data model. Nonlinear difference equations are derived for the eight first and second moments, which include the effect of an error function (erf) saturation-type nonlinearity on the error sequence driving the algorithm. A nonlinear difference equation for the mean norm is explicitly solved using a differential equation approximation and integration by quadratures. The steady-state second-moment weight behavior is evaluated exactly for the erf nonlinearity. Using the above results, the tradeoff between the extent of error saturation, steady-state excess mean-square error, and rate of algorithm convergence is studied. The tradeoff shows that (1) starting with a sign detector, the convergence rate is increased by nearly a factor of two for each additional bit, and (2) as the number of bits is increased further, the additional bit by very little in convergence speed, asymptotically approaching the behavior of the linear algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>