Steady-state Excess Mean Square Error Research Articles

Steady-state mean-square performance analysis of the block-sparse maximum Versoria criterion

The maximum Versoria criterion algorithm (MVC) exhibits lower steady-state misalignment and less complexity as compared to the maximum correntropy criterion (MCC) algorithm in the scenario of non-Gaussian impulsive noises. However, few scholars have discussed improving the MVC algorithm in sparse channels. This paper presents a block-sparse MVC (BS-MVC) algorithm by introducing the regularization norm, which has desirable performance in block-sparse channels and has excellent robustness to non-Gaussian conditions with impulsive noises. The steady-state excess mean-square error (EMSE) is discussed in Gaussian and non-Gaussian noise conditions. The effectiveness of BS-MVC and the theoretical expression is validated using multiple simulations. The BS-MVC achieves a lower steady-state mean-square deviation (MSD) and maintains a fast convergence rate compared with MCC and MVC algorithms.

A distributed FxLMS algorithm for narrowband active noise control and its convergence analysis

A conventional multichannel narrowband active noise control (MNANC) system can effectively suppress low-frequency periodic noise. However, a large number of secondary sources and error sensors are required in the system, which leads to high computational complexity. To address this problem, a diffusion narrowband filtered-x least-mean-square (DNFxLMS) algorithm is proposed based on distributed acoustic sensor networks. The computational burden is dispersed among the nodes over the networks. A comprehensive statistical analysis of the algorithm is conducted to better understand its convergence properties. Specifically, we investigate the convergence behavior of the weight-error vector in the mean and mean-square sense. Based on this analysis, the stability bound of the system is discussed. The expressions for steady-state mean-square deviation (MSD) and excess mean-square error (EMSE) are derived. Extensive computer simulations are conducted to verify the theoretical analysis and to evaluate the usefulness of the DNFxLMS algorithm.

Complex multi-kernel random Fourier adaptive algorithms under the complex kernel risk-sensitive p-power loss

The complex least mean square (CLMS) adaptive algorithm based on the minimum mean square error (MSE) criterion has been widely used for linear applications with Gaussian noises in complex domain. However, the MSE criterion suffers from performance degeneration in the presence of non-Gaussian noise. To address this issue, a novel complex kernel risk-sensitive p-power loss (CKRSP-L) criterion is first constructed to combat non-Gaussian noises. Then, based on the constructed CKRSP-L criterion, a novel complex kernel risk-sensitive p-power (CKRSP) algorithm is proposed to provide robustness to non-Gaussian noises and performance improvement for linear systems, simultaneously. Further, to extend the CKRSP algorithm into nonlinear systems, a novel complex multi-kernel random Fourier mapping (CMRFM) is proposed to transform the original input data into a complex multi-kernel random Fourier features space (CMRFFS), and thus another novel complex multi-kernel random Fourier kernel risk-sensitive p-power (CMRFKRSP) algorithm is presented for nonlinear applications in complex domain. Finally, the steady-state excess mean square errors (SEMSEs) of CKRSP and CMRFKRSP are also calculated for theoretical analysis of performance. Monte Carlo simulations conducted in different noise environments validate the correctness of the obtained SEMSEs and performance advantages of CKRSP and CMRFKRSP.

Robust Time-Varying Parameter Proportionate Affine-Projection-Like Algorithm for Sparse System Identification

Due to its low computational burden, the affine-projection-like (APL) adaptive filtering algorithm has been extensively studied for colored signal input. Recently, a robust APL algorithm was designed by adopting the M-estimate cost function in impulsive noise environment; however, its convergence rate is very slow for sparse system identification. This paper proposed a proportionate APL M-estimate (PAPLM) algorithm, which is derived by using the proportionate matrix to heighten the convergence rate. To maintain good steady-state performance of the PAPLM algorithm, a time-varying parameter PAPLM (TV-PAPLM) algorithm is proposed, which uses a modified exponential function to adjust the time-varying parameter according to the ratio of the mean square score function to the system noise variance. Moreover, the steady-state excess mean-square error performance of PAPLM algorithm is analyzed and obtained in detail. Simulation results reveal that the proposed PAPLM and TV-PAPLM algorithms achieve fast convergence rate and good steady-state performance in sparse system identification.

Projected Kernel Least Mean $p$ -Power Algorithm: Convergence Analyses and Modifications

Sparsified kernel adaptive filters (SKAFs) is an attractive filtering solution with low memory and computational complexity. Most of existing SKAFs are based on the mean square error (MSE) criterion under Gaussian noise assumption for its simplicity and convenience. When the assumption deviates largely from the underlying truth, the performance of these methods could degrade significantly. In this paper, we propose a novel SKAF, named as projected kernel least mean $p$ -power algorithm (PKLMP), based on the mean $p$ -power error (MPE) criterion and vector projection (VP) method. We provide convergence analyses in terms of the stead-state MSE, based on a Taylor expansion method, and derive the lower and upper bounds for the steady-state excess MSE. We also conduct mean convergence analysis for PKLMP, and derive convergence conditions. To exploit the information in the desired outputs, we further derive a modified PKLMP by smoothing the desired signal. Finally, a simple and effective online variable kernel centers strategy is proposed to improve the filtering performance of the proposed KAFs. Simulation results under a static function estimation, a chaotic time-series prediction, and two real-world time-series predictions are conducted and validate the effectiveness of the proposed PKLMP algorithms.

Minimum kernel risk sensitive mean p-power loss algorithms and their performance analysis

The least mean square (LMS) algorithm is optimal for combating Gaussian noises owing to the used minimum mean square error (MSE) criterion in its loss function. However, the MSE criterion is not efficient for non-Gaussian noises. To this end, a new robust minimum kernel risk sensitive mean p-power loss (MKRSP) algorithm is proposed to provide robustness against impulsive noises and improve filtering accuracy, simultaneously. The MKRSP algorithm can generalize the minimum kernel risk sensitive loss (MKRSL) algorithm by setting p=2, and improve filtering accuracy by setting a proper p. Further, the random-Fourier-features MKRSP (RMKRSP) algorithm is developed for improving robustness and filtering accuracy of MKRSP. And the steady-state excess mean square errors (SEMSEs) of MKRSP and RMKRSP are calculated for theoretical analysis. The correctness of the obtained SEMSEs are verified by simulations, and the accuracy advantages of MKRSP and RMKRSP are also confirmed in different noise environments.

On the Stochastic Modeling of the NSVR–IAF–PNLMS Algorithm for Correlated Gaussian Input Data

This paper presents a stochastic model for the normalized smoothed variation rate individual-activation-factor proportionate normalized least-mean-square (NSVR–IAF–PNLMS) algorithm. Specifically, taking into account correlated Gaussian input data, model expressions are derived for predicting the mean weight vector, gain distribution matrix, NSVR metric, learning curve, weight-error correlation matrix, and steady-state excess mean-square error. Such expressions are obtained by considering the time-varying characteristics of the gain distribution matrix. Simulation results are shown confirming the accuracy of the proposed model for different operating conditions.

Variants of Partial Update Augmented CLMS Algorithm and Their Performance Analysis

Naturally complex-valued information or those presented in complex domain are effectively processed by an augmented complex least-mean-square (ACLMS) algorithm. In some applications, the ACLMS algorithm may be too computationally- and memory-intensive to implement. In this paper, a new algorithm, termed partial-update ACLMS (PU-ACLMS) algorithm is proposed, where only a fraction of the coefficient set is selected to update at each iteration. Doing so, two types of partial-update schemes are presented referred to as the sequential and stochastic partial-updates, to reduce computational load and power consumption in the corresponding adaptive filter. The computational cost for full-update PU-ACLMS and its partial-update implementations are discussed. Next, the steady-state mean and mean-square performance of PU-ACLMS for non-circular complex signals are analyzed and closed-form expressions of the steady-state excess mean-square error (EMSE) and mean-square deviation (MSD) are given. Then, employing the weighted energy-conservation relation, the EMSE and MSD learning curves are derived. The simulation results are verified and compared with those of theoretical predictions through numerical examples.

Tracking Analysis of Gaussian Kernel Signed Error Algorithm for Time-Variant Nonlinear Systems

This brief establishes a novel kernel-based model with a random walk variation of the optimum weight coefficients to characterize the time-variant nonlinear system. Then, the steady-state tracking performance of the kernel signed error algorithm (KSEA) with Gaussian kernel is analyzed for the proposed time-variant nonlinear system in the presence of non-Gaussian impulsive noise. The theoretical findings enable us to determine the optimal step-size that minimizes the steady-state excess mean-square error under this non-stationary environment. Simulation results illustrate the usefulness and accuracy of the derived analytical models for characterizing the steady-state tracking behavior of Gaussian KSEA.

Steady-State Mean-Square Performance Analysis of the Affine Projection Sign Algorithm

Due to the robustness against impulsive noises and colored inputs, the affine projection sign algorithm (APSA) is widely used in practice. However, the steady-state mean-square performance of the algorithm has not been analyzed. In this brief, we perform the steady-state mean-square analysis of the APSA by using the energy conservation arguments, the Price’s theorem and some reasonable assumptions. The range of the step size is provided to guarantee the stability of the algorithm. The expression of the steady-state excess mean-square error (EMSE) is also presented. Simulation results confirm the accuracy of the proposed expressions under different operating scenarios.

Widely Linear Complex-Valued Estimated-Input LMS Algorithm for Bias-Compensated Adaptive Filtering With Noisy Measurements

In this paper, a novel widely linear complex-valued estimated-input adaptive filter (WLC-EIAF) is first proposed for processing noisy input and output data in the complex domain. The WLC-EIAF consists of two steps: (i) estimation of noise-free input and (ii) update of the weight vector, which is realized by alternating the minimization of an instantaneous perturbation with both input and output data. Based on the WLC-EIAF method and adopting the least mean-square (LMS) scheme, a widely linear complex-valued estimated-input LMS (WLC-EILMS) algorithm is developed. It is able to achieve an unbiased parameter estimation and, thus, outperforms the widely linear complex-valued LMS (WL-CLMS) algorithm in the presence of noisy input and output. In particular, for Gaussian signals, closed-form expressions are derived for its steady-state excess mean-square error performance. Furthermore, the linear complex-valued estimated-input LMS and linear real-valued estimated-input LMS algorithms are presented, which are two simplified versions of the WLC-EILMS for circular and real-valued signals, respectively. Simulation results demonstrate that the proposed methods achieve significantly improved performance in terms of mean-square deviation and mean-square error when compared to the WL-CLMS and CLMS algorithms.

Affine Projection Versoria Algorithm for Robust Adaptive Echo Cancellation in Hands-Free Voice Communications

In vehicle hands-free communication systems and video teleconferencing systems, echoes are typically encountered. Adaptive filters are usually employed to remove the echoes in these applications. However, large outliers and highly correlated speech input signals are two key factors that limit the performance of the adaptive filters. In this paper, we propose an affine projection Versoria (APV) algorithm, which is obtained by maximizing the summation of Versoria-cost reusing with a constraint on the square of the L 2-norm of the filter weight vector difference. In this way, the features of the Versoria-cost maximization and data reusing are combined and, consequently, the proposed APV algorithm obtains robustness to the large outliers and accelerates the convergence rate for the correlated input signals. The complexity of the proposed APV algorithm is analyzed and then a fast recursive filtering technique is introduced to reduce its complexity. A stability analysis proves that the proposed APV algorithm is convergent. In addition, an analytical expression of the steady-state excess mean-square error for the proposed APV algorithm has been derived and simulated results are in agreement with the theoretical analysis result. Simulations on echo channel estimation and echo cancellation show that the proposed APV algorithm performs much better than the maximum Versoria criterion, affine projection sign, and affine projection algorithms.

Steady-State and Tracking Analyses of the Improved Proportionate Affine Projection Algorithm

The performance analysis of the improved proportionate affine projection (IPAP) algorithm is performed in this brief. Based on the energy-conservation arguments, the steady-state analysis of the IPAP algorithm is performed, which provides a general expression of steady-state excess mean-square error for the proportionate-type affine projection algorithms. The tracking behavior is also studied and the step size that optimizes the tracking performance is provided. Simulation results confirm the accuracy of the proposed expressions under different operating scenarios.

Random Fourier Filters Under Maximum Correntropy Criterion

Random Fourier adaptive filters (RFAFs) project the original data into a high-dimensional random Fourier feature space (RFFS) such that the network structure of filters is fixed while achieving similar performance with kernel adaptive filters. The commonly used error criterion in RFAFs is the well-known minimum mean-square error (MMSE) criterion, which is optimal only under the Gaussian noise assumption. However, the MMSE criterion suffers from instability and performance deterioration in the presence of non-Gaussian noises. To improve the robustness of RFAFs against large outliers, the maximum correntropy criterion (MCC) is applied to RFFS, generating a novel robust random Fourier filter under maximum correntropy (RFFMC). To further improve the filtering accuracy, a random-batch RFFMC (RB-RFFMC) is also presented. In addition, a theoretical analysis on the convergence characteristics and steady-state excess mean-square error of RFFMC and RB-RFFMC is provided to validate their superior performance. Simulation results illustrate that RFFMC and its extension provide desirable filtering performance from the aspects of filtering accuracy and robustness, especially in the presence of impulsive noises.

Complex Kernel Risk-Sensitive Loss: Application to Robust Adaptive Filtering in Complex Domain

Recently, the maximum complex correntropy criterion (MCCC) algorithm has shown its superiority for adaptive filter in complex domain. Compared with the traditional mean square error (MSE) criterion-based algorithms, MCCC uses complex correntropy as similarity measure, which is robust even in the presence of impulse noise. However, the error performance surface of complex correntropic loss (CC-loss) is not optimal, which is steep nearby the optimal solution and flat far from the optimal solution. In this paper, we propose a new similarity measure in complex domain, namely, complex kernel risk-sensitive loss (CKRSL). Based on CKRSL, we derive a new adaptive filter algorithm in the complex domain by using the gradient-based method, i.e., the CKRSL algorithm. Compared with the MCCC algorithm, the CKRSL algorithm has faster convergence rate and higher filtering accuracy. Meanwhile, it is robust against outlier. In addition, we provide the stability analysis and the steady-state excess MSE for the CKRSL algorithm. Simulation results confirm the correctness of the theoretical results and the superiority of the CKRSL algorithm.

The Data‐Reusing MCC‐Based Algorithm and Its Performance Analysis

Maximum correntropy criterion (MCC) provides a robust optimality criterion for non-Gaussian signal processing. In this paper, the weight update equation of the conventional MCC-based adaptive filtering algorithm is modified by reusing the past K input vectors, forming a class of data-reusing MCC-based algorithm, called DRMCC algorithm. Comparing with the conventional MCCbased algorithm, the DR-MCC algorithm provides a much better convergence performance when the input data is correlated. The mean-square stability bound of the DRMCC algorithm has been studied theoretically. For both Gaussian noise case and non-Gaussian noise case, the expressions for the steady-state Excess mean square error (EMSE) of DR-MCC algorithm have been derived. The relationship between the data-reusing order and the steadystate EMSEs is also analyzed. Simulation results are in agreement with the theoretical analysis.

A new pre-whitening transform domain LMS algorithm and its application to speech denoising

In this paper, we propose a new pre-whitening transform domain LMS algorithm. The main idea is to introduce a pre-whitening using a simple finite impulse response decorrelation filter of order one before applying the transform to reinforce its decorrelation. The resulting algorithm has the advantage of using any transform even with low decorrelation. This advantage can be exploited to consider transforms having lower computational and structural complexities than those of the classical transforms. For this purpose, we also investigate the use of other transforms, namely the parametric Fourier and Hartley transforms. This investigation is accomplished by studying the eigenvalue spreads obtained by a given parametric transform and then finding the value of the parameter corresponding to the minimum eigenvalue spread, which is equivalent to the best mean square error (MSE) convergence behavior. This approach provides new attractive transforms for the proposed algorithm. Moreover, we consider the adaptive speech denoising as an application to evaluate the performance of the proposed algorithm. The comparisons between the proposed and conventional algorithms for different transforms are performed in terms of the computational complexity, MSE convergence speed, reached steady state level, residual noise in the denoised signal, steady state excess MSE, misadjustment and output SNR.

Adaptive Networks Under Non-Stationary Conditions: Formulation, Performance Analysis, and Application

In this paper, we thoroughly investigate the tracking behavior of a wide range of adaptive networks under non-stationary conditions. Under these conditions, we study and analyze the mean-square-error performance of all centralized, incremental, and diffusion algorithms for adaptive networks. The closed forms of the steady-state mean-square deviation (MSD) and excess mean-square error (EMSE) criteria are extracted for these algorithms. In addition, we apply our findings to the time-varying autoregressive (TVAR) modeling problem, which is a significant and common engineering application. Generally, time-varying parameters are extracted from a single-point observation; however, the employment of numerous observations may be necessary in some cases. Therefore, we consider a set of observations in the corresponding model identification problem. It is shown that doing so improves the resolution of the estimated parameters. Moreover, we theoretically examine various types of adaptive algorithms for the TVAR model identification problem. Furthermore, node and network behavior analysis is performed for both transient and steady-state conditions. The analysis proves that regardless of the stationary conditions, neither centralized nor incremental adaptive algorithms dominate universally. However, among diffusion algorithms, the adapt-then-combine (ATC) approach demonstrates superior performance, followed by the combine-then-adapt (CTA) approach and the non-cooperative approach (treated as a special case of the diffusion strategy), in that order. The theoretical findings are well supported by simulation results.

Analysis of the TDLMS algorithm operating in a nonstationary environment

This paper presents a stochastic analysis of the transform-domain least-mean-square (TDLMS) algorithm operating in a nonstationary environment (time-varying plant) with real-valued correlated Gaussian input data, from which the analysis for the stationary environment follows as a special case. In this analysis, accurate model expressions are derived describing the transformed mean weight-error behavior, learning curve, transformed weight-error covariance matrix, steady-state excess mean-square error (EMSE), misadjustment, step size for minimum EMSE, degree of nonstationarity, as well as a relationship between misadjustment and degree of nonstationarity. Based on these model expressions, the impact of the algorithm parameters on its performance is discussed, clarifying the behavior of the algorithm vis-à-vis the nonstationary environment considered. Simulation results for the TDLMS algorithm are shown by using the discrete cosine transform, which confirm the accuracy of the proposed model for both transient and steady-state phases.

A new variable length NLMS adaptive algorithm

The Fractional Tap-length Normalized Least Mean-Square (FT-NLMS) adaptive algorithm, a member of the variable tap-length LMS algorithms family, has attracted attention recently due to its robustness and low computational complexity compared with existing variable tap-length algorithms. The parameter Δ in the FT-NLMS algorithm controls convergence speed and bias from the optimum tap-length, which necessitates a compromise between them due to its fixed value. Therefore, a well adjusted time-varying Δ(n) can provide both fast convergence speed and small steady state bias. In this paper, a new robust variable Δ(n) FT-NLMS is proposed that ensures a large value of Δ(n) when the algorithm is far from the optimum with Δ(n) decreasing as the adaptive filter approaches the optimum even in the presence of uncorrelated noise disturbance. Transient and steady state performance analysis of the proposed algorithm will be presented, where expressions for the algorithm steady state excess Mean-Square Error (MSE), steady state tap-length of the adaptive filter, and steady state E{Δ(∞)} are derived. Simulation examples are presented in different experimental conditions, which will demonstrate the advantages of the proposed algorithm over existing variable tap-length NLMS algorithms. Moreover, simulation results will show that the derived theoretical expressions can predict very well the actual behavior of the proposed algorithm.