Excess Mean Square Error Research Articles

Affine Projection Versoria Algorithm for Robust Adaptive Echo Cancellation in Hands-Free Voice Communications

In vehicle hands-free communication systems and video teleconferencing systems, echoes are typically encountered. Adaptive filters are usually employed to remove the echoes in these applications. However, large outliers and highly correlated speech input signals are two key factors that limit the performance of the adaptive filters. In this paper, we propose an affine projection Versoria (APV) algorithm, which is obtained by maximizing the summation of Versoria-cost reusing with a constraint on the square of the L 2-norm of the filter weight vector difference. In this way, the features of the Versoria-cost maximization and data reusing are combined and, consequently, the proposed APV algorithm obtains robustness to the large outliers and accelerates the convergence rate for the correlated input signals. The complexity of the proposed APV algorithm is analyzed and then a fast recursive filtering technique is introduced to reduce its complexity. A stability analysis proves that the proposed APV algorithm is convergent. In addition, an analytical expression of the steady-state excess mean-square error for the proposed APV algorithm has been derived and simulated results are in agreement with the theoretical analysis result. Simulations on echo channel estimation and echo cancellation show that the proposed APV algorithm performs much better than the maximum Versoria criterion, affine projection sign, and affine projection algorithms.

Recursive Minimum Complex Kernel Risk-Sensitive Loss Algorithm.

The maximum complex correntropy criterion (MCCC) has been extended to complex domain for dealing with complex-valued data in the presence of impulsive noise. Compared with the correntropy based loss, a kernel risk-sensitive loss (KRSL) defined in kernel space has demonstrated a superior performance surface in the complex domain. However, there is no report regarding the recursive KRSL algorithm in the complex domain. Therefore, in this paper we propose a recursive complex KRSL algorithm called the recursive minimum complex kernel risk-sensitive loss (RMCKRSL). In addition, we analyze its stability and obtain the theoretical value of the excess mean square error (EMSE), which are both supported by simulations. Simulation results verify that the proposed RMCKRSL out-performs the MCCC, generalized MCCC (GMCCC), and traditional recursive least squares (RLS).

Robust least mean logarithmic square adaptive filtering algorithms

The conventional logarithm cost-based adaptive filters, e.g., the least mean logarithmic square (LMLS) algorithm, cannot combat impulsive noises at the filtering process. To address this issue, we present a novel robust least mean logarithmic square (RLMLS) algorithm by using a generalized logarithmic cost function. The proposed RLMLS algorithm can provide robustness against impulsive noises with the improvement of filtering accuracy. For theoretical analysis, the mean square analysis of RLMLS is provided in terms of the mean square deviation (MSD) and excess mean-square error (EMSE) with a white Gaussian reference. For further performance improvement in different noises, the variable step-size RLMLS (VSSRLMLS) based on the statistics of error is proposed to improve the convergence rate and steady-state mean square error, simultaneously. Analytical results and superiorities of RLMLS and VSSRLMLS in the context of system identification are supported by simulations from the aspects of filtering accuracy and robustness in Gaussian and impulse noises.

Steady-State and Tracking Analyses of the Improved Proportionate Affine Projection Algorithm

The performance analysis of the improved proportionate affine projection (IPAP) algorithm is performed in this brief. Based on the energy-conservation arguments, the steady-state analysis of the IPAP algorithm is performed, which provides a general expression of steady-state excess mean-square error for the proportionate-type affine projection algorithms. The tracking behavior is also studied and the step size that optimizes the tracking performance is provided. Simulation results confirm the accuracy of the proposed expressions under different operating scenarios.

Random Fourier Filters Under Maximum Correntropy Criterion

Random Fourier adaptive filters (RFAFs) project the original data into a high-dimensional random Fourier feature space (RFFS) such that the network structure of filters is fixed while achieving similar performance with kernel adaptive filters. The commonly used error criterion in RFAFs is the well-known minimum mean-square error (MMSE) criterion, which is optimal only under the Gaussian noise assumption. However, the MMSE criterion suffers from instability and performance deterioration in the presence of non-Gaussian noises. To improve the robustness of RFAFs against large outliers, the maximum correntropy criterion (MCC) is applied to RFFS, generating a novel robust random Fourier filter under maximum correntropy (RFFMC). To further improve the filtering accuracy, a random-batch RFFMC (RB-RFFMC) is also presented. In addition, a theoretical analysis on the convergence characteristics and steady-state excess mean-square error of RFFMC and RB-RFFMC is provided to validate their superior performance. Simulation results illustrate that RFFMC and its extension provide desirable filtering performance from the aspects of filtering accuracy and robustness, especially in the presence of impulsive noises.

A modified non-negative LMS algorithm for online system identification

Adaptive filters are useful solutions for system identification problem where an optimization problem is used to formulated the estimation of the unknown model coefficients. The nonnegativity constraint is one of the most frequently used constraint which can be imposed to avoid physically unreasonable solutions and to comply with physical characteristics. In this letter, we propose a new variant of non-negative least mean square (NNLMS) that has a less mean square error (MSE) value and faster convergence rate. We provide both mean weight behavior and transient excess mean-square error analysis for proposed algorithm. Simulation results validate the theoretical analysis and show the effectiveness of our proposed algorithm.

Model-driven online parameter adjustment for zero-attracting LMS

Abstract Zero-attracting least-mean-square (ZA-LMS) algorithm has been widely used for online sparse system identification. Similarly to most adaptive filtering algorithms and sparsity-inducing regularization techniques, ZA-LMS appears to face a trade-off between convergence speed and steady-state performance, and between sparsity level and estimation bias. It is therefore important, but not trivial, to optimally set the algorithm parameters. To address this issue, a variable-parameter ZA-LMS algorithm is proposed in this paper, based on a model of the stochastic transient behavior of the ZA-LMS. By minimizing the excess mean-square error (EMSE) at each iteration on the basis of a white input assumption, we obtain closed-form expression of the step-size and regularization parameter. To improve the performance, we introduce the same strategy for the reweighted ZA-LMS (RZA-LMS). Simulation results illustrate the effectiveness of the proposed algorithms and highlight their performance through comparisons with state-of-the-art algorithms, in the case of white and correlated inputs.

Correntropy-Based Evolving Fuzzy Neural System

In this paper, a correntropy-based evolving fuzzy neural system (CEFNS) is proposed for approximation of nonlinear systems. Different from the commonly used mean-square error criterion, correntropy has a strong outliers rejection ability through capturing the higher moments of the error distribution. Considering the merits of correntropy, this paper brings contributions to build evolving fuzzy neural system (EFNS) based on the correntropy concept to achieve a more stable evolution of the rule base and update of the rule parameters instead of the commonly used mean-square error criterion. The correntropy-EFNS (CEFNS) begins with an empty rule base, and all rules are evolved online based on the correntropy criterion. The consequent part parameters are tuned based on the maximum correntropy criterion, where the correntropy is used as the cost function so as to improve the non-Gaussian noise rejection ability. The steady-state convergence performance of the CEFNS is studied through the calculation of the steady-state excess mean square error (EMSE) in two cases: Gaussian noise; and non-Gaussian noise. Finally, the CEFNS is validated through a benchmark system identification problem, a Mackey-Glass time series prediction problem as well as five other real-world benchmark regression problems under both noise-free and noisy conditions. Compared with other EFNSs, the simulation results show that the proposed CEFNS produces better approximation accuracy using the least number of rules and training time and also owns superior non-Gaussian noise handling capability.

Improved proportionate-type sparse adaptive filtering under maximum correntropy criterion in impulsive noise environments

An improved proportionate adaptive filter based on maximum correntropy criterion (IP-MCC) is proposed for identifying systems with variable sparsity in an impulsive noise environment. Utilization of the MCC mitigates the effect of the impulsive noise while improved proportionate concept exploits the underlying system sparsity to improve the convergence rate. Performance analysis of the proposed IP-MCC reveals that the steady-state excess mean square error (EMSE) of the proposed IP-MCC filter is similar to that of MCC filter. Extensive simulations demonstrate that the proposed IP-MCC outperforms state-of-the-art algorithms in terms of convergence rate and detailed complexity analysis reveals that IP-MCC requires much less computational effort.

Efficient adaptive noise cancellation techniques in an IOT Enabled Telecardiology System

An increasing number of elderly­­­­ and disabled people urge the need for a health care monitoring system which has the capabilities for analyzing patient health care data to avoid preventable deaths. Medical Telemetry is becoming a key tool in assisting patients living remotely where a “Real-time Remote Critical Health Care Monitoring System” (RRCHCMS) can be utilized for the same. The RRCHCMS is capable of receiving and transmitting data from a remote location to a location that has the capability to diagnose the data and affect decision making and further providing assistance to the patient. During the cardiac analysis, several artifacts solidly affect the ST segment, humiliate the signal quality, frequency resolution, and results in large amplitude signals in ECG that simulate PQRST waveform and cover up the miniature features that are useful for clinical monitoring and diagnosis. In this paper, several leaky based adaptive filter structures for cardiac signal improvement are discussed. The Circular Leaky Least Mean Square (CLLMS) algorithm being the steepest drop strategy for dropping the mean squared error gives a better result in comparison with the Least Mean Square (LMS) algorithm. To enlarge the filtering ability some variants of LMS, Normalized Least Mean Square (NLMS), CLLMS, Variable Step Size CLLMS (VSS-CLLMS) algorithms are used in both time domain (TD) and frequency domain (FD). At last, we applied this algorithm on cardiac signals occurred due to MIT-BIH database. The performance of CLLMS algorithm is better compared to LLMS counterparts in conditions of Signal to Noise Ratio Improvement (SNRI), Excess Mean Square Error (EMSE) and Misadjustment (MSD). When compared to all other algorithms VSS-CLLMS gives superior SNRI. These values are 13.5616dB and 13.7592dB for Baseline Wander (BW) and Muscle Artifact (MA) removal.

Complex Kernel Risk-Sensitive Loss: Application to Robust Adaptive Filtering in Complex Domain

Recently, the maximum complex correntropy criterion (MCCC) algorithm has shown its superiority for adaptive filter in complex domain. Compared with the traditional mean square error (MSE) criterion-based algorithms, MCCC uses complex correntropy as similarity measure, which is robust even in the presence of impulse noise. However, the error performance surface of complex correntropic loss (CC-loss) is not optimal, which is steep nearby the optimal solution and flat far from the optimal solution. In this paper, we propose a new similarity measure in complex domain, namely, complex kernel risk-sensitive loss (CKRSL). Based on CKRSL, we derive a new adaptive filter algorithm in the complex domain by using the gradient-based method, i.e., the CKRSL algorithm. Compared with the MCCC algorithm, the CKRSL algorithm has faster convergence rate and higher filtering accuracy. Meanwhile, it is robust against outlier. In addition, we provide the stability analysis and the steady-state excess MSE for the CKRSL algorithm. Simulation results confirm the correctness of the theoretical results and the superiority of the CKRSL algorithm.

Transfer Learning in Adaptive Filters: The Nearest Instance Centroid-Estimation Kernel Least-Mean-Square Algorithm

We propose a novel nearest-neighbors approach to organize and curb the growth of radial basis function network in kernel adaptive filtering (KAF). The nearest-instance-centroid-estimation (NICE) kernel least-mean-square (KLMS) algorithm provides an appropriate time-space tradeoff with good performance. Its centers in the input/feature space are organized by quasi-orthogonal regions for greatly simplified filter evaluation. Instead of using all centers to evaluate/update the function approximation at every new point, a linear search among the iteratively-updated centroids determines the partial function to be used, naturally forming locally-supported partial functionals. Under this framework, partial functionals that compose the adaptive filter are quickly stored/retrieved based on input, each corresponding to a specialized “spatial-band” subfilter. The filter evaluation becomes the update of one of the subfilters, creating a content addressable filter bank (CAFB). This CAFB is incrementally updated for new signal applications with mild constraints, always using the past-learned partial filter sums, opening the door for transfer learning and significant efficiency for new data scenarios, avoiding training from scratch as have been done since the invention of adaptive filtering. Using energy conservation relation, we show the sufficient condition for mean square convergence of the NICE-KLMS algorithm and establish the upper and lower bounds of steady-state excess-mean-square-error (EMSE). Simulations on chaotic time-series prediction demonstrate similar levels of accuracy as existing methods, but with much faster computation involving fewer input samples. Simulations on transfer learning using both synthetic and real-world data demonstrate that NICE CAFB can leverage previously learned knowledge to related task or domain.

Design of an Intelligent q-LMS Algorithm for Tracking a Non-stationary Channel

Tracking of a time-varying channel is a challenging task, especially when channel is non-stationary. In this work, we propose a time-varying q-LMS algorithm to efficiently track a random-walk channel. To do so, we first perform tracking analysis of the q-LMS algorithm in a non-stationary environment and then derive the expressions for the transient and steady-state tracking excess mean-square-error (EMSE). Thus, we evaluate an optimum value of q parameter which minimizes the tracking EMSE. Next, by utilizing the derived optimum q, we design a time-varying mechanism to vary the parameter q according to the estimation of instantaneous error energy which provides faster convergence in the initial phase while attain a lower EMSE near final stages of adaptation.

Performance Analysis of Incremental LMS Over Flat Fading Channels

We study the effect of fading in the communication channels between sensor nodes on the performance of incremental least mean square (ILMS) algorithm, and derive steady state performance metrics, including the mean-square deviation (MSD), excess mean-square error (EMSE) and mean-square error (MSE). We obtain conditions for mean convergence of the ILMS algorithm and show that in the presence of fading channels, the ILMS algorithm is asymptotically biased. Furthermore, the dynamic range for the mean stability depends only on the mean channel gain, and under simplifying technical assumptions, we show that the MSD, EMSE, and MSE are non-decreasing functions of the channel gain variances, with mean-square convergence to the steady states possible only if the channel gain variances are limited. We derive sufficient conditions to ensure mean-square convergence and verify our results through simulations.

Diffusion LMS With Correlated Regressors II: Performance

Over the past decade, diffusion LMS and real-time consensus (RTC) adaptive network algorithms have been widely used and studied. However, their performance analysis has relied on white regressor assumptions, which are not satisfied in practice. Here, we consider both temporally correlated regressors and noise and give mean squared deviation (MSD) and excess mean squared error (EMSE) performance results for the first time. Furthermore, we employ a novel second-order mixed time-scale stochastic averaging approach to study realization-wise fluctuations of the parameter estimation error. Critically, our new results show that both temporally correlated regressors and noise affect the performance of the algorithms, even for small gains. Simulations illustrate the results. This is Part II of an earlier work on stability. Together, these works show that the Adapt-Then-Combine (ATC), Combine-Then-Adapt (CTA), and RTC algorithms are strikingly similar in terms of stability and performance in the slow adaptation regime. In particular, ATC, CTA, and RTC have the same network MSD and EMSE to first order. Moreover, we conclude that existing MSD and EMSE results are insufficient to first order when both the regressors and noise are temporally correlated.

Q-State Space Least Mean Family of Algorithms

It is generally known that model-based estimation algorithms (such as Kalman filter and its family) perform better than the non-model-based algorithms [such as least mean square (LMS), recursive least squares] due to extra information available in terms of system dynamics (which can be used to provide state space model of the system). However, the computational complexity of the model based algorithms is very high. On the other hand, the convergence performance of model based least mean type algorithms [such as state space least mean (SSLM) algorithms] is slower and highly dependent on the step-size choice. Thus, the larger step size can provide faster convergence but gives poor steady-state excess mean square error (EMSE). To meet this conflicting demand, we propose to employ the q-calculus to minimize the generalized least mean cost function. The main advantage of using the q-calculus is that it can provide a nonlinear correction term in the adaptation of the state estimate vector. Consequently, this results in an intelligent adaptation by providing both faster convergence in the initial phase of adaptation and a lower steady-state EMSE in the final phase. The developed algorithms are termed as q-state space least mean (q-SSLM) algorithms. The performance of the proposed q-state space least mean square (q-SSLMS) algorithm is also investigated both in terms of convergence in the mean and the mean square sense. The supremacy of the proposed algorithm is validated by performing several simulations and it is also contrasted with the performance of the well-known Kalman filter. Finally, the theoretical convergence analysis is also validated via simulations.

Convergence performance analysis of an adaptive kernel width MCC algorithm

A fixed kernel width in MCC algorithm imposes a trade-off among robustness, convergence rate and steady-state accuracy. With a variable kernel width, the adaptive kernel width MCC (AMCC) algorithm can improve the learning speed of the MCC algorithm especially when the initial weight vector is far away from the optimal weight vector. In this paper, the steady-state excess mean square error (EMSE) of the AMCC algorithm is studied based on energy conservation relation. In addition, a novel convergence measure called initial convergence rate is introduced to evaluate the convergence speed at the beginning of the learning. Simulation experiments are carried out to verify the theoretical analysis and confirm the desirable performance of the AMCC algorithm in several different non-Gaussian noise environments.

Frequency domain quaternion adaptive filters: Algorithms and convergence performance

Recently, adaptive filtering was extended to quaternion-valued systems. Quaternion-valued algorithms exhibit improved geometrical properties compared with real- and complex-valued algorithms. Moreover, working in the frequency domain allows a fast execution along with a good convergence performance. In this work, we propose three different quaternion-valued adaptive algorithms operating in the frequency domain. Convergence properties are also analyzed: in particular, the step size stability range is obtained in relation to the eigenvalues of the input autocorrelation matrix and the Excess Mean Square Error (EMSE) is expressed in relation to the algorithm parameters. Finally, simulations support the proposal.

Efficient Cardiac Signal Enhancement Techniques Based on Variable Step Size and Data Normalized Hybrid Signed Adaptive Algorithms

In remote health care monitoring, the extraction of high resolution cardiac signals is an important task. For this the cardiac signal (CS) needed to be enhanced. Among various filtering techniques, the adaptive noise cancellation (ANC) is a promising methodology. In adaptive filtering least mean square (LMS) algorithm is the fundamental enhancement algorithm. However, it suffers with slow convergence and weight drift problem in non-stationery environment. In order to improve the performance of ANC this paper proposes to implement an ANC methodology based on variable step size on the normalization of fundamental LMS algorithm for CS enhancement. Based on such strategy this research implements ANC using hybrid algorithm called variable normalized LMS (VNLMS) algorithm. Further, to improve the convergence characteristics, to filter the ability and to minimize computational complexity some versions of VNLMS algorithms are implemented too. Finally, the proposed ANCs are tested using real CS obtained from MIT-BIH data base. The performance evaluation is carried based on signal to noise ratio improvement (SNRI) and excess mean square error (EMSE).

Efficient and Low Complexity Noise Cancellers for Cardiac Signal Enhancement using Proportionate Adaptive Algorithms

Objectives: To enhance the quality of Cardiac Signal for perfect diagnosis by the doctor. Methods/Statistical Analysis: We are introducing some adaptive filter structures for Cardiac Signal (CS) enhancement for the extraction of high resolution cardiac signals and these structures were based on the Proportionate Normalized Least Mean Square (PNLMS) algorithm. The main advantage of PNLMS over the conventional techniques is extraction of sparse coefficients, suitably weighing them and fast convergence. These ANCs are tested using MIT-BIH database to compare the performance. Findings: We consider Signal to Noise Ratio (SNR), Excess Mean Square Error (EMSE), Misadjustment (MSD), convergence curves and residual error plots as performance measures. Among the ANCs tested, PMNSRLMA based ANC is found to be better with reference to the considered performance measures and computational complexity. The average SNRI achieved by this ANC is 18.8856dBs for PLI elimination, 8.7580dBs for BW elimination, 8.5106dBs for MA elimination and 8.5012dBs for EM elimination. From the above results it is clear that in practical biotelemetry applications to minimize the computational complexity of the noise canceller we combine PNLMS with signature algorithms. Again, to reduce the complexity in the denominator of the normalized recursion, we use maximum normalized version of PNLMS. Finally, these variations result in seven algorithms in addition to PNLMS. Based on these algorithms, we develop various Adaptive Noise Cancellers (ANCs) to eliminate artifacts present in the CS and to present best quality signal to the doctor for diagnosis. Application/ Improvement: The standard of Cardiac Signal can be enhanced by improving data acquisition methods.