Steady-state Misadjustment Research Articles

A distributed learning based on robust diffusion SGD over adaptive networks with noisy output data

Outliers and noises are unavoidable factors that cause performance of the distributed learning algorithms to be severely reduced. Developing a robust algorithm is vital in applications such as system identification and forecasting stock market, in which noise on the desired signals may intensely divert the solutions. In this paper, we propose a Robust Diffusion Stochastic Gradient Descent (RDSGD) algorithm based on the pseudo-Huber loss function which can significantly suppress the effect of Gaussian and non-Gaussian noises on estimation performances in the adaptive networks. Performance and convergence behavior of RDSGD are assessed in presence of the α-stable and Mixed-Gaussian noises in the stationary and non-stationary environments. Simulation results show that the proposed algorithm can achieve both higher convergence rate and lower steady-state misadjustment than the conventional diffusion algorithms and several robust algorithms.

A robust diffusion recursive generalized modified Blake-Zisserman algorithm for distributed estimation under an adaptive kernel width

In adaptive networks, the performance of distributed estimation is degraded in the presence of non-Gaussian noises. A number of robust criteria for diffusion approaches, such as generalized maximum correntropy and hyperbolic cosine function have been developed towards non-Gaussian/impulsive background noises. However, these algorithms are affected by high steady-state misadjustment. Therefore, to improve the robustness under non-Gaussian noise and decrease steady-state misadjustment, a generalized modified Blake-Zisserman (GMBZ) robust loss function is represented in this study. Also, we propose a new robust diffusion recursive least squares (RLS) based on GMBZ. Additionally, to enhance tracking ability in non-stationary environments, the proposed method is extended by an adaptive strategy for kernel width selection. The convergence analysis and the steady-state performance of the proposed method are also discussed. Simulation results demonstrate the effectiveness and robustness of the proposed algorithms for system identification scenarios in the presence of α-stable noise in stationary and non-stationary environments.

General Robust Proportionate NSAF Algorithm With a Step-Size Converter

To identify sparse systems in the presence of impulsive noises, we propose a general robust proportionate normalized subband adaptive filtering (R-PNSAF) algorithm. Furthermore, to achieve fast convergence and low steady-state misadjustment, we develop a step-size converter (SSC) for R-PNSAF which results in the SSC-R-PNSAF algorithm, which selects the optimal step-size by comparing the mean square deviation at each iteration of the algorithm under given different step-sizes. Simulation results demonstrate the superiority of the proposed scheme in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula> -stable noise scenario over the competing techniques.

Sparsity-Aware Robust Normalized Subband Adaptive Filtering Algorithms With Alternating Optimization of Parameters

This brief proposes a unified sparsity-aware robust normalized subband adaptive filtering (SA-RNSAF) algorithm for identification of sparse systems under impulsive noises. The proposed SA-RNSAF algorithm generalizes different algorithms by defining the robust criterion and sparsity-aware penalty. Furthermore, by alternating optimization of the parameters (AOP) of the algorithm, including the step-size and the sparsity penalty weight, we develop the AOP-SA-RNSAF algorithm, which not only exhibits fast convergence but also obtains low steady-state misadjustment for sparse systems. Simulations in various noise scenarios have verified that the proposed AOP-SA-RNSAF algorithm outperforms existing techniques.

Robust NLMS algorithms with combined step-size against impulsive noises

In the presence of impulsive noises, the normalized least mean M-estimate (NLMM) algorithm has behaved better robustness and convergence than the normalized least mean square (NLMS) algorithm. In order to further solve the trade-off of the NLMM algorithm between convergence rate and steady-state misadjustment, we design a combined step-size (CSS) scheme that combines large and small step-sizes through an adaptive mixing factor, and the resulting CSS-NLMM algorithm obtains fast convergence and low steady-state misadjustment simultaneously. Importantly, the proposed CSS scheme can be straightforwardly extended to other robust NLMS algorithms. Moreover, the performance analysis of the CSS-NLMM algorithm is provided. Simulation results in impulsive noises have supported the effectiveness of the proposed CSS-NLMM algorithm and its performance analysis.

Adaptive Filter based on Absolute Average Error Adaptive Algorithm for Modeling System

Adaptive identification of the bandpass finite impulse response (FIR) filtering system is proposed through this paper using variable step-size least mean square (VSS-LMS) algorithm called absolute average error-based adjusted step-size LMS as an adapted algorithm. This algorithm used to design an adaptive FIR filter by calculating the absolute averaged value for the recently assessed error with the previous one. Then, the step size has been attuned accordingly with consideration of the slick transition of the step size from bigger to smaller to score an achievement through high convergence rate and low steady-state misadjustment over the other algorithms used for the same purpose. The simulation results through the computer demonstrate remarkable performance compared to the traditional algorithm of LMS and another VSS-LMS algorithm (normalized LMS) which used in this paper for the designed filter. The powerful of the filter has been served in the identification system, bandpass filter has been chosen to be identified in the proposed adaptive system identification. It reports conceivable enhancements in the modeling system concerning the time of convergence, which is well-defined as a fast and steady-state adjustment defined with a low level. The designed filter identified the indefinite system with less than 10 samples; meanwhile, other algorithms were taking more than 20 samples for identification. Alongside the fine behavior of preserving the tradeoff between miss adjustment and the capability of tracking, the fewer calculations and computations regarding the algorithm requirement make the applied real-time striking.

Full Mean-Square Analysis of Affine Combination of Two Complex-Valued LMS Filters for Second-Order Non-Circular Inputs

The affine combination of two complex-valued least-mean-squares filters (aff-CLMS) addresses the trade-off between fast convergence rate and small steady-state misadjustment error. However, a rigorous analysis of the aff-CLMS algorithm for second-order non-circular inputs is still under investigation. To this end, the focus in this letter is on the full mean-square analysis of the aff-CLMS algorithm, in which the transient analyses of the mixing parameter, as well as the standard and complementary weight-error covariance matrices, are completed. In addition, we derive the closed-form solutions of the steady-state weight-error power and its complementary version of the aff-CLMS. Finally, the effectiveness of the theoretical analysis is supported by computer simulations.

Complex-valued proportionate affine projection Versoria algorithms and their combined-step-size variants for sparse system identification under impulsive noises

In real life, many engineering problems are modeled in the complex domain (CD). This paper proposes two complex-valued proportionate adaptive filtering (CPAF) algorithms in the CD for identifying the complex sparse systems in impulsive noises. The proposed CPAF algorithms are derived by maximizing the reuse of Versoria cost subjected to the weighted squared Euclidean norm of the filter tap-weight vector difference with the proportionate matrices as the weight of the weighted Euclidean norm. To address the tradeoff problem between the fast convergence rate and low steady-state misadjustment of the CPAF algorithms, two combined-step-size (CSS) CPAF algorithms are also proposed in the CD, which are obtained by introducing a time-varying step-size bound (TVSSB) into the constraint condition of the cost function of the CPAF algorithms. The TVSSB is achieved adaptively by using a modified sigmoidal function (MSF) to combine a large step-size and a small one. In order to be robust against impulsive noises, the MSF is adapted with the aid of a stochastic gradient ascent algorithm which is obtained by maximizing the Versoria cost function with respect to the real part of the a priori prediction error. Simulations on complex sparse system identification under impulsive noises in the CD demonstrate that the performance of the proposed CPAF algorithms is superior to that of the complex-valued variants of the affine projection Versoria, affine projection sign (APS), proportionate APS (PAPS), and improved PAPS (IPAPS) algorithms, also validate the effectiveness of the proposed CSS-CPAF algorithms.

Pseudo affine projection assisted multitasking approach for power quality improvement in grid‐interactive photovoltaic (PV) system

The electronically interfaced photovoltaic (PV) system and its control approach form an intrinsic part of the distributed generation (DG) system. This study proposes pseudo affine projection (PAP)-assisted multitasking control scheme for grid-integrated DG inverter under input grid voltage disturbances. The PAP approach is based on the convergence of active weighted values and is independent of the input signal property, which manifests a discrete characteristic. Additionally, it exhibits a unique capability of updating weight based on multiple, delayed input signal vectors. The fundamental distinguishing feature of the proposed PAP is its intrinsic learning behaviour and therefore it provides an optimal solution to the ingrained paradox between steady-state misadjustment and convergence speed compared to its conventional counterpart affine projection. The PAP-assisted algorithm practically enables DG inverter to perform multiple tasks including suppression of source current harmonics, unbalancing, and neutral current. It also provides load reactive current support with unity power factor operation while transferring the maximum active power into the utility grid. The practicality of the proposed PAP algorithm is investigated through MATLAB/Simulink software. Finally, the experimental results recorded with the dSPACE-1104 based real-time control platform show good to excellent agreement with simulations in the steady-state, as well as transient phenomena.

Improved robust‐mixed‐norm‐based controller for grid‐tied PV systems under voltage disturbances

The grid-tied solar photovoltaic system requires a multiobjective control approach for performing the dual operation of active power transfer at unity power factor and harmonic filtering. In this context, this study proposes an improved robust-mixed-norm (RMN) filter-based multiobjective control strategy for integrating the solar photovoltaic system with the utility grid. The standard RMN filtering control exhibits slow convergence, requires large sampling time and involves high computational cost. Therefore, a switching parameter is designed to toggle between least mean square and least absolute deviation for extracting the benefits of both filtering techniques as per requirement. The switching parameter will also help in minimising the computational intensiveness during hardware implementation and successfully confronts the conflict between convergence speed and steady-state misadjustment in the presence of outliers. The proposed control is implemented with the help of multiple delayed input vectors to extract the fundamental weight component from the non-sinusoidal load current. In addition, the effects of grid voltage distortion and imbalance have been detached by using complex vector filter-based synchronisation technique. The practicality of the proposed multiobjective approach has been verified under different operating conditions of supply and load through both MATLAB/Simulink tool and low-cost microcontroller-STM32F407VGT6-based laboratory prototype.

Sparsity-driven adaptive enhancement of underwater acoustic tonals for passive sonars.

Acoustic tonals, radiated by underwater and surface vehicles, are an important feature for passive sonar detection. An adaptive line enhancer (ALE) is usually employed in passive sonar systems as a preprocessing step to enhance the acoustic tonals from these vehicles. Unfortunately, the performance of the conventional ALE is limited by the high steady-state misadjustment, which is caused by the weight noise in the adaptation process. This paper makes use of the frequency-domain sparsity of these tonals to develop better ALEs for passive sonars. The adaptation of the proposed ALE is performed in the frequency domain. Three typical sparse penalties, l1-norm, log-sum, and l0-pseudo-norm, are incorporated into the cost function of the frequency-domain adaptation, which yield three sparsity-driven ALEs: zero-attracting (ZA), reweighted zero-attracting (RZA), and l0. The simulation shows that the signal-to-noise ratio gains of the ZA-ALE, RZA-ALE, and l0-ALE are 5.9, 8.7, and 9.7 dB, higher than that of the conventional ALE, respectively. The results of processing the real data also validate that all the sparsity-driven ALEs outperform the conventional ALE, and the l0-ALE performs the best. The proposed sparsity-driven l0-ALE is thus a promising candidate for passive sonars to enhance the tonals.

Block-Sparsity Log-Sum-Induced Adaptive Filter for Cluster Sparse System Identification

In this work, an effective adaptive block-sparsity log-sum least mean square (BSLS-LMS) algorithm is proposed to improve the convergence performance of cluster sparse system identification. The main idea of the proposed scheme is to add a new block-sparsity induced term into the cost function of the LMS algorithm. We utilize the ${l}_{{1}}$ norm of adaptive tap weights and log-sum as a mixed constraint, via optimizing the cost function through the gradient descent method, the proposed adaptive filtering method can iteratively move the identified signals towards the optimal solutions, and finally identify the unknown system accurately. The cluster-sparse system response, with block length and arbitrary average sparsity, is generated by a Markov-Gaussian (M-G) model. For the white Gaussian input data, the theoretical formulas of the steady-state mis-adjustment and convergence behaviors of the BSLS-LMS are derived in a general sparse system and a block sparse system, respectively. Numerical experiments demonstrate that the proposed adaptive BSLS-LMS algorithm achieves much better convergence behavior than conventional sparse signal recovery solutions. The experimental study also verifies the consistency between the simulation results and the theoretical analysis.

Sparsity‐based adaptive line enhancer for passive sonars

Adaptive line enhancers (ALEs) have been widely used in passive sonars for enhancing narrowband discrete components (known simply as lines or tonals) radiated by surface or underwater targets. Unfortunately, the performance of the conventional ALE based on the least-mean-square (LMS) adaptive algorithm is limited by the high steady-state misadjustment, which limits the output signal-to-noise ratio (SNR) of the conventional ALE. To break through the limit of the conventional ALE, this study proposes a sparsity-based ALE (SALE). The SALE takes into account the frequency-domain sparsity of narrowband discrete components and realises the adaption in the frequency domain. A l 1 -norm sparse penalty is incorporated into authors’ frequency-domain adaption to reduce the misadjustment dramatically. The simulations show that the SNR gain of the SALE is 6.3 dB higher than that of the conventional ALE. The results of processing the real data collected in the sea trial also demonstrate that authors’ proposed SALE outperforms the conventional ALE. To reduce the computational complexity of the SALE, a fast SALE (FSALE) is also proposed. In authors’ setup, the computational complexity of the FSALE is one order of magnitude lower than that of the SALE, with comparable SNR performance.

Stability Bound of the Initial Mean-Square Deviation of High-Order Stochastic Gradient Adaptive Filtering Algorithms

The paper derives the stability bound of the initial mean-square deviation of an adaptive filtering algorithm based on minimizing the 2Lth moment of the estimation error, with L being an integer greater than 1. The analysis is done for a time-invariant plant with even input probability density function. Dependence of the stability bound on the algorithm step-size, type of the noise distribution, signal-to-noise ratio (SNR), and L is studied. It is shown that the stability bound is decreasing in the step-size. The stability bound decreases as the tail of the noise probability density function becomes lighter for the same steady-state misadjustment. This result is surprising since, for example, it is expected that the stability of the least mean 2Lth algorithm for binary noise distribution is better than that for uniform distribution. The stability bound is proportional to the reciprocal of the SNR, another surprising result. Finally, the stability bound is decreasing in L, which implies a limitation on the ability to improve the performance of the adaptive filter by increasing the order of the cost function. Theoretical results are supported by simulations.

Component‐wise variable step‐size diffusion least mean square algorithm for distributed estimation

In this study, the authors propose a novel component-wise variable step-size diffusion least mean square (CWVSS-DLMS) algorithm for distributed estimation. Different from the traditional variable step-size DLMS (VSS-DLMS) algorithms in which the updating of all components in the weight vector are the same, the step sizes vary from each other on all components at each iteration in the CWVSS-DLMS algorithm. After deriving the CWVSS-DLMS algorithm, they perform theoretical analysis in terms of mean stability and mean-square behaviour. They have also compared the performance of the CWVSS-DLMS algorithm with several other DLMS algorithms through numerical simulations in both stationary and non-stationary environments. Simulation results show that the performance of the CWVSS-DLMS algorithm is more outstanding than the fixed step-size DLMS algorithm, several non-component-wise VSS-DLMS algorithms and existing component-wise VSS-DLMS algorithms in balancing high convergence rates and low steady-state misadjustment. Moreover, they have investigated the performance of the CWVSS-DLMS algorithm for estimating sparse parameter in a distributed way. Simulation results show that the CWVSS-DLMS algorithm can yield satisfying performance in sparsely distributed estimation regardless of the degree of sparsity in the real parameter.

Mean square performance evaluation in frequency domain for an improved adaptive feedback cancellation in hearing aids

Abstract We consider an adaptive linear prediction based feedback canceller for hearing aids that exploits two (an external and a shaped) noise signals for a bias-less adaptive estimation. In particular, the bias in the estimate of the feedback path is reduced by synthesizing the high-frequency spectrum of the reinforced signal using a shaped noise signal. Moreover, a second shaped (probe) noise signal is used to reduce the closed-loop signal correlation between the acoustic input and the loudspeaker signal at low frequencies. A power-transfer-function analysis of the system is provided, from which the effect of the system parameters and adaptive algorithms [normalized least mean square (NLMS) and recursive least square (RLS)] on the rate of convergence, the steady-state behaviour and the stability of the feedback canceller is explicitly found. The derived expressions are verified through computer simulations. It is found that, as compared to feedback canceller without probe noise, the cost of achieving an unbiased estimate of the feedback path using the feedback canceller with probe noise is a higher steady-state misadjustment for the RLS algorithm, whereas a slower convergence and a higher tracking error for the NLMS algorithm.

Diffusion Robust Variable Step-Size LMS Algorithm Over Distributed Networks

In this paper, we propose a novel diffusion robust variable step-size least mean square (DRVSS-LMS) algorithm that is insensitive to impulsive noise for distributed estimation in the network. Conventional diffusion least mean square algorithms are based on the assumption that the background noise obeys Gaussian distribution. However, the performances of these algorithms are severely degraded when impulsive noises occur in the network. By introducing the Huber objective function which can significantly suppress the effect of impulsive noise on estimation performances, we introduce a novel method to respectively deal with the abnormal nodes carrying data contaminated by impulsive noise and the normal nodes without being contaminated by impulsive noise. In addition, the proposed algorithm is assigned with variable step-sizes to further improve the performances of distributed estimation. Simulation results show that the proposed DRVSS-LMS algorithm can achieve both higher convergence rate and lower steady-state misadjustment than several existing robust diffusion LMS algorithms in the presence of impulsive noise.

Stabilization of High-Order Stochastic Gradient Adaptive Filtering Algorithms

The paper is concerned with stabilizing the family of adaptive filtering algorithms based on minimizing the 2Lth moment of the estimation error, with L being an integer greater than 1. Stabilization is attained via a proposed normalization of the algorithm. Mean square stability of the normalized algorithm is proved for a Markov plant for all L > 1. Transient and steady-state performances of the algorithm are analyzed for a time-invariant plant. Expressions are derived for the steady-state misadjustment and convergence time. Tradeoff between the transient and steady-state performances is evaluated. Dependence of this tradeoffon the value of L is studied. This tradeoff is compared with the tradeoff of the NLMS algorithm. The proposed algorithm is dramatically superior to the NLMS algorithm for sub-Gaussian noise. The superiority increases with L, initial mean-square deviation and signal-to-noise ratio. Analytical results are supported by simulations.

NLMS Algorithm Based on a Variable Parameter Cost Function Robust Against Impulsive Interferences

The conventional step-size scaler (SSS) normalized least-mean-square algorithm is robust against impulsive noise. However, the constant parameter in the SSS needs to be controlled to satisfy the conflicting requirements of fast convergence rate and low steady-state misadjustment. Therefore, to address this problem, an adaptive approach for the parameter in the cost function is proposed in this brief. The proposed approach is then tested in system identification and acoustic echo-cancelation scenarios, which have demonstrated that the proposed approach is effective and robust against non-Gaussian impulsive interferences.

Widely linear UKF constant modulus algorithm for blind adaptive beamforming

Based on a uniform linear array, a new widely linear unscented Kalman filter-based constant modulus algorithm (WL-UKF-CMA) for blind adaptive beamforming is proposed. The new algorithm is designed according to the constant modulus criterion and takes full advantage of the noncircular property of the signal of interest (SOI), significantly increasing the output signal-to-interference-plus-noise ratio (SINR), enhancing the convergence speed and decreasing the steady-state misadjustment. Since it requires no known training data, the proposed algorithm saves a large amount of the available spectrum. Theoretical analysis and simulation results are presented to demonstrate its superiority over the conventional linear least mean square-based CMA (L-LMS-CMA), the conventional linear recursive least square-based CMA (L-RLS-CMA), WL-LMS-CMA, WL-RLS-CMA and L-UKF-CMA.