Steady-state Mean-square Deviation Research Articles

A variable step size total least squares affine-projection-like algorithm: Formula derivation and performance analysis

In the errors-in-variables (EIV) model, the steady state characteristics of the affine-projection-like (APL) algorithm are poor. Hence, the total least squares APL (TLS-APL) algorithm different from the bias-compensated is proposed, which can reduce the adverse influence of input and output noise on weight update. Then, the local extremum of the TLS-APL algorithm is verified, that is, the expected parameter is the optimal value of the iterative estimation. Moreover, the mean, mean square, and steady-state mean square deviation (MSD) performance of the proposed TLS-APL algorithm are deduced by using the Kronecker product and vec operator, respectively. To balance the convergence characteristics and steady-state characteristics, the step size optimization of the TLS-APL algorithm, namely the variable step size TLS-APL (VSS-TLS-APL) algorithm, is derived from the analysis of MSD. Considering the existence of gradient errors, the fitting effects of theoretical steady-state MSD curves and actual steady-state MSD curves under different parameters are analyzed separately, which proves the rationality of theoretical derivation. In addition, the VSS-TLS-APL algorithm with optimized step size is compared with the TLS-APL algorithm with fixed step size in the application of system identification, and the effect of the variable step size (VSS) is verified. The experiments of system identification and acoustic echo cancelation prove that the VSS-TLS-APL algorithm is superior to the existing known algorithms.

Steady-State Performance Analysis of the Arctangent LMS Algorithm With Gaussian Input

The adaptive filtering algorithms based on the arctangent cost function framework have shown robustness against impulsive noise. In this brief, the standard least mean square (LMS) algorithm under this framework is concentrated on, which is called the arctangent LMS (ATLMS) algorithm. The steady-state excess mean square error (EMSE) and mean square deviation (MSD) of the ATLMS algorithm are analyzed using the energy conservation relation. In stationary environment, both Gaussian and non-Gaussian situations are discussed. The closed-form expressions of the steady-state EMSE and MSD are obtained using Taylor’s expansion. In non-stationary environment, a first-order random-walk model is used for modeling the time-varying optimal weight. Theoretical steady-state performance and the optimal step size are derived. Simulation results under different noise environments verify the validity of our theoretical findings.

Steady-state mean-square performance analysis of the block-sparse maximum Versoria criterion

The maximum Versoria criterion algorithm (MVC) exhibits lower steady-state misalignment and less complexity as compared to the maximum correntropy criterion (MCC) algorithm in the scenario of non-Gaussian impulsive noises. However, few scholars have discussed improving the MVC algorithm in sparse channels. This paper presents a block-sparse MVC (BS-MVC) algorithm by introducing the regularization norm, which has desirable performance in block-sparse channels and has excellent robustness to non-Gaussian conditions with impulsive noises. The steady-state excess mean-square error (EMSE) is discussed in Gaussian and non-Gaussian noise conditions. The effectiveness of BS-MVC and the theoretical expression is validated using multiple simulations. The BS-MVC achieves a lower steady-state mean-square deviation (MSD) and maintains a fast convergence rate compared with MCC and MVC algorithms.

Performance Analysis of the Standard Constrained Maximum Versoria Criterion Based Adaptive Algorithm

Recently, robust adaptive filtering approaches relied on the Maximum Versoria Criterion (MVC) have gained the attention of researchers and have been widely studied. In this brief, with the energy conservation approach, transient and steady-state mean-square deviation (MSD) analysis of the standard constrained MVC (CMVC) are derived under both Gaussian and non-Gaussian noise distributions. For a Gaussian noise condition, an accurate solution is obtained, while for non-Gaussian noise conditions, we used approximate Taylor's expansion to derive the transient and steady-state MSD. Finally, we evaluated the theoretical analysis with some numerical simulations of system identification in different Gaussian and non-Gaussian noise scenarios to validate the finding.

Performance analysis of the augmented complex-valued least mean kurtosis algorithm

The augmented complex-valued least mean kurtosis (ACLMK) algorithm was proposed to enhance the performance of adaptive filters by utilizing the negated kurtosis of the error signal. This paper provides a comprehensive physical insight into its stochastic behaviors and a more thorough performance analysis. By using Isserlis’ theorem, the evolutions of both the covariance and complementary covariance matrices of the weight error vector are established, based on which the expressions for transient mean-square deviation (MSD) and excess mean-square error (EMSE) are derived. Moreover, the higher-order moments of the measurement noise are also taken into account. Different from the existing framework which treats the ACLMK as a variable step-size augmented complex-valued least mean square (ACLMS) algorithm, the steady-state mean-square performance of the ACLMK is analyzed through the approximate uncorrelating transform (AUT) to yield the theoretical steady-state MSD and EMSE. The proposed framework of performance analysis is shown to provide more accurate models for the theoretical MSD and MSE. Simulation results are provided to validate the theoretical findings for second-order noncircular input signals in second-order circular and noncircular noisy environments.

A distributed FxLMS algorithm for narrowband active noise control and its convergence analysis

A conventional multichannel narrowband active noise control (MNANC) system can effectively suppress low-frequency periodic noise. However, a large number of secondary sources and error sensors are required in the system, which leads to high computational complexity. To address this problem, a diffusion narrowband filtered-x least-mean-square (DNFxLMS) algorithm is proposed based on distributed acoustic sensor networks. The computational burden is dispersed among the nodes over the networks. A comprehensive statistical analysis of the algorithm is conducted to better understand its convergence properties. Specifically, we investigate the convergence behavior of the weight-error vector in the mean and mean-square sense. Based on this analysis, the stability bound of the system is discussed. The expressions for steady-state mean-square deviation (MSD) and excess mean-square error (EMSE) are derived. Extensive computer simulations are conducted to verify the theoretical analysis and to evaluate the usefulness of the DNFxLMS algorithm.

An improved mean-square performance analysis of the diffusion least stochastic entropy algorithm

The diffusion least stochastic entropy (DLSE) algorithm yields a better performance than the diffusion complex-valued least mean-square (DCLMS) algorithm for second-order noncircular noise. The existing theoretical steady-state mean-square deviation (MSD) shows a low accuracy due to the use of improper estimations and approximations. Besides, its transient MSD has not been thoroughly studied. This paper provides an improved performance analysis. We first improve the method of estimating the circularity coefficient (CC) of the error signal, which is required for the update equation of the DLSE. This improvement leads to a better match between the theoretical and simulated MSD and makes the transient MSD analysis mathematically tractable. The MSD analysis is performed by using the Gaussian moment factorizing theorem with some statistical assumptions. Besides, another key performance measure, i.e., the excess mean-square error (EMSE) of the DLSE is also analyzed. Simulation results validate the theoretical findings of the DLSE algorithm for second-order noncircular input signals and noise.

A reweighted l0-norm-constraint LMS algorithm for sparse system identification

The l0-norm-constraint algorithm is widely used in sparse system identification due to its attractive performance. However, the algorithm is sensitive to the tuning parameters and its convergence speed can be further improved due to the small attraction range of the zero attractor. This paper proposes a reweighted l0-norm-constraint Least Mean Square (l0-RLMS) algorithm which expands the attraction range of the zero attractor to accelerate the convergence with even lower mean-square deviation (MSD) value and lower sensitivity to the tuning parameters. The theoretical analysis of the proposed algorithm, along with numerical simulations and comparisons with the latest sparse algorithms, is carried out. The analysis and simulations show that the l0-RLMS algorithm has lower steady-state MSD, lower sensitivity of tuning parameters and lower complexity than the l0-norm-constraint algorithm.

An Optimized Zero-Attracting LMS Algorithm for the Identification of Sparse System

This paper introduces an optimized zero-attractor to improve the performance of least mean square (LMS)-based algorithms for the identification of sparse system. Compared with previous LMS-based algorithms for sparse system identification, the performance of the proposed optimized zero-attracting LMS (OZ-LMS) is much less sensitive to the tuning parameters and measurement noise power, and performs much better for sparse system. Comprehensive performance analysis of the mean-square deviation (MSD) of OZ-LMS is derived in detail. Moreover, the parameter selection rules for optimal steady-state MSD are discussed. Simulation results, using white Gaussian noise and speech input signals, show improved performance over existing methods. Furthermore, we show that the numerical results of OZ-LMS agree with the theoretical predictions.

A full mean-square analysis of CNSAF algorithm for noncircular inputs

A full performance analysis of complex normalized subband adaptive filter (CNSAF) algorithm will provide guidelines for designing the adaptive filter. However, because of the noncircular characteristic of complex-value signal, the complementary mean-square performance analysis of the CNSAF algorithm has not been presented in the literature. In order to give the detailed theoretical expressions of the CNSAF algorithm, the present study first analyzes the mean-square deviation (MSD) with the energy-conservation method, and then the complementary mean-square derivation (CMSD) behavior is given using pseudo-energy-conservation method. Analytical expressions are obtained for the transient MSD and CMSD of the CNSAF algorithm. Also, the steady-state MSD and CMSD are predicted based on the closed-form expressions. Besides, the analysis results are not constrained by the distribution of input signals. Finally, simulation results obtained for diffferent inputs present a good consistence with the analytical results.

On the Convergence Behavior of Partitioned-Block Frequency-Domain Adaptive Filters

Partitioned-block frequency-domain adaptive filter (PBFDAF) algorithms have become very popular, in particular, for acoustic signal processing. However, the stochastic behavior of PBFDAFs has not been extensively examined. This paper presents a comprehensive statistical analysis for a family of the overlap-save PBFDAF algorithms with 50% overlap, including the transient and steady-state performance. The frequency-domain equations of the PBFDAFs are transformed into the time-domain counterparts, which allows us to carry out the analysis completely in the time domain. By means of the independence assumption and vectorization operation, theoretical models for the mean and mean-square behavior of the PBFDAF algorithms are established without restricting the distribution of the inputs to being Gaussian. Specifically, the mean weight behavior and corresponding steady-state solution are provided. Closed-form expressions for the mean-square deviation (MSD) and mean-square error (MSE) are derived. The upper bound on the step size for the mean and mean-square stability of the PBFDAFs is specified. The theoretical model presents new insights into the convergence properties of the PBFDAFs with a sufficient number of coefficients. It was revealed that both the constrained and unconstrained PBFDAFs converge to the Wiener solution. However, the mean weight vector of the unconstrained PBFDAF algorithms cannot converge to the true solution for any inputs, while that of the constrained version can. The presented theory explains why the steady-state MSD of the unconstrained PBFDAF algorithm is larger than that of the constrained version but their minimum MSE is the same. Monte Carlo simulations provide very good support for our theory.

Steady-state mean-square deviation analysis of improved ℓ0-norm-constraint LMS algorithm for sparse system identification

The ℓ0-norm-constraint LMS (ℓ0-LMS) algorithm is one of the widely used sparse LMS algorithms for the identification of sparse system, and its performance is quite attractive compared to other precursors. However, ℓ0-LMS is still confronted with some limitations on the optimal parameter selection and the estimated coefficient accuracy of sparse system identification. In this paper, we proposed an improved ℓ0-LMS (ℓ0-ILMS) algorithm to address these limitations. The convergence condition and the parameter selection rules for optimal steady-state mean-square deviation (MSD) of ℓ0-ILMS are discussed. Compared with ℓ0-LMS, the steady-state MSD of ℓ0-ILMS is lower and less sensitive to the tuning parameters and measurement noise power. Numerical simulations comparing the performance of standard LMS, ℓ0-LMS and ℓ0-ILMS demonstrate the effectiveness of ℓ0-ILMS.

A switched variable step size NLMS adaptive filter

This paper studies the statistical behavior of a switched variable step size Normalized Least Mean Square adaptive filter. The purpose of the switching is to obtain a Normalized Least Mean Square adaptive filter combination with fast convergence and small steady-state mean-square weight deviation (MSD). Previous works studied variable step-size Least Mean Square (LMS) and Normalized Mean Square (NLMS) adaptive filters using a measured quantity (say instantaneous squared error) in a nonlinear recursion for varying the step size. Here, it is proposed to avoid the nonlinear recursion in a novel way that is based upon a simple physical premise: For several practical applications, what is required is a faster initial convergence rate phase, followed by a slower convergence rate period that allows for a desired steady-state performance. The fast NLMS transient behavior is obtained with a fixed step size. After convergence, the step size is reduced to a smaller value to satisfy a desired tradeoff between the second transient and the desired steady-state MSD. Thus, during the first transient, the Switched VSS-NLMS adaptive weights will converge faster or as fast as any VSS-NLMS algorithm. Of course, during the second phase, some variable step size algorithms may eventually outperform the Switched VSS-NLMS algorithm. However, in many cases, this can be a small price to pay in exchange for the theoretical and computational simplicity. No new theory is needed to predict the behavior of the nonlinear recursion for the step size. Furthermore, the analytical theory for the MSD of the new scheme is well-known.

Diffusion Average-Estimate Bias-Compensated LMS Algorithms Over Adaptive Networks Using Noisy Measurements

In this paper, we consider the problem of distributed estimation over adaptive networks in the presence of noisy input, output and communication links. First, a diffusion average-estimate bias-compensated LMS (D-ABC-LMS) algorithm is devised for processing these noisy measurements. Different from the existing diffusion schemes, it consists of three steps: (i) bias-compensated weight update, (ii) average estimation and (iii) weight combination, where the second step utilizes a moving average technique to process imprecise exchange weights caused by communication link noise. Then, we analyze the stability and convergence of the D-ABC-LMS algorithm, and derive closed-form expressions to predict its steady-state mean-square deviation (MSD) and network MSD (NMSD). In addition, by introducing one more step via the adaptive mixture of the noisy exchange weight vectors and their denoised estimates, we propose a diffusion mixed average-estimate bias-compensated LMS (D-MABC-LMS) method for increasing the convergence rate of the D-ABC-LMS scheme. Finally, computer simulation results show the superiority of the proposed algorithms over previously reported techniques using noisy measurements over adaptive networks.

On the Influence of Bias-Correction on Distributed Stochastic Optimization

Various bias-correction methods such as EXTRA, gradient tracking methods, and exact diffusion have been proposed recently to solve distributed deterministic optimization problems. These methods employ constant step-sizes and converge linearly to the exact solution under proper conditions. However, their performance under stochastic and adaptive settings is less explored. It is still unknown whether, when and why these bias-correction methods can outperform their traditional counterparts with noisy gradient and constant step-sizes. This work studies the performance of exact diffusion under the stochastic and adaptive setting, and provides conditions under which exact diffusion has superior steady-state mean-square deviation (MSD) performance than traditional algorithms without bias-correction. In particular, it is proven that this superiority is more evident over sparsely-connected network topologies such as lines, cycles, or grids. Conditions are also provided under which exact diffusion method can or degrade the performance of traditional methods. Simulations are provided to validate the theoretical findings.

Influence of input noises on the mean-square performance of the normalized subband adaptive filter algorithm

The normalized subband adaptive filter (NSAF) algorithm has gained a large amount of attention owning to its fast convergence rate for correlated inputs. However, the convergence analysis of the NSAF algorithm has not been researched extensively, especially in the presence of noisy inputs which is frequently encountered in the applications of system identification and channel estimation. In this paper, we perform the performance analysis of the NSAF algorithm under noisy inputs. According to several reasonable assumptions and approximations, the expressions of the transient-state and steady-state mean-square deviation (MSD) are derived. Simulations under different kinds of environments confirm the accuracy of our theoretical expressions.

M73 GENOME-WIDE ANALYSIS OF PHARMACOGENOMIC EFFECTS ON ALCOHOL USE DISORDER TREATMENT OUTCOMES

A stochastic analysis of the probabilistic least-mean-square (Prob-LMS) algorithm would be a useful guideline for designing the adaptive filter. However, no analytical expressions for the stochastic analysis of the Prob-LMS algorithm have been reported in the literature. Hence, this paper analyzes the mean-deviation and mean-square-deviation (MSD) behavior of the Prob-LMS algorithm for the general case of an unknown Gauss-Markov channel. Analytical expressions are derived for the transient and steady-state MSD of the Prob-LMS algorithm. Monte Carlo simulations for fixed and time varying channels show excellent agreement between the simulated and theoretical MSD for a wide range of parameters such as SNR, filter length and input signal statistics. Monte Carlo MSD simulation results are presented for the Prob-LMS algorithm which compare favorably to several well-known VSS algorithms.

Mean-square-deviation analysis of probabilistic LMS algorithm

A stochastic analysis of the probabilistic least-mean-square (Prob-LMS) algorithm would be a useful guideline for designing the adaptive filter. However, no analytical expressions for the stochastic analysis of the Prob-LMS algorithm have been reported in the literature. Hence, this paper analyzes the mean-deviation and mean-square-deviation (MSD) behavior of the Prob-LMS algorithm for the general case of an unknown Gauss-Markov channel. Analytical expressions are derived for the transient and steady-state MSD of the Prob-LMS algorithm. Monte Carlo simulations for fixed and time varying channels show excellent agreement between the simulated and theoretical MSD for a wide range of parameters such as SNR, filter length and input signal statistics. Monte Carlo MSD simulation results are presented for the Prob-LMS algorithm which compare favorably to several well-known VSS algorithms.

A Unified Approach to the Statistical Convergence Analysis of Frequency-Domain Adaptive Filters

The frequency-domain adaptive filter (FDAF) algorithms are used in many applications due to their computational efficiency and good convergence performance. Many efforts have been made to analyze the convergence behavior of FDAF in the past. However, the previous analyses are based on coarse approximations of overlap-save procedures or small step-size assumptions and hence came to inaccurate predictions of the transient and steady-state performance. Moreover, the rigorous step-size bound in the mean-square sense has not been provided so far. To address these problems, we carry out an extensive analysis of the convergence behaviors for a family of FDAFs based on the overlap-save structure. Using a unified update equation of four FDAFs, the state recursions of the mean weight-error vector and the weight-error covariance matrix are worked out rigorously in the frequency domain, which are then used to investigate the mean-square deviation (MSD) and mean-square error (MSE) during the transient phase. In addition, we obtain the analytical results on the steady-state MSD and MSE, and the bound on the step size for both the mean and mean-square stabilities. Specifically, the analysis presented here does not restrict the regression data to being Gaussian or white. Computer simulations in a system identification scenario confirmed that the proposed theoretical results are much more accurate than the previous approaches.

Partial Diffusion Kalman Filtering for Distributed State Estimation in Multiagent Networks.

Many problems in multiagent networks can be solved through distributed learning (state estimation) of linear dynamical systems. In this paper, we develop a partial-diffusion Kalman filtering (PDKF) algorithm, as a fully distributed solution for state estimation in the multiagent networks with limited communication resources. In the PDKF algorithm, every agent (node) is allowed to share only a subset of its intermediate estimate vectors with its neighbors at each iteration, reducing the amount of internode communications. We analyze the stability of the PDKF algorithm and show that the algorithm is stable and convergent in both mean and mean-square senses. We also derive a closed-form expression for the steady-state mean-square deviation criterion. Furthermore, we show theoretically and by numerical examples that the PDKF algorithm provides a trade-off between the estimation performance and the communication cost that is extremely profitable.