White Gaussian Input Research Articles

Transient analysis of zero attracting NLMS algorithm without Gaussian inputs assumption

The zero attracting normalized least mean square (ZA-NLMS) algorithm achieves lower steady-state error than the normalized least mean square (NLMS) algorithm for sparse system identification. Most of the available analytical results on several versions of the zero attracting least mean square algorithms assume white Gaussian inputs. This paper presents the individual weight error variance (IWV) analysis of the ZA-NLMS algorithm without Gaussian inputs assumption. The IWV analysis is based on exact individual weight error relation and used to derive the transient and steady-state behavior of the ZA-NLMS algorithm without restricting the input to being Gaussian or white, whereas some assumptions are introduced to overcome weight nonlinearity in evaluating certain expectations involved. Extensive simulations are used to verify the analysis results presented.

Stochastic analysis of the least mean fourth algorithm for non-stationary white Gaussian inputs

This paper studies the stochastic behavior of the least mean fourth (LMF) algorithm for a system identification framework when the input signal is a non-stationary white Gaussian process. The unknown system is modeled by the standard random-walk model. A theory is developed which is based upon the instantaneous average power and the instantaneous average squared power in the adaptive filter taps. A recursion is derived for the instantaneous mean square deviation of the LMF algorithm. This recursion yields interesting results about the transient and steady-state behaviors of the algorithm with time-varying input power. The theory is supported by Monte Carlo simulations for sinusoidal input power variations.

A New Variable Regularized QR Decomposition-Based Recursive Least M-Estimate Algorithm—Performance Analysis and Acoustic Applications

This paper proposes a new variable regularized QR decompPosition (QRD)-based recursive least M-estimate (VR-QRRLM) adaptive filter and studies its convergence performance and acoustic applications. Firstly, variable L2 regularization is introduced to an efficient QRD-based implementation of the conventional RLM algorithm to reduce its variance and improve the numerical stability. Difference equations describing the convergence behavior of this algorithm in Gaussian inputs and additive contaminated Gaussian noises are derived, from which new expressions for the steady-state excess mean square error (EMSE) are obtained. They suggest that regularization can help to reduce the variance, especially when the input covariance matrix is ill-conditioned due to lacking of excitation, with slightly increased bias. Moreover, the advantage of the M-estimation algorithm over its least squares counterpart is analytically quantified. For white Gaussian inputs, a new formula for selecting the regularization parameter is derived from the MSE analysis, which leads to the proposed VR-QRRLM algorithm. Its application to acoustic path identification and active noise control (ANC) problems is then studied where a new filtered-x (FX) VR-QRRLM ANC algorithm is derived. Moreover, the performance of this new ANC algorithm under impulsive noises and regularization can be characterized by the proposed theoretical analysis. Simulation results show that the VR-QRRLM-based algorithms considerably outperform the traditional algorithms when the input signal level is low or in the presence of impulsive noises and the theoretical predictions are in good agreement with simulation results.

A New Variable Regularized Transform Domain NLMS Adaptive Filtering Algorithm—Acoustic Applications and Performance Analysis

This paper proposes a new regularized transform domain normalized LMS (R-TDNLMS) algorithm and studies its mean and mean square convergence performances. The proposed algorithm extends the conventional TDNLMS algorithm by imposing a regularization term on the filter coefficients to reduce the variance of estimators due to the lacking of excitation in a certain frequency band or in the presence of modeling errors. Difference equations describing the mean and mean square convergence behaviors of this algorithm are derived so as to characterize its convergence condition and steady-state excess mean square error (MSE). It shows that regularization can help to reduce the MSE by trading slight bias for variance. Based on this analysis, a new formula to select the regularization parameter for white Gaussian inputs is proposed, which leads to a new variable regularized TDNLMS (VR-TDNLMS) algorithm. Computer simulations are conducted to examine the improved convergence performance, steady-state MSE and robustness to power-varying inputs of the proposed algorithm and verify the effectiveness of the theoretical analysis. Furthermore, the application of the proposed VR-TDNLMS algorithm to the design and implementation of acoustic system identification and active noise control (ANC) systems show that they considerably outperforms traditional TDNLMS algorithms at low excitation or in the presence of modeling errors. Moreover, the theoretical analysis provides simple design formulas for achieving a given excess MSE (EMSE) and step-size bound for stable operation.

Performance Analysis of $l_0$ Norm Constraint Least Mean Square Algorithm

As one of the recently proposed algorithms for sparse system identification, $l_0$ norm constraint Least Mean Square ($l_0$-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The performance of $l_0$-LMS is quite attractive compared with its various precursors. However, there has been no detailed study of its performance. This paper presents all-around and throughout theoretical performance analysis of $l_0$-LMS for white Gaussian input data based on some reasonable assumptions. Expressions for steady-state mean square deviation (MSD) are derived and discussed with respect to algorithm parameters and system sparsity. The parameter selection rule is established for achieving the best performance. Approximated with Taylor series, the instantaneous behavior is also derived. In addition, the relationship between $l_0$-LMS and some previous arts and the sufficient conditions for $l_0$-LMS to accelerate convergence are set up. Finally, all of the theoretical results are compared with simulations and are shown to agree well in a large range of parameter setting.

Behavior of the least mean square algorithm with a periodically time‐varying input power

SUMMARYThe paper analyzes the transient and steady‐state performances of a least mean square algorithm in the rarely‐studied situation of a time‐varying input power. A scenario of periodic pulsed variation of the input power is considered. The analysis is carried out in the context of tracking a Markov plant with a white Gaussian input. It is shown that the mean square deviation (MSD) converges to a periodic sequence having the same period as that of the variation of the input power. Expressions are derived for the convergence time and the steady‐state peak MSD. Surprisingly, it is found that neither the transient performance nor the steady‐state performance degrades with rapid variation of the input power. On the other hand, slow input power variation causes degradation in both the transient and steady‐state performances for given amplitude of variation of the input power. In the case of a time‐invariant plant, neither rapid nor slow variation of the input power causes degradation in the steady‐state performance. On the other hand, there is degradation in the transient performance for slow variation of the input power. Copyright © 2012 John Wiley & Sons, Ltd.

Mean-square stability of the Normalized Least-Mean Fourth algorithm for white Gaussian inputs

Normalized forms of adaptive algorithms are usually sought in order to obtain convergence properties independent of the input signal power. Such is the case of the well-known Normalized LMS (NLMS) algorithm. The Least-Mean Fourth (LMF) adaptive algorithm has been shown to outperform LMS in different situations. However, the LMF stability is dependent on both the signal power and on the adaptive weights initialization. This paper studies the behavior of two normalized forms of the LMF algorithm for Gaussian inputs. Contrary to what could be expected, the mean-square stability of both normalized algorithms is shown to be dependent upon the input signal power. Thus, the usefulness of the NLMF algorithm is open to question.

Stochastic analysis of an error power ratio scheme applied to the affine combination of two LMS adaptive filters

The affine combination of two adaptive filters that simultaneously adapt on the same inputs has been actively investigated. In these structures, the filter outputs are linearly combined to yield a performance that is better than that of either filter. Various decision rules can be used to determine the time-varying parameter for combining the filter outputs. A recently proposed scheme based on the ratio of error powers of the two filters has been shown by simulation to achieve nearly optimum performance. The purpose of this paper is to present a first analysis of the statistical behavior of this error power scheme for white Gaussian inputs. Expressions are derived for the mean behavior of the combination parameter and for the adaptive weight mean-square deviation. Monte Carlo simulations show good to excellent agreement with the theoretical predictions.

Adaptive Nonlinear System Identification in the Short-Time Fourier Transform Domain

In this paper, we introduce an adaptive algorithm for nonlinear system identification in the short-time Fourier transform (STFT) domain. The adaptive scheme consists of a parallel combination of a linear component, represented by crossband filters between subbands, and a quadratic component, which is modeled by multiplicative cross-terms. We adaptively update the model parameters using the least-mean-square (LMS) algorithm, and derive explicit expressions for the transient and steady-state mean-square error (MSE) in frequency bins for white Gaussian inputs. We show that estimation of the nonlinear component improves the MSE performance only when the power ratio of nonlinear to linear components is relatively high. Furthermore, as the number of crossband filters increases, a lower steady-state MSE may be obtained at the expense of slower convergence. Experimental results support the theoretical derivations.

三角多項式を用いたＨａｍｍｅｒｓｔｅｉｎモデルの同定

This paper studies a procedure for identifying Hammerstein systems from input-output data, where the static nonlinearity is approximated by the trigonometric polynomials. Signals generated by the trigonometric function with white Gaussian inputs are analyzed in terms of the persistent excitation of linear time-invariant systems. Based on the analysis, an identification algorithm for Hammerstein systems is proposed via a subspace identification method. Numerical simulation results are also included to illustrate the effectiveness of the present algorithm.

An Affine Combination of Two LMS Adaptive Filters—Transient Mean-Square Analysis

This paper studies the statistical behavior of an affine combination of the outputs of two least mean-square (LMS) adaptive filters that simultaneously adapt using the same white Gaussian inputs. The purpose of the combination is to obtain an LMS adaptive filter with fast convergence and small steady-state mean-square deviation (MSD). The linear combination studied is a generalization of the convex combination, in which the combination factor lambda(n) is restricted to the interval (0,1). The viewpoint is taken that each of the two filters produces dependent estimates of the unknown channel. Thus, there exists a sequence of optimal affine combining coefficients which minimizes the mean-square error (MSE). First, the optimal unrealizable affine combiner is studied and provides the best possible performance for this class. Then two new schemes are proposed for practical applications. The mean-square performances are analyzed and validated by Monte Carlo simulations. With proper design, the two practical schemes yield an overall MSD that is usually less than the MSDs of either filter.

Stochastic Gradient Algorithm Under (h,φ)-Entropy Criterion

Motivated by the work of Erdogmus and Principe, we use the error (h,φ)-entropy as the supervised adaptation criterion. Several properties of the (h,φ)-entropy criterion and the connections with traditional error criteria are investigated. By a kernel estimate approach, we obtain the nonparametric estimator of the instantaneous (h,φ)-entropy. Then, we develop the general stochastic information gradient algorithm, and derive the approximate upper bound for the step size in the adaptive linear neuron training. Moreover, the (h,φ) pair are optimized to improve the performance of the proposed algorithm. For the finite impulse response identification with white Gaussian input and noise, the exact optimum φ function is derived. Finally, simulation experiments verify the results and demonstrate the noticeable performance improvement that may be achieved by the optimum (h,φ)-entropy criterion.

静的非線形奇関数をもつＨａｍｍｅｒｓｔｅｉｎモデルの同定

This paper studies a procedure for identifying a Hammerstein model equipped with a static nonlinear odd function, where the only output is measured and the input is white Gaussian. A statistical property of the process generated by the static nonlinear odd function with a white Gaussian input is analyzed by means of a moment generating function. A proposed procedure for identifying the Hammerstein model consists of two steps : The first step is to identify the linear dynamical part via a stochastic subspace identification method based on a block LQ-decomposition; the second step is to estimate the static nonlinear odd function by using distribution functions. Numerical simulation results are also included.

Capacity of Multivariate Channels with Multiplicative Noise: Random Matrix Techniques and Large-N Expansions (2)

We study memoryless, discrete time, matrix channels with additive white Gaussian noise and input power constraints of the form Y i = ∑ j H ij X j + Z i , where Y i , X j and Z i are complex, i = 1… m, j = 1… n, and H is a complex m× n matrix with some degree of randomness in its entries. The additive Gaussian noise vector is assumed to have uncorrelated entries. Let H be a full matrix (non-sparse) with pairwise correlations between matrix entries of the form E[H ik H * jl] = 1/n C ij D kl, where C, D are positive definite Hermitian matrices. Simplicities arise in the limit of large matrix sizes (the so called large-n limit) which allow us to obtain several exact expressions relating to the channel capacity. We study the probability distribution of the quantity f(H) = log (1+PH † SH) . S is non-negative definite and hermitian, with TrS = n and P being the signal power per input channel. Note that the expectation E[f(H)], maximised over S, gives the capacity of the above channel with an input power constraint in the case H is known at the receiver but not at the transmitter. For arbitrary C, D exact expressions are obtained for the expectation and variance of f(H) in the large matrix size limit. For C = D = I, where I is the identity matrix, expressions are in addition obtained for the full moment generating function for arbitrary (finite) matrix size in the large signal to noise limit. Finally, we obtain the channel capacity where the channel matrix is partly known and partly unknown and of the form α; I+ β H, α,β being known constants and entries of H i.i.d. Gaussian with variance 1/n. Channels of the form described above are of interest for wireless transmission with multiple antennae and receivers.

Optimal Sequences and Sum Capacity of Symbol Asynchronous CDMA Systems

The optimal signature sequences that maximize the sum capacity of a direct sequence code-division multiple-access (CDMA) system are characterized in the general case of symbol delay profile and user power constraints. It is shown that the optimal sum capacity of the symbol asynchronous system equals that of the symbol synchronous system with the same user power constraints. With the optimal signature sequence set, the maximum sum capacity is achieved with white Gaussian input signals. The existence of the optimal signature sequence set is proved by the proposal of an explicit construction method for arbitrary user delay profiles and power constraints.

A Gaussian Input Is Not Too Bad

We consider the problem of choosing a robust input for communicating over an input constrained additive-noise channel where the noise distribution is arbitrary. We show that the mutual information rate achievable using a white Gaussian input never incurs a loss of more than half a bit per sample with respect to the power constrained capacity. For comparison, for the family of colored Gaussian noise channels a white Gaussian input loses at most log(e)/2e/spl ap/0.265 bit per sample with respect to the optimum water-pouring solution. For general input constraints, we derive a formula for choosing the best input in the min-max capacity loss (bound) sense. The bound on the capacity loss is tight for pulse position modulation (PPM) in the presence of a bursty jammer.

A cumulant based algorithm for the identification of input–output quadratic systems

The identification of a special class of polynomial models is pursued in this paper. In particular a parameter estimation algorithm is developed for the identification of an input–output quadratic model excited by a zero mean white Gaussian input and with the output corrupted by additive measurement noise. Input–output crosscumulants up to the fifth order are employed and the identification problem of the unknown model parameters is reduced to the solution of successive triangular linear systems of equations that are solved at each step of the algorithm. Simulation studies are carried out and the proposed methodology is compared with two least squares type identification algorithms, the output error method and a combination of the instrumental variables and the output error approach. The proposed cumulant based algorithm and the output error method are tested with real data produced by a robotic manipulator.

Simultaneous Estimation of System and Input Parameters from Output Measurements

System identification of very large structures is of necessity accomplished by analyzing output measurements, as in the case of ambient vibration surveys. Conventional techniques typically identify system parameters by assuming (arguably) that the input is locally Gaussian white, and in so doing, effectively reduce the number of degrees of freedom of the estimation problem to a more tractable number. This paper describes a new approach that has several novel attributes, among them, elimination of the need for the Gaussian white input assumption. The approach involves a filter applied to an identification problem formulated in the frequency domain. The filter simultaneously estimates both system parameters and input excitation characteristics. The estimates we obtain are not guaranteed to be unique (as is true in all other approaches: simultaneous estimation of both system and input possesses too many degrees of freedom to guarantee uniqueness); but we do, nonetheless, identify system parameters and input excitation characteristics that are physically plausible and intuitively reasonable, without making input excitation assumptions. Simulated and laboratory experimental data are used to verify the algorithm and demonstrate its advantages over conventional approaches.

Recursive ARMAX speech analysis based on a glottal source model with phase compensation

A sophisticated new speech analysis method based on a Glottal-ARMAX (Auto Regressive and Moving Average eXogenous) model with phase compensation is proposed in this paper. A Glottal-ARMAX model consists of two kinds of inputs, i.e., glottal source model excitation and a white Gaussian input, and a vocal tract ARMAX model. The proposed method can estimate the phase-compensated excitation of a glottal source model and vocal tract parameters simultaneously with pitch synchronous. In the method, ARMAX identification using a modified MIS (Model Identification System) method is adopted to estimate ARMAX parameters, and the hybrid approach of genetic algorithm (GA) and simulated annealing (SA) is employed to solve efficiently the non-linear simultaneous optimization between glottal sources and the parameters of vocal tract ARMAX model. Furthermore, in order to compensate phase distortion, phase compensation using an all-pass filter is introduced within a generation loop in the GA method. Experiments using the Glottal-ARMAX synthetic speech and natural speech uttered by a male speaker demonstrate the efficiency of the proposed method.

Transient and tracking performance bounds of the sign-sign algorithm

The paper provides a rigorous tracking analysis of the sign-sign algorithm when used in the identification of a time-varying plant with a white Gaussian input. The plant parameters vary according to a random walk model. The assumptions allow nonstationarity of the plant input, plant noise, and increments of the plant parameters. Upper bounds are derived for the long-term averages of the mean of the weight misalignment norm, mean absolute error, mean square weight misalignment, and mean square error. These bounds hold for all values of the algorithm step size, all initial filter weight settings, and all degrees of nonstationarity of the plant input, plant noise, and plant parameter increments. Lower bounds of the mean square weight misalignment and mean square error are also derived. The step sizes that minimize the above bounds are derived. A transient analysis of the algorithm is done in the case of a time-invariant plant. A tight lower bound of the convergence time is derived. The above analytical results are supported by computer simulations.