Step-size Parameter Research Articles

A variable-parameter normalized mixed-norm (VPNMN) adaptive algorithm

Since both the least mean-square (LMS) and least mean-fourth (LMF) algorithms suffer individually from the problem of eigenvalue spread, so will the mixed-norm LMS-LMF algorithm. Therefore, to overcome this problem for the mixed-norm LMS-LMF, we are adopting here the same technique of normalization (normalizing with the power of the input) that was successfully used with the LMS and LMF separately. Consequently a new normalized variable-parameter mixed-norm (VPNMN) adaptive algorithm is proposed in this study. This algorithm is derived by exploiting a time-varying mixing parameter in the traditional mixed-norm LMS-LMF weight update equation. The time-varying mixing parameter is adjusted according to a well-known technique used in the adaptation of the step-size parameter of the LMS algorithm. In order to study the theoretical aspects of the proposed VPNMN adaptive algorithm, our study also addresses its convergence analysis, and assesses its performance using the concept of energy conservation. Extensive simulation results corroborate our theoretical findings and show that a substantial improvement, in both convergence time and steady-state error, can be obtained with the proposed algorithm. Finally, the VPNMN algorithm proved its usefulness in a noise cancellation application where it showed its superiority over the normalized least-mean square algorithm.

New Scaled Three-Term Memoryless vm-Methods for Solving Nonlinear Unconstraint Problems

Four new scaled three-term memoryless VM methods for solving nonlinear unconstrained problems are presented. The basic idea is to deal with Al-Bayati's (1991) and Biggs's (1983) self-scaling VMupdates in the frame of new scaled CG-methods. Birgin-Martinez (2001) and Abbo (2007) positive parameters are used to scale these spectral CG-methods. The new search directions are reset to the standard Steepest Descent (SD) direction when Powell's (1977) restarting criterion holds. Andrei's (2010) acceleration scheme of the step-size parameter has been employed in the new proposed methods to improve the efficiency of such methods. Under common assumptions; the new methods are proved to be globally convergent. Computational results for a set consisting of 100 unconstrained optimization test problems show that the new methods substantially outperforms the scaled memoryless BFGS method.

Steady-State Analysis of Diffusion LMS Adaptive Networks With Noisy Links

In this correspondence, we analyze the effects of noisy links on the steady-state performance of diffusion least-mean-square (LMS) adaptive networks. Using the established weighted spatial-temporal energy conservation argument, we derive a variance relation which contains moments that represent the effects of noisy links. We evaluate these moments and derive closed-form expressions for the mean-square deviation (MSD), excess mean-square error (EMSE) and mean-square error (MSE) to explain the steady-state performance at each individual node. The derived expressions, supported by simulations, reveal that unlike the ideal link case, the steady-state MSD, EMSE, and MSE curves are not monotonically increasing functions of the step-size parameter when links are noisy. Moreover, the diffusion LMS adaptive network does not diverge due to noisy links.

Robust Distributed Learning in Wireless Sensor Network using Efficient Step Size

The distributed learning methods in wireless sensor network are giving better performance when the noise is uniformly distributed in all the sensor nodes. In general practice the noise variance at different nodes varies non uniformly. If constant step size is used for all the nodes then the learning performance will be poor. In order to improve the robustness of incremental least mean squares (ILMS) adaptive learning algorithm against the spatial variation of observation noise statistics over the network, an efficient step–size assignment is presented here. When the noise variance information of all the nodes are available, then the step size parameter in the adaptive algorithm can adjust to obtain better performance. The proposed distributed algorithm is simulated in MATLAB and the performance is evaluated in terms of mean square deviation (MSD), excess mean square error (EMSE) and mean square error (MSE). The simulation result shows the robust against spatial noise variance.

A Novel Orthonormalization Matrix Based Fast and Stable DPM Algorithm for Principal and Minor Subspace Tracking

We note that the well-known Fast Rayleigh's quotient-based Adaptive Noise Subspace (FRANS), FRANS with Householder transformation (HFRANS), and fast data projection method (FDPM) algorithms all inherit from the data projection method (DPM) algorithm, but with different orthonormalization matrices. Starting from the DPM, we analyze the orthonormalization matrices of all these algorithms and develop a novel orthonormalization matrix for our algorithm. Based on this novel orthonormalization matrix, a fast and stable implementation of the DPM algorithm which has the merits of both the FRANS and FDPM approaches is investigated for principal and minor subspace tracking. The proposed algorithm can switch between the principal and minor subspace tracking with a simple sign change of its step size parameter. Moreover, it reaches the 3 <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">np</i> lower bound of the dominant complexity and guarantees the orthonormality of the tracked subspace. The numerical stability of our algorithm is established theoretically and tested numerically. The strengths and weaknesses of the proposed algorithm to some existing subspace tracking algorithms are demonstrated using a de facto benchmark example. Simulation results are presented to demonstrate the effectiveness of the tracking algorithm advocated.

Implementation of FastICA on DSP for Blind Source Separation

Fast independent component analysis (FastICA) algorithm separates the independent sources from their mixtures by measuring non-Gaussian. However, it may lead to the slowing down of convergence speed in the algorithm with improper step, even non-convergence, To overcome these shortcomings and meet the needs of the separation of mixed sound signals, improved FastICA algorithm is used in this paper, which converges much faster and does not need to select the step size parameters manually. Moreover, a detailed description of blind source separation on DSP platform is concluded. Finally, the improved algorithm is applied to the voice signal separation, whose experimental results demonstrate the effectiveness of the presented hardware FastICA as expected.

A Grey Synthesis Approach to Efficient Architecture Design for Temporal Difference Learning

Temporal difference (TD) constitutes a class of methods for learning predictions in multistep prediction problems. The most important application of these methods is to temporal credit assignment in reinforcement learning. Although these TD procedures work in theory and in principle, its success is contingent on proper selection of parametric values. As well, its learning is majorly based on repeated exposures, which may not always be practical or feasible. This paper examines the issues of the efficient and general implementation of TD for hardware implementation of reinforcement learning algorithms by synthesizing the series of discounted sum of rewards along time. The proposed algorithm eliminates all step size parameters and improves data efficiency based on a synthetic approach of Grey theory. This paper also presents the stability of the proposed algorithm from the viewpoint of Grey theory. The algorithm along with a critic-actor reinforcement learning model is implemented in a System-on-a-Programmable-Chip (SOPC) board. In addition to comparing with the renowned model, adaptive heuristic critic (AHC), the results of experiments demonstrate that the proposed control mechanism can learn to control a system with very little a priori knowledge. Meanwhile, the effect of uncertainty in interactions between the system and the environment can be relaxed to some extent in the learning process of the proposed reinforcement learning agent.

Analysis of Acoustic Feedback/Echo Cancellation in Multiple-Microphone and Single-Loudspeaker Systems Using a Power Transfer Function Method

In this work, we analyze a general multiple-microphone and single-loudspeaker audio processing system, where a multichannel adaptive system is used to cancel the effect of acoustic feedback/echo, and a beamformer processes the feedback/echo canceled signals. We introduce and derive an accurate approximation of a frequency domain measure - the power transfer function - and show how it can be used to predict the convergence rate, system stability bound and the steady-state behavior of the entire cancellation system across frequency and time. We consider three example adaptive algorithms in the cancellation system: the least mean square, normalized least mean square, and the recursive least squares algorithms. Furthermore, we derive expressions to determine the step size parameter in the adaptive algorithms to achieve a desired system behavior, e.g., convergence rate at a specific frequency. Finally, we compare and discuss the performance of all three adaptive algorithms, and we verify the derived expressions through simulation experiments.

Statistical Analysis of Steepest Descend and LMS Detection Algorithms for MIMO Systems

We study the performance of steepest descent (SD) and least mean square (LMS) algorithms applied to linear detection for multiple-input multiple-output (MIMO) systems in a correlated Rayleigh fading environment. By using random matrix theory, we first study stability for a fixed step size parameter. Then, we consider two always-stable channel-adaptive strategies for the choice of the step size and analytically evaluate their performance. Finally, we derive bounds on the mean value of misadjustment for the LMS algorithm.

Accelerated iterative hard thresholding

The iterative hard thresholding algorithm (IHT) is a powerful and versatile algorithm for compressed sensing and other sparse inverse problems. The standard IHT implementation faces several challenges when applied to practical problems. The step-size and sparsity parameters have to be chosen appropriately and, as IHT is based on a gradient descend strategy, convergence is only linear. Whilst the choice of the step-size can be done adaptively as suggested previously, this letter studies the use of acceleration methods to improve convergence speed. Based on recent suggestions in the literature, we show that a host of acceleration methods are also applicable to IHT. Importantly, we show that these modifications not only significantly increase the observed speed of the method, but also satisfy the same strong performance guarantees enjoyed by the original IHT method.

The Delta-Phase Spectrum With Application to Voice Activity Detection and Speaker Recognition

For several reasons, the Fourier phase domain is less favored than the magnitude domain in signal processing and modeling of speech. To correctly analyze the phase, several factors must be considered and compensated, including the effect of the step size, windowing function and other processing parameters. Building on a review of these factors, this paper investigates a spectral representation based on the Instantaneous Frequency Deviation, but in which the step size between processing frames is used in calculating phase changes, rather than the traditional single sample interval. Reflecting these longer intervals, the term delta-phase spectrum is used to distinguish this from instantaneous derivatives. Experiments show that mel-frequency cepstral coefficients features derived from the delta-phase spectrum (termed Mel-Frequency delta-phase features) can produce broadly similar performance to equivalent magnitude domain features for both voice activity detection and speaker recognition tasks. Further, it is shown that the fusion of the magnitude and phase representations yields performance benefits over either in isolation.

Application of adaptive noise canceller based on fixed-point algorithm for real-time somatosensory evoked potential monitoring

Objective To efficiently detect somsatosensory evoked potential (SEP) using field programmable gate array (FPGA) real-time system, fixed-point algorithm based adaptive noise canceller (ANC) was designed to improve signal to noise ratio (SNR). Methods With the optimization of important parameters that influence the performance of fixed-point algorithm ANC, the performance was compared to that of floating-point algorithm ANC which was isolated from the effect of quantization error. Results In the simulation study, the outputs of fixed-point-based ANC showed a little higher distortion from real SEP signals than that of floating-point algorithm ANC. In the optimal selection of μ value, fixed-point algorithm ANC could get as good results as floating-point algorithm. Conclusion With appropriate parameter values, fixed-point algorithm ANC is able to improve SNR of SEP as well as that of fixed-point algorithm ANC. Key words: Somsatosensory evoked potential; Fixed-point algorithm; Adaptive noise canceller; Step-size parameter, Distortion index

An alternating extragradient method for total variation-based image restoration from Poisson data

Variational models are a valid tool for edge-preserving image restoration from data affected by Poisson noise. This paper deals with total variation and hypersurface regularization in combination with the Kullbach Leibler divergence as a data fidelity function. We propose an iterative method, based on an alternating extragradient scheme, which is able to solve, in a numerically stable way, the primal–dual formulation of both total variation and hypersurface regularization problems. In this method, tailored for general smooth saddle-point problems, the stepsize parameter can be adaptively computed so that the convergence of the scheme is proved under mild assumptions. In the numerical experience, we focus the attention on the artificial smoothing parameter that makes different the total variation and hypersurface regularization. A set of experiments on image denoising and deblurring problems is performed in order to evaluate the influence of this smoothing parameter on the stability of the proposed method and on the features of the restored images.

Enhancement of Convergence Properties of Adaptive CDMA Receiver over Rayleigh Flat Fading Channel

CDMA system is a promising multiple access technique which extremely increases the channel capacity and supports multiple-access and multimedia transmission. However, the performance of CDMA system degrades in presence of multipath and dense multiple-access environment .Usually an adaptive filter is utilized to compensate the effect of multipath and multiple- access interference and signal fading over the channel. This paper provides a variable step size algorithm to accelerate convergence of the adaptive filter. The optimum performance is achieved by governing the rate of convergence and the amount of steady state excess mean square error (MSE).The performance is optimized in thispaper by two main methods mainly: variable step size parameters and gradient adaptively regularization parameter. The first method is implemented through variable step size least mean square algorithm VSSLMS, modified variable step size MVSS algorithm and variable step size affine projection algorithm VS-APA. On the other hand, the second method isimplemented using generalized normalized gradient descent algorithm GNGD and generalized square-error-regularized LMS algorithm GSER-LMS. This paper concerns with comparative study of the five mentioned adaptive algorithms,(VSSLMS – MVSS – VS-APA – GNGD – GSER-LMS), to overcome the multipath and multiple-access problems of CDMA systems. The mentioned algorithms had the advantages of fast convergence and low steady state MSE, these advantages candidate the mention algorithms to improve the bit error rate (BER) performance of the CDMA system, in the presence of multipath and multiple-access environment. The results indicate that the proposed algorithm provides faster convergence compared with the fixed step size and fixed regularization parameters algorithms.

Steady-State Analysis of Incremental LMS Adaptive Networks With Noisy Links

Recently proposed adaptive networks assume perfect communication among the nodes. In this correspondence, we extend existing analysis to study the performance of incremental least mean square (LMS) adaptive networks in a more realistic case in which communication links between nodes are considered noisy. More precisely, using weighted spatial-temporal energy conservation relation, we arrive a variance relation which contains moments that represent the effects of noisy links. We evaluate these moments and derive closed-form expressions for the mean-square deviation (MSD), excess mean-square error (EMSE) and mean-square error (MSE) to explain the steady-state performance at each individual node. The derived expressions have good match with simulations. However, the main result is that unlike the ideal link case, the steady-state MSD, EMSE, and MSE curves are not monotonically increasing functions of the step-size parameter when links are noisy. We illustrate this behavior and also find the optimal step-size in a closed-form (for a special case) which minimizes the steady-state values of MSD, EMSE, and MSE in each node. Simulations are also provided to clarify the derived theoretical results.

오차 제곱 가중치기반 랩 계수 갱신을 적용한 동시 등화기

동시 등화는 통신 채널을 블라인드 등화 시 수렵 특성 개선에 유용하다. 그러나 동시 등화는 등화기가 정상상태에 수렴한 후에도 동시 적응을 계속함으로써 성능 개선이 제한적이다. 본 논문에서는 동시 등화의 수렴 특성과 함께 정상상태 성능을 개선하기 위해, 가변 수렴상수와 가중치기반의 탭 계수 갱신을 적용하는 새로운 동시 등화 기법을 제안한다. 제안하는 동시 vsCMA+DD 등화는 가변 수렴상수 CMA(variable step-size CMA: vsCMA)와 판정의거 (decision-directed: DD) 알고리즘의 오차 신호를 사용하여 가중치를 산출하고, 이를 이용하여 두 등화기를 각각 가중 갱신한다. 제안 방법은 vsCMA에 의해 CMA의 오차 성능을 개선하고, 가중치기반의 탭 계수 갱신에 의해 수렴 속도와 정상상대 성능을 개선하였다. 모의실험을 통해 제안 방식의 성능 개선을 검증하였다. For blind equalization of communication channels, concurrent equalization is useful to improve convergence characteristics. However, the concurrent equalization will result in limited performance enhancement by continuing concurrent adaptation with two algorithms after the equalizer converges to steady-state. In this paper, to improve the convergence characteristics and steady-state performance of the concurrent equalization, proposed is a new concurrent equalization technique with variable step-size parameter and weight-based tap coefficients update. The proposed concurrent vsCMA+DD equalization calculates weight factors using error signals of the variable step-size CMA (vsCMA) and DD (decision-directed) algorithm, and then updates the two equalizers based on the weights respectively. The proposed method, first, improves the error performance of the CMA by the vsCMA, and enhances the steady-state performance as well as the convergence speed further by the weight-based tap coefficients update. The performance improvement by the proposed scheme is verified through simulations.

Unsupervised kernel least mean square algorithm for solving ordinary differential equations

In this paper a novel method is introduced based on the use of an unsupervised version of kernel least mean square (KLMS) algorithm for solving ordinary differential equations (ODEs). The algorithm is unsupervised because here no desired signal needs to be determined by user and the output of the model is generated by iterating the algorithm progressively. However, there are several new approaches in literature to solve ODEs but the new approach has more advantages such as simple implementation, fast convergence and also little error. Furthermore, it is also a KLMS with obvious characteristics. In this paper the ability of KLMS is used to estimate the answer of ODE. First a trial solution of ODE is written as a sum of two parts, the first part satisfies the initial condition and the second part is trained using the KLMS algorithm so as the trial solution solves the ODE. The accuracy of the method is illustrated by solving several problems. Also the sensitivity of the convergence is analyzed by changing the step size parameters and kernel functions. Finally, the proposed method is compared with neuro-fuzzy [21] approach.

What are the analytical conditions for which a blind equalizer will loose the convergence state?

In this paper, I propose for the noiseless, real and two independent quadrature carrier case some approximated conditions on the step-size parameter, on the equalizer’s tap length and on the channel power, related to the nature of the chosen equalizer and input signal statistics, for which a blind equalizer will not converge anymore. These conditions are valid for type of blind equalizers where the error that is fed into the adaptive mechanism that updates the equalizer’s taps can be expressed as a polynomial function of the equalized output of order three like in Godard’s algorithm. Since the channel power is measurable or can be calculated if the channel coefficients are given, there is no need anymore to carry out any simulation with various step-size parameters and equalizer’s tap length for a given equalization method and input signal statistics in order to find the maximum step-size parameter for which the equalizer still converges.

A novel expression for the achievable MSE performance obtained by blind adaptive equalizers

In this paper, I propose for the noisy, real, and two independent quadrature carrier case, an approximated closed-form expression for the achievable minimum mean square error (MSE) performance obtained by blind equalizers where the error that is fed into the adaptive mechanism which updates the equalizer’s taps can be expressed as a polynomial function of the equalized output of order three like in Godard’s algorithm. The proposed closed-form expression for the achievable MSE is based on the step-size parameter, on the equalizer’s tap length, on the channel power, on the signal to noise ratio (SNR), on the nature of the chosen equalizer, and on the input signal statistics. Since the channel power is measurable or can be calculated if the channel coefficients are given, there is no need anymore to carry out any simulation with various step-size parameters, different values for the signal to noise ratio (SNR) and equalizer’s tap length for a given equalization method, and input signal statistics in order to find the MSE performance in the convergence state.

Variable step-size based method for acoustic feedback modeling and neutralization in active noise control systems

This paper investigates the issue of the acoustic feedback during online operation of active noise control (ANC) systems. In the existing approach, two FIR filters are used for this task: adaptive for feedback path modeling (FBPM) and fixed for feedback neutralization (FBN). Previously, a simplified method is proposed where these two tasks of modeling and neutralization have been combined into one feedback path modeling and neutralization (FBPMN) adaptive filter. Here we introduce an intuition based variable step size (VSS) parameter, for LMS equation of FBPMN filter. This VSS is motivated from the fact that the error signal of FBPMN filter contains a disturbance-component that is decreasing in nature. The computer simulations are carried out for single-channel and multichannel ANC systems. It is demonstrated that the proposed method achieves better performance than the existing methods.