Incremental Least Mean Square Research Articles

Robust Incremental Least Mean Square Algorithm With Dynamic Combiner

In distributed wireless networks, the adaptation process depends on the information being shared between various nodes. The global minimum, is therefore, likely to be affected when the information shared between the nodes gets corrupted. This could happen due to several reasons namely link failure, noisy environment and erroneous data etc. In this research, we propose a computationally efficient robust incremental least mean square (RILMS) algorithm to resolve the aforementioned issues. Essentially, a fusion step is introduced in the framework of the incremental least mean square (ILMS). Prior to adaptation at a node, the information shared by the neighbouring node is fused with the temporally preceding information of the node using an efficient combiner. An adaptive fusion strategy is proposed resulting in dynamic weight assignment for the fusion step. Closed form expression for the steady-state excess mean square error (EMSE) is derived and the performance of the proposed algorithm is evaluated for the noisy link environments and compared to the existing algorithms. Extensive experiments show the efficacy of the proposed approach compared to the contemporary methods. The proposed algorithm is found to be robust against the link failure and local node divergence problems. The improved performance of the proposed RILMS algorithm comes with a significant reduction in computational complexity compared to the convex combination based ILMS (CILMS) approach.

Coordinate-Descent Adaptation over Hamiltonian Multi-Agent Networks.

The incremental least-mean-square (ILMS) algorithm is a useful method to perform distributed adaptation and learning in Hamiltonian networks. To implement the ILMS algorithm, each node needs to receive the local estimate of the previous node on the cycle path to update its own local estimate. However, in some practical situations, perfect data exchange may not be possible among the nodes. In this paper, we develop a new version of ILMS algorithm, wherein in its adaptation step, only a random subset of the coordinates of update vector is available. We draw a comparison between the proposed coordinate-descent incremental LMS (CD-ILMS) algorithm and the ILMS algorithm in terms of convergence rate and computational complexity. Employing the energy conservation relation approach, we derive closed-form expressions to describe the learning curves in terms of excess mean-square-error (EMSE) and mean-square deviation (MSD). We show that, the CD-ILMS algorithm has the same steady-state error performance compared with the ILMS algorithm. However, the CD-ILMS algorithm has a faster convergence rate. Numerical examples are given to verify the efficiency of the CD-ILMS algorithm and the accuracy of theoretical analysis.

An Intelligent Adaptive Algorithm for Environment Parameter Estimation in Smart Cities

Least mean squares (LMS) adaptive algorithms are attractive for distributed environment parameter estimation problems in a smart city due to the benefits of cooperation, adaptation, and rapid convergence. To obtain a reliable estimate of the network-wide parameter vector, local results can be further fused by intermediate agents in a distributed incremental way. In this paper, we propose an intelligent variable step size incremental LMS (VSS-ILMS) algorithm to solve the dilemma between fast convergence rate and low mean-square deviation (MSD) in conventional incremental LMS (ILMS) algorithms. The main idea behind our proposal is that the local step-size is adaptively updated by minimizing the MSD in every iteration, where Tikhonov regularization and time-averaging estimation methods are adopted. A theoretical analysis of proposed algorithm is presented in terms of mean square performance and mean step size in a closed form. Simulation results show that VSS-ILMS algorithm outperforms the constant step size ILMS algorithm and several classical variable step-size LMS algorithms. The derived theoretical results shows good agreement with those based on simulated data. For a practical consideration, the proposed algorithm is also verified by the model of target localization in sensor networks.

Performance Analysis of Incremental LMS Over Flat Fading Channels

We study the effect of fading in the communication channels between sensor nodes on the performance of incremental least mean square (ILMS) algorithm, and derive steady state performance metrics, including the mean-square deviation (MSD), excess mean-square error (EMSE) and mean-square error (MSE). We obtain conditions for mean convergence of the ILMS algorithm and show that in the presence of fading channels, the ILMS algorithm is asymptotically biased. Furthermore, the dynamic range for the mean stability depends only on the mean channel gain, and under simplifying technical assumptions, we show that the MSD, EMSE, and MSE are non-decreasing functions of the channel gain variances, with mean-square convergence to the steady states possible only if the channel gain variances are limited. We derive sufficient conditions to ensure mean-square convergence and verify our results through simulations.

An adaptive distributed parameter estimation approach in incremental cooperative wireless sensor networks

This paper studies the distributed estimation problem of in a wireless sensor network (WSN) where the collected observations are used to estimate a deterministic network-wide parameter. We propose an adaptive distributed parameter estimation approach for WSN, named as DI-NLMS, using the incremental least-mean squares (I-LMS) technique and exploiting the spatio-temporal diversity to achieve fast convergence rate and satisfactory steady state performance. In this algorithm, every individual node shares the changes in the surrounding environment with its immediate neighbors such that the information on such changes, that affect convergence rate and steady state performance, can fully characterize the features of the entire network. We deduce the optimal variable step size for I-LMS and give the distributed step size updating strategy. A guideline on how to exploit the spatio-temporal dimensions for LMS-type implementations is outlined and an algorithm is proposed. We derive theoretically the minimal mean-square derivation (MSD) for DI-NLMS in steady state. The simulations for derived theoretical results and target localization application confirm the effectiveness and efficiency of the proposed algorithm.

An Incremental RLS for Distributed Parameter Estimation of IIR Systems Present in Computing Nodes of a Wireless Sensor Network

Two popular learning algorithms namely incremental least mean square (ILMS) and diffusion least mean square (DLMS) have already been reported in literature for distributed parameter estimation using the data collected from sensor nodes. The ILMS has long convergence problem for IIR systems. The DLMS has more communication overhead which leads to excess computational complexity. To alleviate this problem in this paper a fast incremental recursive least mean square algorithm (IRLS) is proposed for IIR systems present in the computing nodes of a wireless sensor network. Simulation results demonstrate that proposed algorithm provide faster convergence and accurate performance in two examples.

An incremental LMS network with reduced communication delay

Successful implementation of incremental adaptive networks requires that the sampling speed of the network be faster than the time between two consecutive measurements taken by nodes. This issue affects the performance of incremental based algorithm in networks with large number of nodes as well as non-stationary environments. In this paper, we introduce a modified incremental least mean-square (ILMS) algorithm that resolves this problem. In the proposed algorithm, every node updates its local estimate when the measurement data are available (temporal processing). When the estimate from the previous node is also available, an adaptive strategy is used to combine the available estimates to further improve the network learning performance (spatial cooperation). We examine the performance of the proposed algorithm in two scenarios including the network with noisy links and non-stationary environment. Simulation results show the superior performance of the proposed algorithm in comparison with the original ILMS algorithm.

A Quality-aware Incremental LMS Algorithm for Distributed Adaptive Estimation

In this paper, we consider the problem of unknown parameter estimation using a set of nodes that are deployed over an area. The recently proposed distributed adaptive estimation algorithms (also known as adaptive networks) are appealing solutions to the mentioned problem when the statistical information of the underlying process is not available or it varies over time. In this paper, our goal is to develop a new incremental least-mean square (LMS) adaptive network that considers the quality of measurements collected by the nodes. Thus, we use an adaptive combination strategy which assigns each node a step size according to its quality of measurement. The adaptive combination strategy improves the robustness of the proposed algorithm to the spatial variations of signal-to-noise ratio (SNR). The performance of our algorithm is more remarkable in inhomogeneous environments when there are some nodes with low SNRs in the network. The simulation results indicate the efficiency of the proposed algorithm.

Incorporating Observation Quality Information into the Incremental LMS Adaptive Networks

In this paper we investigate the effect of observation quality information (OQI) on the performance of a special class of adaptive networks known as distributed incremental least-mean square (DILMS) algorithm. To this aim we consider two different cases: (1) a homogeneous environment where all the nodes have the same observation noise variance (ONV) and (2) an inhomogeneous environment, where different nodes have different ONVs. In the first case we show that, for the same steady-state error, the DILMS algorithm has faster convergence rate in comparison with a non-cooperative scheme. In the second case, we first show that regardless of what ONVs are, the steady-state curves of mean-square deviation, excess mean-square error and mean square error (MSE) in each node are monotonically increasing functions of step-size parameter. Then, to use the OQI, we reformulate the parameter estimation as a constrained optimization problem with MSE criterion as the cost function and ONVs as the constraints. Using the Robbins-Monro method to solve the resultant problem, a new algorithm (which we call noise-constrained incremental LMS algorithm) is obtained which has faster convergence rate than the existing incremental LMS algorithm. Simulation results are also provided to clarify the performance of proposed algorithm.

Distributed and robust parameter estimation of IIR systems using incremental particle swarm optimization

In recent years because of substantial use of wireless sensor network the distributed estimation has attracted the attention of many researchers. Two popular learning algorithms: incremental least mean square (ILMS) and diffusion least mean square (DLMS) have been reported for distributed estimation using the data collected from sensor nodes. But these algorithms, being derivative based, have a tendency of providing local minima solution particularly for minimization of multimodal cost function. Hence for problems like distributed parameters estimation of IIR systems, alternative distributed algorithms are required to be developed. Keeping this in view the present paper proposes two population based incremental particle swarm optimization (IPSO) algorithms for estimation of parameters of noisy IIR systems. But the proposed IPSO algorithms provide poor performance when the measured data is contaminated with outliers in the training samples. To alleviate this problem the paper has proposed a robust distributed algorithm (RDIPSO) for IIR system identification task. The simulation results of benchmark IIR systems demonstrate that the proposed algorithms provide excellent identification performance in all cases even when the training samples are contaminated with outliers.

An optimum step-size assignment for incremental LMS adaptive networks based on average convergence rate constraint

This paper presents an optimum step-size assignment for incremental least-mean square adaptive networks in order to improve its robustness against the spatial variation of observation noise statistics over the network. We show that when the quality of measurement information (in terms of observation noise variances) is available, we can exploit it to adjust the step-size parameter in an adaptive network to obtain better performance. We formulate the optimum step-size assignment as a constrained optimization problem and then solve it via the Lagrange multipliers approach. The derived optimum step-size for each node requires information from other nodes, thus with some justifiable assumptions, near-optimum solutions are derived that depend only on local information. We show that the incremental LMS adaptive network with near-optimal step sizes has improved robustness and steady-state performance. Simulation results are also presented to support the theoretical results.

Performance Limits for Distributed Estimation Over LMS Adaptive Networks

In this work we analyze the mean-square performance of different strategies for distributed estimation over least-mean-squares (LMS) adaptive networks. The results highlight some useful properties for distributed adaptation in comparison to fusion-based centralized solutions. The analysis establishes that, by optimizing over the combination weights, diffusion strategies can deliver lower excess-mean-square-error than centralized solutions employing traditional block or incremental LMS strategies. We first study in some detail the situation involving combinations of two adaptive agents and then extend the results to generic N-node ad-hoc networks. In the later case, we establish that, for sufficiently small step-sizes, diffusion strategies can outperform centralized block or incremental LMS strategies by optimizing over left-stochastic combination weighting matrices. The results suggest more efficient ways for organizing and processing data at fusion centers, and present useful adaptive strategies that are able to enhance performance when implemented in a distributed manner.

Robust Distributed Learning in Wireless Sensor Network using Efficient Step Size

The distributed learning methods in wireless sensor network are giving better performance when the noise is uniformly distributed in all the sensor nodes. In general practice the noise variance at different nodes varies non uniformly. If constant step size is used for all the nodes then the learning performance will be poor. In order to improve the robustness of incremental least mean squares (ILMS) adaptive learning algorithm against the spatial variation of observation noise statistics over the network, an efficient step–size assignment is presented here. When the noise variance information of all the nodes are available, then the step size parameter in the adaptive algorithm can adjust to obtain better performance. The proposed distributed algorithm is simulated in MATLAB and the performance is evaluated in terms of mean square deviation (MSD), excess mean square error (EMSE) and mean square error (MSE). The simulation result shows the robust against spatial noise variance.

Amplify-and-forward scheme in incremental LMS adaptive network with noisy links: Minimum transmission power design

Abstract The performance of adaptive networks, considerably reduces when the communication links between nodes are subject to channel noise. One way to mitigate the effect of noisy links is to use the amplify-and-forward scheme. As the amplifying factor increases, the effect of channel noise decreases at the cost of more power consumption in nodes. In this paper we focus on design of incremental least-mean square (I-LMS) adaptive network with amplify-and-forward scheme. Our goal is to find the minimum amplifying factor at node k − 1 such that the steady-state value of mean-square deviation (MSD) at node k becomes smaller than a desired (predefined) value. We derive an expression for the desired amplifying factor at node k − 1 and provide the simulation results to clarify the derived theoretical result.

Steady-State Analysis of Incremental LMS Adaptive Networks With Noisy Links

Recently proposed adaptive networks assume perfect communication among the nodes. In this correspondence, we extend existing analysis to study the performance of incremental least mean square (LMS) adaptive networks in a more realistic case in which communication links between nodes are considered noisy. More precisely, using weighted spatial-temporal energy conservation relation, we arrive a variance relation which contains moments that represent the effects of noisy links. We evaluate these moments and derive closed-form expressions for the mean-square deviation (MSD), excess mean-square error (EMSE) and mean-square error (MSE) to explain the steady-state performance at each individual node. The derived expressions have good match with simulations. However, the main result is that unlike the ideal link case, the steady-state MSD, EMSE, and MSE curves are not monotonically increasing functions of the step-size parameter when links are noisy. We illustrate this behavior and also find the optimal step-size in a closed-form (for a special case) which minimizes the steady-state values of MSD, EMSE, and MSE in each node. Simulations are also provided to clarify the derived theoretical results.

Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis

This paper presents an efficient adaptive combination strategy for the distributed estimation problem over diffusion networks in order to improve robustness against the spatial variation of signal and noise statistics over the network. The concept of minimum variance unbiased estimation is used to derive the proposed adaptive combiner in a systematic way. The mean, mean-square, and steady-state performance analyses of the diffusion least-mean squares (LMS) algorithms with adaptive combiners are included and the stability of convex combination rules is proved. Simulation results show (i) that the diffusion LMS algorithm with the proposed adaptive combiners outperforms those with existing static combiners and the incremental LMS algorithm, and (ii) that the theoretical analysis provides a good approximation of practical performance.