Diffusion Least-mean Squares Algorithm Research Articles

Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks

This paper proposes a distributed estimation technique utilizing the diffusion least mean-square (LMS) algorithm, specifically designed for sparse systems in which many coefficients of the system are zeros. To efficiently utilize the sparse representation of the system and achieve a promising performance, we have incorporated L0-norm regularization into the diffusion LMS algorithm. This integration is accomplished by employing hard thresholding through a variable splitting method into the update equation. The efficacy of our approach is validated by comprehensive theoretical analysis, rigorously examining the mean stability as well as the transient and steady-state behaviors of the proposed algorithm. The proposed algorithm preserves the behavior of large coefficients and strongly enforces smaller coefficients toward zero through the relaxation of L0-norm regularization. Experimental results show that the proposed algorithm achieves superior convergence performance compared with conventional sparse algorithms.

Diffusion Based Channel Gains Estimation in WSN Using Fractional Order Strategies

In this study, it is proposed that the diffusion least mean square (LMS) algorithm can be improved by applying the fractional order signal processing methodologies. Application of Caputo’s fractional derivatives are considered in the optimization of cost function. It is suggested to derive a fractional order variant of the diffusion LMS algorithm. The applicability is tested for the estimation of channel parameters in a distributed environment consisting of randomly distributed sensors communicating through wireless medium. The topology of the network is selected such that a smaller number of nodes are informed. In the network, a random sleep strategy is followed to conserve the transmission power at the nodes. The proposed fractional order modified diffusion LMS algorithms are applied in the two configurations of combine-then-adapt and adapt-then-combine. The average squared error performance of the proposed algorithms along with its traditional counterparts are evaluated for the estimation of the Rayleigh channel parameters. A mathematical proof of convergence is provided showing that the addition of the nonlinear term resulting from fractional derivatives helps adjusts the autocorrelation matrix in such a way that the spread of its eigenvalues decreases. This increases the convergence as well as the steady state response even for the larger step sizes. Experimental results are shown for different number of nodes and fractional orders. The simulation results establish that the accuracy of the proposed scheme is far better than its classical counterparts, therefore, helps better solves the channel gains estimation problem in a distributed wireless environment. The algorithm has the potential to be applied in other applications related to learning and adaptation.

A model-based variable step-size strategy for proximal multitask diffusion LMS algorithm

Several practical applications, such as distributed spectrum sensing and channel identification in underwater communication networks with multiple sensors, can be modeled as a distributed network with jointly sparse structure. For such a network, the proximal multitask diffusion least mean square (LMS) algorithm has been proposed in the literature, and its performance has been studied thoroughly. Due to the trade-off between convergence speed and steady-state performance in the proximal multitask diffusion LMS algorithm, it is important but not trivial, to set the step-size parameter properly. To address this issue, a variable step-size strategy for the proximal multitask diffusion LMS algorithm is proposed in this paper. Based on the transient model of the proximal multitask diffusion LMS algorithm, and by minimizing an upper-bound of the excess mean-square error (EMSE) at each iteration on the basis of a white input assumption, we obtain a closed-form expression of the step-size parameter. Simulation results illustrate the effectiveness of the proposed strategy and highlight its performance through comparison with other existing variable step-size strategies, in the cases of white and moderately correlated inputs.

Stochastic analysis of the diffusion LMS algorithm for cyclostationary white Gaussian inputs

This paper studies the stochastic behavior of a specific version of the Diffusion Least-Mean Square (DLMS) algorithm in a system identification framework for a cyclostationary white Gaussian input. The considered DLMS version has a fusion center. The input cyclostationary signal is modeled by a white random process with periodically time-varying power. The system parameters vary according to a random walk model. The paper focusses on the behavior of the fusion center for DLMS for the special case when the nodes communicate only with the fusion center and vice versa. Mathematical models are derived for the mean and mean-square-deviation (MSD) behavior of the fusion center adaptive weights as a function of the input cyclostationarity. It is shown that the behavior of the fusion center is the same for both Combine-Then-Adapt (CTA) and Adapt-Then-Combine (ATC) diffusion strategies. Monte Carlo simulations are shown in excellent agreement with the theory. Finally the model is used to study the design of the DLMS algorithm for different nodal step-sizes, cyclostationarities, noise powers and weighting coefficients.

Distributed Adaptive Multi-Task Learning Based on Partially Observed Graph Signals

In this paper, we consider the clustered multi-task learning (MTL) problem with partial observations and develop a diffusion least-mean-square (LMS) algorithm with a distributed cluster-wise sampling strategy. The proposed algorithm converges at the steady-state with the measurements observed only at a subset of the vertices, instead of the entire graph, without significant loss of the steady-state performance. We analyze the performance of the proposed algorithm and further devise a tractable cost function with respect to the sampling probability based on an approximate network Mean-Square-Deviation (MSD) of the learning objectives. We further develop a feasible selection scheme of the sampling probability set to bolster the distributed cluster-wise sampling strategy such that the convergence of the proposed diffusion LMS algorithm is accelerated. Illustrative simulations show the efficiency and the robustness of the proposed algorithm and validate the theoretical results.

Adaptive Parameter Estimation of IIR System-Based WSN Using Multihop Diffusion in Distributed Approach

Distributed estimation of parameters in wireless sensor networks is taken into consideration to reduce the communication overhead of the network which makes the sensor system energy efficient. Most of the distributed approaches in literature, the sensor system is modeled with finite impulse response as it is inherently stable. Whereas in real time applications of WSN like target tracking, fast rerouting requires, infinite impulse response system (IIR) is used to model and that has been chosen in this work. It is assumed that every sensor node is equipped with IIR adaptive system. The diffusion least mean square (DLMS) algorithm is used to estimate the parameters of the IIR system where each node in the network cooperates themselves. In a sparse WSN, the performance of a DLMS algorithm reduces as the degree of the node decreases. In order to increase the estimation accuracy with a smaller number of iterations, the sensor node needs to share their information with more neighbors. This is feasible by communicating each node with multi-hop nodes instead of one-hop only. Therefore the parameters of an IIR system is estimated in distributed sparse sensor network using multihop diffusion LMS algorithm. The simulation results exhibit superior performance of the multihop diffusion LMS over non-cooperative and conventional diffusion algorithms.

A variable step-size diffusion LMS algorithm with a quotient form

A new variable step-size strategy for the least mean square (LMS) algorithm is presented for distributed estimation in adaptive networks using the diffusion scheme. This approach utilizes the ratio of filtered and windowed versions of the squared instantaneous error for iteratively updating the step-size. The result is that the dependence of the update on the power of the error is reduced. The performance of the algorithm improves even though it is at the cost of added computational complexity. However, the increase in computational complexity can be minimized by careful manipulation of the update equation, resulting in an excellent performance-complexity trade-off. Complete theoretical analysis is presented for the proposed algorithm including stability, transient and steady-state analyses. Extensive experimental analysis is then done to show the performance of the proposed algorithm under various scenarios.

Diffusion LMS Based on Message Passing Algorithm

Diffusion least-mean-square (LMS) is an adaptive algorithm that estimates an unknown global vector from its linear measurements obtained at all nodes in a distributed manner when each node in the network needs to track the unknown vector in real-time. The algorithm uses the conventional average consensus protocol in order to combine neighbors' estimates at each node, while another protocol, consensus propagation (CP), is known to achieve faster and exact average consensus when the network has a tree structure. This paper proposes a novel diffusion LMS algorithm using CP, which can be applied for any network by extracting a spanning tree from the original network and can achieve the same solution as the centralized LMS in a fully distributed manner. This paper also proposes an algorithm by using the idea of loopy CP, so that it can be directly applied even when the network is not a tree and shows that its special case results in the diffusion LMS using a novel combination rule. Moreover, we optimize the constants involved in the proposed combination rule in terms of the steady-state mean-square-deviation of the diffusion LMS and show an adaptive implementation of the proposed algorithm. The simulation results demonstrate that the proposed algorithm using CP is beneficial for large-scale networks, and the diffusion LMS with the proposed combination rule achieves better convergence performance than that with the conventional combination rules when the measurement noise power depends on nodes.

Distributed sparse diffusion estimation with reduced communication cost

The issue considered in the current study is the problem of adaptive distributed estimation based on diffusion strategy which can exploit sparsity in improving estimation error and reducing communications. It has been shown that distributed estimation leads to a good performance in terms of the error value, convergence rate, and robustness against node and link failures in wireless sensor networks. However, the main focus of many works in the field of distributed estimation research is on convergence speed and estimation error, neglecting the fact that communications among the nodes require a lot of transmissions. In this work, the focus is on a solution based on sparse diffusion least mean squares (LMS) algorithm, and a new version of sparse diffusion LMS algorithm is proposed which takes both communications and error cost into account. Also, the computation complexity and communication cost for every node of the network, as well as performance analysis of the proposed strategy, is provided. The performance of the proposed method in comparison with the existing methods is illustrated by means of simulations in terms of computational and communicational cost, and flexibility to signal changes.

Performance analysis of diffusion LMS algorithm for cyclostationary inputs

Abstract As a well-known non-stationary signal, the cyclostationary white Gaussian signal widely exists in many practical applications, which is defined as a particular Gaussian noise whose autocorrelation function is cyclically time-varying. This paper presents the performance analysis of the diffusion least mean squares (DLMS) algorithms in distributed networks while the input signals are the cyclostationary white Gaussian process. We analyze mean and mean square behaviors of the DLMS algorithm. It is found that the time-variations of steady-state mean square deviation (MSD) of DLMS algorithm can be ignored when the input signals have the fast variations of input autocorrelation function. Moreover, the robustness of the DLMS algorithm in the H∞ sense for cyclostationary inputs over networks is analyzed. The bound of convergence time of the DLMS algorithm for cyclostationary white Gaussian inputs is also provided in this paper. The simulated results show the validity of the analytical results.

A variable step-size diffusion LMS algorithm over networks with noisy links

The variable step-size (VSS) diffusion least mean square (LMS) algorithms for wireless sensor networks (WSNs) exhibit accelerated initial convergence rate and decreased steady-state deviation compared with the conventional constant step-size counterparts. However, the performance of the VSS diffusion LMS algorithms would be deteriorated if the network links are contaminated by noise. By explicitly considering the link noise impact, we develop a locally sub-optimal step-size scheme for the diffusion LMS algorithm, originated from which we propose an asymptotically locally sub-optimal (ALSO) step-size scheme as well as a VSS diffusion LMS algorithm. The proposed VSS diffusion LMS algorithm is verified through simulations to significantly outperform the current VSS diffusion algorithm in the networks with moderate and high link noise profiles. Simulation results further validate that the proposed VSS diffusion LMS algorithm is superior to the current VSS algorithm in the networks with correlated link noise.

Diffusion Leaky Zero Attracting Least Mean Square Algorithm and Its Performance Analysis

Recently, the leaky diffusion least-mean-square (DLMS) algorithm has obtained much attention because of its good performance for high input eigenvalue spread and low signal-to-noise ratio (SNR). However, the leaky DLMS algorithm may suffer from performance deterioration in the sparse system. To overcome this drawback, the leaky zero attracting DLMS (LZA-DLMS) algorithm is developed in this paper, which adds an l1-norm penalty to the cost function to exploit the property of sparse system. The leaky reweighted zero attracting DLMS (LRZA-DLMS) algorithm is also put forward, which can improve the estimation performance in the presence of time-varying sparsity. Instead of using the l1-norm penalty, in the reweighted version, a log-sum function is employed as the substitution. Based on the weight error variance relation and several common assumptions, we analyze the transient behavior of our findings and determine the stability bound of the step-size. Moreover, we implement the steady state theoretical analysis for the proposed algorithms. Simulations in the context of distributed network system identification illustrate that the proposed schemes outperform various existing algorithms and validate the accuracy of the theoretical results.

Data-Reserved Periodic Diffusion LMS With Low Communication Cost Over Networks

In this paper, we analyze diffusion strategies in which all nodes attempt to estimate a common vector parameter for achieving distributed estimation in adaptive networks. Under diffusion strategies, each node essentially needs to share processed data with predefined neighbors. Although the use of internode communication has contributed significantly to improving convergence performance based on diffusion, such communications consume a huge quantity of power in data transmission. In developing low-power consumption diffusion strategies, it is very important to reduce the communication cost without significant degradation of convergence performance. For that purpose, we propose a data-reserved periodic diffusion least-mean-squares (LMS) algorithm in which each node updates and transmits an estimate periodically while reserving its measurement data even during non-update time. By applying these reserved data in an adaptation step at update time, the proposed algorithm mitigates the decline in convergence speed incurred by most conventional periodic schemes. For a period $p$ , the total cost of communication is reduced to a factor of $1/p$ relative to the conventional adapt-then-combine (ATC) diffusion LMS algorithm. The loss of combination steps in this process leads naturally to a slight increase in the steady-state error as the period $p$ increases, as is theoretically confirmed through mathematical analysis. We also prove an interesting property of the proposed algorithm, namely, that it suffers less degradation of the steady-state error than the conventional diffusion in a noisy communication environment. Experimental results show that the proposed algorithm outperforms related conventional algorithms and, in particular, outperforms ATC diffusion LMS over a network with noisy links.

Performance analysis of diffusion least mean fourth algorithm over network

The diffusion least mean fourth (DLMF) algorithm was initially presented for distributed estimate over network with the sub-Gaussian node noise, which can obtain better performance than diffusion least mean square (DLMS) algorithm. Unfortunately, due to the higher order moment of the weight error in the DLMF algorithm and complexity of the network, it is difficult to analyze the mean square performance of the DLMF algorithm. In this paper, the stochastic behaviors including mean behavior and mean square behavior are investigated when the input signals of all nodes are Gaussian processes and the background noises have symmetric probability density functions. The analytical models of the mean weight error and the mean square deviation (MSD) are derived by using a novel manner, which can accurately predict the algorithm behavior. The simulations are carried out to verify the validity of the analysis and compare the performance of DLMF algorithm with that of the DLMS algorithm under different background noise distributions.

Diffusion Signed LMS Algorithms and Their Performance Analyses for Cyclostationary White Gaussian Inputs

As one of the signed variants of the diffusion least mean square (DLMS) algorithm over networks, the diffusion sign error algorithm has been presented in previous reference. In this paper, we propose two novel signed variants of the DLMS algorithm, i.e., the diffusion signed regressor algorithm and the diffusion sign-sign algorithm. Moreover, this paper analyzes the performance of these three signed variants of the DLMS algorithm for cyclostationary white Gaussian inputs which have periodically time-varying variances. It is assumed that the distributed algorithms are in non-stationary environments. Specifically, the unknown parameter to be identified is time-varying according to the standard random walk model. The analysis models in terms of mean weight behavior and mean square performance are provided, in which, we can find some interesting results. Finally, simulations are carried out to verify the correctness of the proposed analysis model.

Communication-reducing diffusion LMS algorithm over multitask networks

Many practical problems in the filed of distributed estimation happen to be multi-task oriented. Without prior knowledge of clustering structure, i.e. nodes do not know which clusters they belong to beforehand, distributed algorithms for parameter estimation have received great attention in recent years. In most previous work, each node collaborates with all its neighboring nodes at each iteration, which introduces unnecessary communication assumption when any node cooperates with neighboring nodes from different clusters. In this paper, we propose a novel communication-reducing diffusion LMS (Least-Mean-Square) algorithm, called the CR-dLMS algorithm, for estimating true parameters in multi-task environment. Under the CR-dLMS algorithm, we control the probabilities of data fusion from neighboring nodes by minimizing mean-square-deviation (MSD) to reduce communication cost among nodes. Theoretical analysis for the learning behavior of the CR-dLMS algorithm is performed, and simulation results show that the CR-dLMS algorithm can indeed achieve the same estimation performance as several other previous algorithms while great reducing the communication cost among nodes.

The Performance Analysis of Diffusion LMS Algorithm in Sensor Networks Based on Quantized Data and Random Topology

We study the performance of diffusion LMS (Least-Mean-Square) algorithm for distributed parameter estimation problem over sensor networks with quantized data and random topology, where the data are quantized before transmission and the links are interrupted at random times. To achieve unbiased estimation of the unknown parameter, we add dither (small noise) to the sensor states before quantization. We first propose a diffusion LMS algorithm with quantized data and random link failures. We further analyze the stability and convergence of the proposed algorithm and derive the closed-form expressions of the MSD (Mean-Square Deviation) and EMSE (Excess Mean-Square Errors), which characterize the steady-state performance of the proposed algorithm. We show that the convergence of the proposed algorithm is independent of quantized data and random topology. Moreover, the analytical results reveal which factors influence the network performance, and we show that the effect of quantization is the main factor in performance degradation of the proposed algorithm. We finally provide computer simulation results that illustrate the performance of the proposed algorithm and verify the results of the theoretical analysis.

Diffusion adaptive networks with imperfect communications: link failure and channel noise

The article studies the steady-state performance of a diffusion least-mean squares (LMS) adaptive network with imperfect communications where the topology is random (links may fail at random times) and the communication in the channels is corrupted by additive noise. Using the established weighted spatial–temporal energy conservation argument, the authors derive a variance relation which contains moments that represent the effects of noisy links and random topology. The authors evaluate these moments and derive closed-form expressions for the mean-square deviation, excess mean-square error and mean-square error to explain the steady-state performance at each individual node. The mean stability analysis is also provided. The derived theoretical expressions have good match with simulation results. Nevertheless, the important result is that the noisy links are the main factor in performance degradation of a diffusion LMS algorithm running in a network with imperfect communications.

Stability of Diffusion Adaptive Filters

In this paper, we consider the diffusion adaptive filters where a set of sensors is required to collectively estimate time-varying signals (or parameters) from noisy measurements in a way of information diffusion. We will establish the stability of the diffusion least mean square (DLMS) algorithm, without requiring stationarity, independency, and boundedness assumptions of the system signals, which means that our results can be applied to more general and practical class of stochastic systems than those studied in the literature. We will present theoretical results concerning stability and bounds on the mean square error(MSE) of the filtering. We will also show that the network of sensors can cooperate to guarantee the stability of the filtering, even though any single sensor does not have such a capability. This clearly reveals the advantages of the DLMS algorithm vs. standard least mean square (LMS) algorithm. Numerical simulations will also be presented to support the theoretical justifications.

A variable step-size strategy for distributed estimation over adaptive networks

AbstractA lot of work has been done recently to develop algorithms that utilize the distributed structure of anad hocwireless sensor network to estimate a certain parameter of interest. One such algorithm is called diffusion least-mean squares (DLMS). This algorithm estimates the parameter of interest using the cooperation between neighboring sensors within the network. The present work proposes an improvement on the DLMS algorithm by using a variable step-size LMS (VSSLMS) algorithm. In this work, first, the well-known variants of VSSLMS algorithms are compared with each other in order to select the most suitable algorithm which provides the best trade-off between performance and complexity. Second, the detailed convergence and steady-state analyses of the selected VSSLMS algorithm are performed. Finally, extensive simulations are carried out to test the robustness of the proposed algorithm under different scenarios. Moreover, the simulation results are found to corroborate the theoretical findings very well.