Diffusion LMS Algorithm Research Articles

Secure multiple adaptive kernel diffusion LMS algorithm for distributed estimation over sensor networks

Secure multiple adaptive kernel diffusion LMS algorithm for distributed estimation over sensor networks

Diffusion LMS algorithm in the presence of second order nonlinearities with theoretical bounds

In this paper, we propose an algorithm for the estimation and compensation of second-order nonlinearity in Wireless Sensor Networks (WSNs) within a distributed estimation framework. First, we investigate the impact of second-order nonlinearity on the performance of the Diffusion Least Mean Square (DLMS) algorithm and derive an upper bound for the l2-norm of the error caused by nonlinearity. Next, we suggest a distributed algorithm incorporating additional nonlinearity estimation and compensation units. Furthermore, considering second-order nonlinearity, we calculate the Cramer-Rao Bound for estimating both the unknown vector and the nonlinearity coefficient vector, with the Fisher information matrix derived in a closed-form formula. Simulation results demonstrate the effectiveness of the proposed algorithm in enhancing the performance of distributed estimation in the presence of nonlinear sensors in a WSN.

Differential privacy for diffusion LMS algorithm over a distributed network

This article presents a way to improve the Diffusion LMS algorithm in practical scenarios while maintaining data privacy. It focuses on reducing external errors and ensuring accurate estimation of the parameter vector through distributed and adaptive privacy protection. By combining these improvements, the overall performance of the algorithm can be significantly enhanced without compromising Mean Square Deviation (MSD). One important aspect is tuning the privacy noise power based on data sensitivity. This work provides a novel solution for improving the performance and privacy of the Diffusion LMS algorithm with implications for future research. The mean square error and evolution behavior of the algorithms are also analyzed, showing their effectiveness and robustness in preserving privacy and MSD over the network.

Analysis of a Diffusion LMS Algorithm with Probing Delays for Cyclostationary White Gaussian and Non-Gaussian Inputs

Analysis of a Diffusion LMS Algorithm with Probing Delays for Cyclostationary White Gaussian and Non-Gaussian Inputs

A Low Complexity Proportionate Generlized Correntropy-Based Diffusion LMS Algorithm With Closed-Form Gain Coefficients

This brief proposes a correntropy-based proportionate diffusion algorithm with low computational complexity. The proportionate diffusion algorithms in the literature suffer from a high computational complexity due to the complicated calculation process of the coefficients of the gain matrix, i.e., by solving an optimization problem or solving a linear system of equations via a matrix inversion. In this brief, by defining a proper correntropy kernel function and suitably assuming a wise constraint, the gain matrix coefficients are derived in a closed-form formula. In fact, the correntropy kernel function is used to determine the gain coefficients. This closed-form formula significantly reduces the computational complexity of the proposed algorithm. Moreover, a condition of uniform convergence for the distributed estimation algorithm at each time instant is obtained analytically. Simulation results show the lower steady-state Normalized Mean Square Deviation (NMSD) and lower computational complexity of the proposed algorithm compared to some other competing algorithms.

Analysis of a Diffusion LMS Algorithm With Communication Delays for Cyclostationary White Gaussian Inputs

Available analyses of the diffusion LMS (DLMS) algorithm assume that the nodes probe the unknown system with zero delay. This assumption is unrealistic, since the unknown system is usually distant from the nodes. The present paper studies the behavior of the algorithm without this assumption. The analysis is done for a network having a central combiner. This structure reduces the dimensionality of the resulting stochastic models while preserving important diffusion properties. Communication delays between the nodes and the central combiner are also considered in the analysis. The analysis is done for system identification for cyclostationary white Gaussian nodal inputs. Mean and mean-square behaviors of the algorithm are analyzed. It is found that delays in probing the unknown system yield a bias in the algorithm without increasing its convergence time. The communication delays between the nodes and the central combiner increase the convergence time without affecting the steady-state behavior. The stability of the algorithm is not affected by either type of delay. The analysis exactly matches the simulations.

Stochastic analysis of the diffusion LMS algorithm for cyclostationary white Gaussian inputs

This paper studies the stochastic behavior of a specific version of the Diffusion Least-Mean Square (DLMS) algorithm in a system identification framework for a cyclostationary white Gaussian input. The considered DLMS version has a fusion center. The input cyclostationary signal is modeled by a white random process with periodically time-varying power. The system parameters vary according to a random walk model. The paper focusses on the behavior of the fusion center for DLMS for the special case when the nodes communicate only with the fusion center and vice versa. Mathematical models are derived for the mean and mean-square-deviation (MSD) behavior of the fusion center adaptive weights as a function of the input cyclostationarity. It is shown that the behavior of the fusion center is the same for both Combine-Then-Adapt (CTA) and Adapt-Then-Combine (ATC) diffusion strategies. Monte Carlo simulations are shown in excellent agreement with the theory. Finally the model is used to study the design of the DLMS algorithm for different nodal step-sizes, cyclostationarities, noise powers and weighting coefficients.

A Flexible Approach to Interference Cancellation in Distributed Sensor Networks

In a distributed algorithm several processing nodes communicate with each other and incorporate in order to reach a common goal. Each unit has its own locally-observed environment while it can either help or mislead other units when sharing its information. This could be either intentional or simply due to the observation noise. Previous works mainly focus on locating the agents properly and assigning weights based on stationary environment in order to minimize the impact of noisy nodes. We, however, propose a method capable of assigning combination weights in accordance with momentary performance of the neighboring nodes. In order to demonstrate this capability, the proposed algorithm has been applied to well-known diffusion LMS algorithms. Unlike uniform weighting policy, the proposed method is robust to changes caused by malfunctioning nodes.

Adaptive Parameter Estimation of IIR System-Based WSN Using Multihop Diffusion in Distributed Approach

Distributed estimation of parameters in wireless sensor networks is taken into consideration to reduce the communication overhead of the network which makes the sensor system energy efficient. Most of the distributed approaches in literature, the sensor system is modeled with finite impulse response as it is inherently stable. Whereas in real time applications of WSN like target tracking, fast rerouting requires, infinite impulse response system (IIR) is used to model and that has been chosen in this work. It is assumed that every sensor node is equipped with IIR adaptive system. The diffusion least mean square (DLMS) algorithm is used to estimate the parameters of the IIR system where each node in the network cooperates themselves. In a sparse WSN, the performance of a DLMS algorithm reduces as the degree of the node decreases. In order to increase the estimation accuracy with a smaller number of iterations, the sensor node needs to share their information with more neighbors. This is feasible by communicating each node with multi-hop nodes instead of one-hop only. Therefore the parameters of an IIR system is estimated in distributed sparse sensor network using multihop diffusion LMS algorithm. The simulation results exhibit superior performance of the multihop diffusion LMS over non-cooperative and conventional diffusion algorithms.

Robust Minimum Disturbance Diffusion LMS for Distributed Estimation

This brief proposes a robust distributed estimation algorithm in presence of impulsive noise. Impulsive noises are present both in the measurements and in the communication links in a sensor network. The proposed method is essentially a diffusion LMS algorithm with optimized variable coefficients in the adaptation and combination steps. The optimized coefficients are obtained based on the minimum disturbance principle. Moreover, it is shown that the optimized coefficients of the adaptation step are found by solving a linear system of equations, while the optimized coefficients of the combination step are calculated by an eigenvector of a particular matrix. Moreover, the minimum disturbances calculated theoretically and their upper bounds are derived mathematically. Simulation results show the better performance of the proposed minimum disturbance diffusion LMS algorithm over some state-of-the-art algorithms.

Correction-based diffusion LMS algorithms for secure distributed estimation under attacks

In this paper, we mainly study the distributed estimation problem under attacks, which is mainly used to estimate an unknown parameter. To solve this problem, a correction-based secure diffusion least mean square (CS-dLMS) algorithm, which is a hybrid algorithm that composes of a non-cooperative LMS (nc-LMS) algorithm and a correction-based diffusion least-mean squares (C-dLMS) algorithm, is proposed for distributed estimation. The nc-LMS algorithm is used to provide a reliable reference system, which can detect reliable neighbor nodes by setting a threshold under network attacks. The correction-based least mean square algorithm can estimate an unknown parameter by interacting with neighbor nodes. In order to guarantee the mean performance of the CS-dLMS algorithm under attacks, a sufficient condition is proposed. Finally, simulation results are provided to verify the effectiveness of the proposed algorithm and it outperforms the C-dLMS algorithm and nc-LMS algorithm.

Diffusion fused sparse LMS algorithm over networks

The problem of distributed estimation for sparse parameter has attracted much attention in recent years due to its wide applications. In some cases, the sparsity feature may also be found in the differences of successive components of the true parameter, i.e. the non-zero components may be assembled in one or more regions of the true parameter. In this paper, we propose the fused sparse diffusion LMS algorithm for recovering such parameter. The proposed algorithm relies on a sparse regularization term to enforce sparsity of components themselves, and fused sparse regularization term to enforce similarity of adjacent non-zero components. We then provide the condition of the proposed algorithm converging in the mean sense and study mean-square behaviors, which include mean square deviation (MSD) and individual weight error variance (IWEV). To further enhance estimation performance of the fused sparse diffusion algorithm, we provide an adaptive strategy in which the optimal regularization factors can be adjusted at each iteration. Finally, numerical simulations are conducted to show the superiority of our proposed algorithm over several other algorithms.

Online Distributed Learning Over Graphs With Multitask Graph-Filter Models

In this article, we are interested in adaptive and distributed estimation of graph filters from streaming data. We formulate this problem as a consensus estimation problem over graphs, which can be addressed with diffusion LMS strategies. Most popular graph-shift operators such as those based on the graph Laplacian matrix, or the adjacency matrix, are not energy preserving. This may result in an ill-conditioned estimation problem, and reduce the convergence speed of the distributed algorithms. To address this issue and improve the transient performance, we introduce a preconditioned graph diffusion LMS algorithm. We also propose a computationally efficient version of this algorithm by approximating the Hessian matrix with local information. Performance analyses in the mean and mean-square sense are provided. Finally, we consider a more general problem where the filter coefficients to estimate may vary over the graph. To avoid a large estimation bias, we introduce an unsupervised clustering method for splitting the global estimation problem into local ones. Numerical results show the effectiveness of the proposed algorithms and validate the theoretical results.

Online Proximal Learning Over Jointly Sparse Multitask Networks With $\ell _{\infty, 1}$ Regularization

Modeling relations between local optimum parameter vectors to estimate in multitask networks has attracted much attention over the last years. This work considers a distributed optimization problem with jointly sparse structure among nodes, that is, the local solutions have the same sparse support set. Several mixed norm have been proposed to address the jointly sparse structure in the literature. Among several candidates, the (reweighted) $\ell _{\infty,1}$ -norm is element-wise separable, it is more convenient to evaluate their approximate proximal operators. Thus by introducing a (reweighted) $\ell _{\infty,1}$ -norm penalty term at each node, and using a proximal gradient method to minimize the regularized cost, we devise a proximal multitask diffusion LMS algorithm which can promote joint-sparsity. Analyses are provided to characterize the algorithm behavior in the mean and mean-square sense. Simulation results are presented to show its effectiveness, as well as the accuracy of the theoretical findings.

On Reducing the Communication Cost of the Diffusion LMS Algorithm

The rise of digital and mobile communications has recently made the world more connected and networked, resulting in an unprecedented volume of data flowing between sources, data centers, or processes. While these data may be processed in a centralized manner, it is often more suitable to consider distributed strategies such as diffusion as they are scalable and can handle large amounts of data by distributing tasks over networked agents. Although it is relatively simple to implement diffusion strategies over a cluster, it appears to be challenging to deploy them in an ad-hoc network with limited energy budget for communication. In this paper, we introduce a diffusion LMS strategy that significantly reduces communication costs without compromising the performance. Then, we analyze the proposed algorithm in the mean and mean-square sense. Next, we conduct numerical experiments to confirm the theoretical findings. Finally, we perform large scale simulations to test the algorithm efficiency in a scenario where energy is limited.

Maximum Total Correntropy Diffusion Adaptation Over Networks With Noisy Links

Distributed estimation over networks draws much attraction in recent years. In many situations, due to imperfect information communication among nodes, the performance of traditional diffusion adaptive algorithms such as the diffusion LMS (DLMS) may degrade. To deal with this problem, several modified DLMS algorithms have been proposed. However, these DLMS based algorithms still suffer from biased estimation and are not robust to impulsive link noise. In this paper, we focus on improving the performance of diffusion adaptation with noisy links from two aspects: accuracy and robustness. A new algorithm called diffusion maximum total correntropy (DMTC) is proposed. The new algorithm is theoretically unbiased in Gaussian noise, and can efficiently handle the link noises in the presence of large outliers. The adaptive combination rule is applied to further improve the performance. The stability analysis of the proposed algorithm is given. Simulation results show that the DMTC algorithm can achieve good performance in both Gaussian and non-Gaussian noise environments.

Distributed maximum a posteriori estimation under non-stationary condition

Most problems in the field of distributed estimation are studied under the stationary condition, i.e. the parameter of interest is time invariant. However, time-varying parameter may exist in many real systems, which has rarely been considered in the field of distributed estimation. In this paper, we propose a novel distributed diffusion-type algorithm, called the diffusion maximum a posterior (dMAP) algorithm, for tracking the time-varying parameter. The proposed algorithm, which is derived by using the theory of maximum a posteriori estimation, exploits the statistical information of the true parameter varying with time on different entries of the true parameter. Theoretical analyses for the mean stability and mean-square learning behavior are performed for the proposed dMAP algorithm. Finally, we have also carried out numerical simulations to validate theoretical findings and demonstrate the superiority of the proposed dMAP algorithm in tracking the time-varying parameter over several other algorithms, such as the diffusion LMS and diffusion NLMS algorithms.

Performance analysis of diffusion LMS algorithm for cyclostationary inputs

Abstract As a well-known non-stationary signal, the cyclostationary white Gaussian signal widely exists in many practical applications, which is defined as a particular Gaussian noise whose autocorrelation function is cyclically time-varying. This paper presents the performance analysis of the diffusion least mean squares (DLMS) algorithms in distributed networks while the input signals are the cyclostationary white Gaussian process. We analyze mean and mean square behaviors of the DLMS algorithm. It is found that the time-variations of steady-state mean square deviation (MSD) of DLMS algorithm can be ignored when the input signals have the fast variations of input autocorrelation function. Moreover, the robustness of the DLMS algorithm in the H∞ sense for cyclostationary inputs over networks is analyzed. The bound of convergence time of the DLMS algorithm for cyclostationary white Gaussian inputs is also provided in this paper. The simulated results show the validity of the analytical results.

Diffusion Robust Variable Step-Size LMS Algorithm Over Distributed Networks

In this paper, we propose a novel diffusion robust variable step-size least mean square (DRVSS-LMS) algorithm that is insensitive to impulsive noise for distributed estimation in the network. Conventional diffusion least mean square algorithms are based on the assumption that the background noise obeys Gaussian distribution. However, the performances of these algorithms are severely degraded when impulsive noises occur in the network. By introducing the Huber objective function which can significantly suppress the effect of impulsive noise on estimation performances, we introduce a novel method to respectively deal with the abnormal nodes carrying data contaminated by impulsive noise and the normal nodes without being contaminated by impulsive noise. In addition, the proposed algorithm is assigned with variable step-sizes to further improve the performances of distributed estimation. Simulation results show that the proposed DRVSS-LMS algorithm can achieve both higher convergence rate and lower steady-state misadjustment than several existing robust diffusion LMS algorithms in the presence of impulsive noise.

Diffusion LMS With Correlated Regressors II: Performance

Over the past decade, diffusion LMS and real-time consensus (RTC) adaptive network algorithms have been widely used and studied. However, their performance analysis has relied on white regressor assumptions, which are not satisfied in practice. Here, we consider both temporally correlated regressors and noise and give mean squared deviation (MSD) and excess mean squared error (EMSE) performance results for the first time. Furthermore, we employ a novel second-order mixed time-scale stochastic averaging approach to study realization-wise fluctuations of the parameter estimation error. Critically, our new results show that both temporally correlated regressors and noise affect the performance of the algorithms, even for small gains. Simulations illustrate the results. This is Part II of an earlier work on stability. Together, these works show that the Adapt-Then-Combine (ATC), Combine-Then-Adapt (CTA), and RTC algorithms are strikingly similar in terms of stability and performance in the slow adaptation regime. In particular, ATC, CTA, and RTC have the same network MSD and EMSE to first order. Moreover, we conclude that existing MSD and EMSE results are insufficient to first order when both the regressors and noise are temporally correlated.