Multitask Diffusion Research Articles

Multi-Task Diffusion Learning for Time Series Classification

Current deep learning models for time series often face challenges with generalizability in scenarios characterized by limited samples or inadequately labeled data. By tapping into the robust generative capabilities of diffusion models, which have shown success in computer vision and natural language processing, we see potential for improving the adaptability of deep learning models. However, the specific application of diffusion models in generating samples for time series classification tasks remains underexplored. To bridge this gap, we introduce the MDGPS model, which incorporates multi-task diffusion learning and gradient-free patch search (MDGPS). Our methodology aims to bolster the generalizability of time series classification models confronted with restricted labeled samples. The multi-task diffusion learning module integrates frequency-domain classification with random masked patches diffusion learning, leveraging frequency-domain feature representations and patch observation distributions to improve the discriminative properties of generated samples. Furthermore, a gradient-free patch search module, utilizing the particle swarm optimization algorithm, refines time series for specific samples through a pre-trained multi-task diffusion model. This process aims to reduce classification errors caused by random patch masking. The experimental results on four time series datasets show that the proposed MDGPS model consistently surpasses other methods, achieving the highest classification accuracy and F1-score across all datasets: 95.81%, 87.64%, 82.31%, and 100% in accuracy; and 95.21%, 82.32%, 78.57%, and 100% in F1-Score for Epilepsy, FD-B, Gesture, and EMG, respectively. In addition, evaluations in a reinforcement learning scenario confirm MDGPS’s superior performance. Ablation and visualization experiments further validate the effectiveness of its individual components.

M-estimate based diffusion active noise control algorithm over distributed networks and its performance analysis

Multi-channel active noise control (ANC) systems can achieve effective attenuation of low-frequency noise in spatial areas and have been extensively studied. Recent works have focused on the application of multitask diffusion strategies in ANC systems based on wireless acoustic sensor and actuator networks (WASAN), which show the advantages of saving computational burden and enhancing flexibility. However, these works all adopt the mean-square error minimization as the optimization criterion, which leads to severe performance degradation in the presence of impulsive noise interference. Therefore, in this paper, we exploit the M-estimate function and the augmented weight vector of the control filter to formulate a robust minimization problem and propose a distributed augmented diffusion filtered-x least mean M-estimate (DADFxLMM) algorithm for multi-channel ANC systems. The proposed algorithm leverages the M-estimate function to overcome the influence of outliers in the error signal and achieves robustness against impulsive noise, and also eliminates the dependence on the symmetry of the acoustic paths by integrating the augmented weight vectors. Moreover, we replace the score function with the update probability of the weight vector to avoid the need for integration and Price’s theorem, and analyze the mean and mean-square performance of the proposed DADFxLMM algorithm under the contaminated Gaussian (CG) noise model. Simulation results under different system configurations demonstrate that the proposed algorithm outperforms the existing multi-channel ANC algorithms in the presence of impulsive noise interference.

Multi-Task Diffusion Incentive Design for Mobile Crowdsourcing in Social Networks

Multi-Task Diffusion Incentive Design for Mobile Crowdsourcing in Social Networks

An augmented diffusion algorithm with bidirectional communication for a distributed active noise control system.

Recent studies on diffusion adaptation for distributed active noise control (DANC) systems have attracted significant research interest due to their balance between computational burden and stability compared to conventional centralized and decentralized adaptation schemes. The conventional multitask diffusion FxLMS algorithm assumes that the converged solutions of all control filters are consistent to each other, which is unrealistic in practice hence results in inferior performance in noise reduction. An augmented diffusion FxLMS algorithm has been proposed to overcome this problem, which adopts a neighborhood-wide adaptation and node-based combination approach to mitigate the bias in the converged solution of the multitask diffusion algorithms. However, the improvement comes at the expense of a higher computational burden and communication cost. All existing DANC systems, including the multitask and augmented diffusion algorithms, assume one-way communication between nodes. By contrast, this paper proposes a bidirectional communication scheme for the augmented diffusion algorithm to further reduce the memory requirement, computational burden, and communication cost. Simulation results in the free field and with measured room impulse responses both demonstrate that the proposed augmented diffusion algorithm with bidirectional communication can achieve a faster convergence speed than that based on one-way communication with a lower memory, computation, and communication burden.

Robust multitask diffusion normalized M-estimate subband adaptive filter algorithm over adaptive networks

In recent years, the multitask diffusion least mean square (MD-LMS) algorithm has been extensively applied in distributed parameter estimation and target tracking of multitask network. However, its performance degrades under colored input signals or impulsive noises. To overcome these two drawbacks, this paper introduces the subband adaptive filter (SAF) into a multitask network for the first time, and a robust multitask diffusion normalized M-estimate subband adaptive filtering (MD-NMSAF) algorithm is proposed, which solves the global network optimization problem based on the modified Huber (MH) function in a distributed manner, achieves robustness to impulsive noise, and significantly improves the convergence performance of the MD-LMS algorithm. Compared with existing robust multitask diffusion affine projection algorithms (MD-APAs), the computational complexity of the proposed MD-NMSAF algorithm is greatly reduced. In addition, we analyze the mean and mean-square stability conditions of MD-NMSAF, provide theoretical models characterizing the network mean square deviation (MSD) behaviors of transient and steady-state, and further verify their correctness by computer simulations. Simulation results under different network topologies, input signals, filter lengths and impulsive noise types fully demonstrate the performance advantages of the proposed MD-NMSAF algorithm over its competitors in terms of steady-state estimation accuracy and convergence speed.

Distributed Active Noise Control Based on an Augmented Diffusion FxLMS Algorithm

Multichannel active noise control (ANC) systems have been widely investigated for low-frequency noise attenuation over a spatial region. Using a conventional centralized control strategy based on the multichannel filtered- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</i> least mean square (F <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</i> LMS) algorithm has been demonstrated to be effective for multichannel ANC systems, but the high computational burden restricts its practical applications. Meanwhile, a decentralized control strategy suffers from stability problems although it has been successful in reducing the computational load. Recently, distributed control strategies, such as the multitask diffusion adaptation scheme, have been introduced to ANC systems and shown to mitigate the stability problems in decentralized systems. However, distributed ANC systems using the diffusion F <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</i> LMS algorithm require strong symmetry of acoustic paths because of the dependence on node-based adaptation and neighborhood-based combination. To overcome this limitation, this paper proposes an Augmented Diffusion F <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</i> LMS algorithm with neighborhood-based adaptation and node-based combination. A theoretical formulation and convergence analysis are presented and simulations are performed to compare the proposed algorithm with existing ones under different system configurations for tonal, multi-tonal, narrowband and broadband signals. Simulation results demonstrate that the proposed algorithm has the same noise reduction performance as centralized method even if the acoustic paths are strongly asymmetrical, which is superior over existing distributed multitask diffusion strategy.

Robust Multi-Task Diffusion Least Mean M-Estimate Adaptive Algorithm and Its Performance Analysis

Multi-task diffusion LMS (MDLMS) is an useful algorithm to infer multiple related parameter vectors concurrently through the cooperation of nodes among clusters. However, it must be pointed out that its optimal estimation can only be realized under Gaussian noise. In practical applications, this ideal noise environment is not easy to meet. For example, there will be some impulsive noise interference. At this time, it is urgent to propose a robust multi-task algorithm against impulsive noise. The modified Huber (MH) function is considered in the new robust multi-task algorithm (i.e., multi-task diffusion least mean M-estimate (MDLMM)) proposed in this brief since its simple structure and outstanding effect of restraining impulsive interference. After that, we analyze the mean and mean square performance of the MDLMM algorithm, and give the stability conditions (in the sense of mean and mean square) and the closed-form expressions of theoretical transient and steady-state network mean square derivation (NMSD) under some assumptions. Numerical simulations verify the accuracy of the theoretical analysis results and the superiority of MDLMM over other robust multi-task algorithms.

A model-based variable step-size strategy for proximal multitask diffusion LMS algorithm

Several practical applications, such as distributed spectrum sensing and channel identification in underwater communication networks with multiple sensors, can be modeled as a distributed network with jointly sparse structure. For such a network, the proximal multitask diffusion least mean square (LMS) algorithm has been proposed in the literature, and its performance has been studied thoroughly. Due to the trade-off between convergence speed and steady-state performance in the proximal multitask diffusion LMS algorithm, it is important but not trivial, to set the step-size parameter properly. To address this issue, a variable step-size strategy for the proximal multitask diffusion LMS algorithm is proposed in this paper. Based on the transient model of the proximal multitask diffusion LMS algorithm, and by minimizing an upper-bound of the excess mean-square error (EMSE) at each iteration on the basis of a white input assumption, we obtain a closed-form expression of the step-size parameter. Simulation results illustrate the effectiveness of the proposed strategy and highlight its performance through comparison with other existing variable step-size strategies, in the cases of white and moderately correlated inputs.

Performance of Clustered Multitask Diffusion LMS Suffering From Inter-Node Communication Delays

This brief studies a clustered multitask diffusion least mean-square strategy that accounts for communication delays in the inter- and intra-cluster information exchanges. We conduct detailed performance analysis and establish convergence criteria, both in mean and mean-square senses, to investigate the effect of inter-node communication delays on the stability and estimation performance. In particular, we derive convergence conditions and closed-form expressions for transient and steady-state mean square deviation. The concepts are verified using simulation examples, which show a precise match between theoretical and experimental steady-state MSD results.

A Secure Distributed Information Sharing Algorithm Based on Attack Detection in Multi-Task Networks

Recently there were research works on multi-task estimation. However, there exists an issue that nodes may suffer from malicious attacks when sending data in the adversarial multi-task network environment. To address this issue, we propose a secure distributed information sharing algorithm based on attack detection. In our work, via determining an adaptive threshold, a distributed detection algorithm based on the correlation between tasks and a safe multi-task diffusion least mean square (SM-DLMS) algorithm are proposed, which not only reduce communication costs but also effectively decrease the impact of attacks. We also theoretically analyze the stability performance of the proposed SM-DLMS algorithm. Meanwhile, the robustness and effectiveness of the proposed algorithm under network-attacks are illustrated through numerical simulations.

Multitask diffusion affine projection sign algorithm and its sparse variant for distributed estimation

Distributed adaptation over multitask networks has attracted particular attention due to its enhanced modeling capacity compared to that over conventional single-task networks. Most of the existing works derive their multitask adaptive algorithms using mean-square error (MSE) or least squares (LS) criterion, leading to multitask LMS- or LS-type algorithms. These algorithms, however, may suffer from deteriorated convergence rate or even divergence in impulsive noise environments. In order to address this problem, we propose a robust diffusion affine projection sign algorithm for multitask parameter estimation. The algorithm is derived by using the method of data reusing and minimizing the weighted sum of the l1-norms of some intermediate error vectors plus a similarity term subject to constraints on intermediate weight vectors at each agent. The multitask similarity relationship is characterized by the distance regularization among weight vectors. Furthermore, a variant of this algorithm, which is obtained by further regularizing the cost function by the l0-norm of the intermediate weight vector at each agent, is presented to promote convergence rate for jointly sparse parameter vectors estimation.

Improving the Performance of Multitask Diffusion APA via Controlled Inter-Cluster Cooperation

In this paper, we consider the problem of estimating multiple parameter vectors over a sensor network in a multitasking framework and under temporally-correlated input conditions. For this, an efficient clustered multitask diffusion affine projection algorithm (APA) is proposed that enjoys both intra-cluster and inter-cluster cooperation via diffusion. It is, however, shown that while collaboration in principle is a useful step to enhance the performance of a network, uncontrolled mode of inter-cluster collaboration can at times be detrimental to its convergence performance, especially near steady-state. To overcome this, a controlled form of inter-cluster collaboration is proposed by means of a control variable which helps in maintaining the collaboration in right direction. The proposed controlled multitask strategy attains improved performance in terms of both transient and steady-state mean square deviation (MSD) vis-a-vis existing algorithms, as also confirmed by simulation studies. We carry out a detailed performance analysis of the proposed algorithm, obtain stability bounds for its convergence in both mean and mean-square senses, and derive expressions for the network level MSD. Simulation results reveal that the proposed scheme performs consistently well even in the absence of cluster information.

Online Proximal Learning Over Jointly Sparse Multitask Networks With $\ell _{\infty, 1}$ Regularization

Modeling relations between local optimum parameter vectors to estimate in multitask networks has attracted much attention over the last years. This work considers a distributed optimization problem with jointly sparse structure among nodes, that is, the local solutions have the same sparse support set. Several mixed norm have been proposed to address the jointly sparse structure in the literature. Among several candidates, the (reweighted) $\ell _{\infty,1}$ -norm is element-wise separable, it is more convenient to evaluate their approximate proximal operators. Thus by introducing a (reweighted) $\ell _{\infty,1}$ -norm penalty term at each node, and using a proximal gradient method to minimize the regularized cost, we devise a proximal multitask diffusion LMS algorithm which can promote joint-sparsity. Analyses are provided to characterize the algorithm behavior in the mean and mean-square sense. Simulation results are presented to show its effectiveness, as well as the accuracy of the theoretical findings.

Multi-Task oriented data diffusion and transmission paradigm in crowdsensing based on city public traffic

As mobile smart devices become increasingly popular and are equipped with increasingly powerful sensors, they have been pervasively applied in crowdsensing as effective tools to solve large-scale sensing tasks in urban areas. Task requesters can allocate sensing tasks to mobile nodes through a crowdsensing platform, eliminating the cost of deploying and maintaining large numbers of fixed sensors. However, several kinds of crowdsensing tasks (e.g., audio and video transmission) that generate large-scale sensed data may bring high network traffic costs to participants using a 3G/4G network, which may affect their satisfaction. In this paper, we build a data diffusion and transmission paradigm in crowdsensing based on City Public Traffic System (PTS), and thoroughly discuss a paradigm for Multi-Task diffusion and transmission within budget constraints. This paradigm makes full use of the advantages of a bus in PTS to realize the rapid transmission of large-scale sensed data: predictable trajectory, wide coverage area, fast moving speed and long contact duration among passengers. Further, we also propose a new data transmission algorithm called BUI-BA that chooses mobile nodes to transfer data by maximizing the transmission utility increment. The experimental results demonstrate that BUI-BA has better overall performance than widely used Greedy and effSense, achieving a tradeoff between overall transmission utility and transmission redundancy.

A Multitask Diffusion Strategy With Optimized Inter-cluster Cooperation

We consider a multitask estimation problem where nodes in a network are divided into several connected clusters, with each cluster performing a least-mean-squares estimation of a different random parameter vector. Inspired by the adapt-then-combine diffusion strategy, we propose a multitask diffusion strategy whose mean stability can be ensured whenever individual nodes are stable in the mean, regardless of the inter-cluster cooperation weights. In addition, the proposed strategy is able to achieve an asymptotically unbiased estimation, when the parameters have same mean. We also develop an inter-cluster cooperation weights selection scheme that allows each node in the network to locally optimize its inter-cluster cooperation weights. Numerical results demonstrate that our approach leads to a lower average steady-state network mean-square deviation, compared with using weights selected by various other commonly adopted methods in the literature.

An Informed Multitask Diffusion Adaptation Approach to Study Tremor in Parkinson's Disease

In this paper, a network-based approach for studying the relation between the tremor intensity and the brain connectivity of Parkinson's patients is introduced. We propose an adaptive multitask diffusion strategy to estimate the underlying model between the gait information and the electroencephalography signals. Furthermore, the method incorporates an S-transform-based connectivity measure that performs well even on a single-trial basis. The estimated connectivity values are then combined with the combination weights of the multitask diffusion strategy to model the relation between tremor and the brain signals. The outcome is an enhanced brain connectivity measure representing its time-space relation to the tremor. The results show how the differences between the connectivity values of patients with mild and severe hand tremor are most distinguishable when using the proposed method.

Multitask Diffusion Adaptation OverAsynchronous Networks

The multitask diffusion LMS is an efficient strategy to simultaneously infer, in a collaborative manner, multiple parameter vectors. Existing works on multitask problems assume that all agents respond to data synchronously. In several applications, agents may not be able to act synchronously because networks can be subject to several sources of uncertainties such as changing topology, random link failures, or agents turning on and off for energy conservation. In this paper, we describe a model for the solution of multitask problems over asynchronous networks and carry out a detailed mean and mean-square error analysis. Results show that sufficiently small step-sizes can still ensure both stability and performance. Simulations and illustrative examples are provided to verify the theoretical findings.