Federated Learning Research Articles

Implementing federated learning over VPN-based wireless backhaul networks for healthcare systems

Federated learning (FL) is a popular method where edge devices work together to train machine learning models. This study introduces an efficient network for analyzing healthcare records. It uses VPN technology and applies a federated learning approach over a wireless backhaul network. The study compares different wireless backhaul channels, including terahertz (THz), E/V band (mmWave), and microwave, for their effectiveness. We looked closely at a suggested FL network that uses VPN technology over awireless backhaul network. We compared it with the standard method and found that using the FedAvg algorithm with Terahertz (THz) for communication gave the best accuracy. The time it took to reach a conclusion improved a lot, going from 55 seconds to an impressive 38 seconds. This emphasizes how having a faster communication link makes FL networks work much better. Furthermore, a three-step plan was executed to boost security, adopting a multi-layered method to safeguard the FL network and its confidential data. The first step involves integrating a private network into the current telecom infrastructure, establishing an initial layer of security. To enhance security further, licensed frequency channels are introduced, providing an extra layer of protection. The highest level of security is achieved by combining a private network with licensed frequency channels, complemented by an additional layer of security through VPN-based measures. This comprehensive strategy ensures the application of strong security protocols.

Efficient Federated Recommender System with Adaptive Model Pruning and Momentum-based Batch Adjustment

With the development of 5G communication and smart devices, the prosperity of online content has boosted the research of Recommender System (RS). Due to the data scarcity problem, researchers employ knowledge transfer techniques to improve the accuracy of RS. Data sharing or data augmentation are promising methods for such a problem, but the data suffers from privacy leakage during sharing. Thus, Federated Learning (FL) has been adopted to collaboratively train recommender models while preserving data privacy. Federated Recommender System (FRS) combines FL and RS to provide distributed recommendation services to the users. However, the existing FRS suffers from massive communication and system heterogeneity, where the necessity of model transmission and the diversity of the clients bring significant communication overhead to the system. In this paper, we propose Efficient Federated Recommender System with Adaptive Model Pruning and Momentum-based Batch Adjustment ( eFRSA 2 ) to reduce the communication overhead of FRS. eFRSA 2 contains two modules. Adaptive Model Pruning utilizes magnitude pruning to reduce the communication volume and adaptively modifies the compression ratios of different clients to maintain the model accuracy. Momentum-based Batch Adjustment adjusts the local training batch number by a similar method of gradient descent with momentum to align the local computation time of the clients and reduce the communication overhead. The experimental results demonstrate that eFRSA 2 can reduce up to 90% communication volume and mitigate the system heterogeneity by over 75%, demonstrating the priority of eFRSA 2 in training efficiency. Source code can be found at https://github.com/shhjwu5/eFRSA2.

Combining federated learning and control: A survey

AbstractThis survey provides an overview of combining federated learning (FL) and control to enhance adaptability, scalability, generalization, and privacy in (nonlinear) control applications. Traditional control methods rely on controller design models, but real‐world scenarios often require online model retuning or learning. FL offers a distributed approach to model training, enabling collaborative learning across distributed devices while preserving data privacy. By keeping data localized, FL mitigates concerns regarding privacy and security while reducing network bandwidth requirements for communication. This survey summarizes the state‐of‐the‐art concepts and ideas of combining FL and control. The methodical benefits are further discussed, culminating in a detailed overview of expected applications, from dynamical system modelling over controller design, focusing on adaptive control, to knowledge transfer in multi‐agent decision‐making systems.

A Systematic Literature Review on Cyber Attack Detection in Software-Define Networking (SDN)

The increasing complexity and sophistication of cyberattacks pose significant challenges to traditional network security tools. Software-defined networking (SDN) has emerged as a promising solution because of its centralized management and adaptability. However, cyber-attack detection in SDN settings remains a vital issue. The current literature lacks comprehensive assessment of SDN cyber-attack detection methods including preparation techniques, benefits and types of attacks analysed in datasets. This gap hinders the understanding of the strengths and weaknesses of various detection approaches. This systematic literature review aims to examine SDN cyberattack detection, identify strengths, weaknesses, and gaps in existing techniques, and suggest future research directions in this critical area. A systematic approach was used to review and analyse various SDN cyberattack detection techniques from 2017--2024. A comprehensive assessment was conducted to address these research gaps and provide a comprehensive understanding of different detection methods. The study classified attacks on SDN planes, analysed detection datasets, discussed feature selection methods, evaluated approaches such as entropy, machine learning (ML), deep learning (DL), and federated learning (FL), and assessed metrics for evaluating defense mechanisms against cyberattacks. The review emphasized the importance of developing SDN-specific datasets and using advanced feature selection algorithms. It also provides valuable insights into the state-of-the-art techniques for detecting cyber-attacks in SDN and outlines a roadmap for future research in this critical area. This study identified research gaps and emphasized the importance of further exploration in specific areas to increase cybersecurity in SDN environments.

Federated Learning : A Paradigm Shift in Collaborative Machine Learning

Federated learning (FL) has emerged as an exceptionally promising method within the realm of machine learning, enabling multiple entities to jointly train a global model while maintaining decentralized data. This paper presents a comprehensive review of federated learning methodologies, applications, and challenges. We begin by elucidating the fundamental concepts underlying FL, including federated optimization algorithms, communication protocols, and privacy-preserving techniques. Subsequently, we delve into various domains where FL has found significant traction, examples include healthcare, finance, and the Internet of Things (IoT), showcasing successful deployments and innovative strategies. Furthermore, we discuss the inherent challenges associated with federated learning, such as communication overhead, heterogeneity of data sources, and privacy concerns, and explore state- of-the-art solutions proposed in literature. Finally, we outline future research directions in federated learning, including advancements in privacy-preserving techniques, scalability improvements, and extension of FL to emerging domains. This thorough examination provides a valuable asset for researchers, practitioners, and policymakers keen on grasping the panorama of federated learning and its ramifications for collaborative machine learning in dispersed settings.

Recent Advancements in Federated Learning: State of the Art, Fundamentals, Principles, IoT Applications and Future Trends

Federated learning (FL) is creating a paradigm shift in machine learning by directing the focus of model training to where the data actually exist. Instead of drawing all data into a central location, which raises concerns about privacy, costs, and delays, FL allows learning to take place directly on the device, keeping the data safe and minimizing the need for transfer. This approach is especially important in areas like healthcare, where protecting patient privacy is critical, and in industrial IoT settings, where moving large numbers of data is not practical. What makes FL even more compelling is its ability to reduce the bias that can occur when all data are centralized, leading to fairer and more inclusive machine learning outcomes. However, it is not without its challenges—particularly with regard to keeping the models secure from attacks. Nonetheless, the potential benefits are clear: FL can lower the costs associated with data storage and processing, while also helping organizations to meet strict privacy regulations like GDPR. As edge computing continues to grow, FL’s decentralized approach could play a key role in shaping how we handle data in the future, moving toward a more privacy-conscious world. This study identifies ongoing challenges in ensuring model security against adversarial attacks, pointing to the need for further research in this area.

LFL-COBC: Lightweight Federated Learning on Blockchain-Based Device Contribution Allocation

In the distributed cyber-physical systems (CPSs) within the industrial domain, the volume of data produced by interconnected devices is escalating at an unprecedented pace, presenting novel opportunities to enhance service quality through data sharing. Nevertheless, data privacy protection emerges as a significant challenge for data providers in wireless networks. This paper puts forward a solution integrating blockchain and lightweight federated learning, designated as LFL-COBC, which aims to tackle the issues related to data privacy and device performance optimization. We initially analyze multiple dimensions influencing the performance of computing devices, such as mining capacity, data quality, computational efficiency and local device deviation, which are crucial for augmenting user engagement. Based on these dimensions, we deduce a set of cooperation strategies for selecting the optimal committee members and rewarding the contributions of node devices equitably, thereby stimulating cooperation between users and servers. To intelligently and automatically detect device anomalies and alleviate the operational burden, a convolutional neural network (CNN) model is employed. Additionally, to address the escalating cost of customer participation and the potential data explosion issue, a near-optimal model pruning algorithm is designed. This algorithm can make the model obtained from the training of node equipment lightweight, thereby reducing the load of federated learning and the blockchain, as well as enhancing the overall efficiency of the system. The efficacy of our approach is demonstrated through numerical experiments on the HDFS and BGL public data sets. Experimental results indicate that the LFL-COBC scheme can effectively safeguard data privacy and optimize device performance concurrently, providing an effective solution for device anomaly detection in CPSs.

FLSAD: Defending Backdoor Attacks in Federated Learning via Self-Attention Distillation

Federated Learning (FL), as a distributed machine learning framework, can effectively learn symmetric and asymmetric patterns from large-scale participants. However, FL is susceptible to malicious backdoor attacks through attackers injecting triggers into the backdoored model, resulting in backdoor samples being misclassified as target classes. Due to the stealthy nature of backdoor attacks in FL, it is difficult for users to discover the symmetric and asymmetric backdoor properties. Currently, backdoor defense methods in FL cause model performance degradation while reducing backdoors. In addition, some methods will assume the existence of clean samples, which does not match the realistic scenarios. To address such issues, we propose FLSAD, an effective backdoor defense method in FL via self-attention distillation. FLSAD can recover the triggers using an entropy maximization estimator. Based on the recovered triggers, we leverage the self-attention distillation to eliminate the backdoor. Compared with the baseline backdoor defense methods, FLSAD can reduce the success rates of different state-of-the-art backdoor attacks to 2% on four real-world datasets through extensive evaluation.

Interpreting Federated Learning (FL) Models on Edge Devices by Enhancing Model Explainability with Computational Geometry and Advanced Database Architectures

Federated learning (FL) on edge devices has emerged as a promising approach for decentralized model training, enabling data privacy and efficiency in distributed networks. However, the complexity of these models presents significant challenges in terms of transparency and interpretability, which are critical for trust and accountability in real-world applications. This paper explores the integration of explainable AI techniques to enhance model interpretability within federated learning systems. By incorporating computational geometry, we aim to optimize model structure and decision-making processes, providing clearer insights into how models generate predictions. Additionally, we examine the role of advanced database architectures in managing the complexity of federated learning models on edge devices, ensuring efficient data handling and storage. Together, these approaches contribute to a more transparent, efficient, and scalable framework for federated learning on edge networks, addressing key challenges in both model explainability and performance optimization. This review highlights recent advancements and suggests future directions for research at the intersection of federated learning (FL), edge computing, explainability, and computational techniques.

Clipper: Online Joint Client Sampling and Power Allocation for Wireless Federated Learning

Communication overhead is a main bottleneck in federated learning (FL) especially in the wireless environment due to the limited data rate and unstable radio channels. The communication challenge necessitates holistic selection of participating clients that accounts for both the computation needs and communication cost, as well as judicious allocation of the limited transmission resource. Meanwhile, the random unpredictable nature of both the training data samples and the communication channels requires an online optimization approach that adapts to the changing system state over time. In this work, we consider a general framework of online joint client sampling and power allocation for wireless FL under time-varying communication channels. We formulate it as a stochastic network optimization problem that admits a Lyapunov-typed solution approach. This leads to per-training-round subproblems with a special bi-convex structure, which we leverage to propose globally optimal solutions, culminating in a meta algorithm that provides strong performance guarantees. We further study three specific FL problems covering multiple scenarios, namely with IID or non-IID data, whether robustness against data drift is required, and with unbiased or biased client sampling. We derive detailed algorithms for each of these problems. Simulation with standard classification tasks demonstrate that the proposed communication-aware algorithms outperform their counterparts under a wide range of learning and communication scenarios.

Federated learning with tensor networks: a quantum AI framework for healthcare

Abstract The healthcare industries frequently handle sensitive and proprietary data, and due to strict
privacy regulations, they are often reluctant to share data directly. In today’s context, Federated
Learning (FL) stands out as a crucial remedy, facilitating the rapid advancement of distributed machine
learning while effectively managing critical concerns regarding data privacy and governance.
The fusion of federated learning and quantum computing represents a groundbreaking interdisciplinary
approach with immense potential to revolutionize various industries, from healthcare to
finance. In this work, we propose a federated learning framework based on quantum tensor networks
that takes advantage of the principles of many-body quantum physics. Currently, there are
no known classical tensor networks implemented in federated settings. Furthermore, we investigated
the effectiveness and feasibility of the proposed framework by conducting a differential privacy analysis
to ensure the security of sensitive data across healthcare institutions. Experiments on popular
medical image datasets show that the federated quantum tensor network model achieved a mean
receiver-operator characteristic area under the curve (ROC-AUC) of 91-98%, outperforming several
state-of-the-art federated learning methods while using fewer parameters. Experimental results
demonstrate that the quantum federated global model, consisting of highly entangled tensor network
structures, showed better generalization and robustness and achieved higher testing accuracy,
surpassing the performance of locally trained clients under unbalanced data distributions among
healthcare institutions.

Multi‐Objective Federated Averaging Algorithm

ABSTRACTThe recent global trend is the convergence of information and communications technology (ICT). By applying ICT in various fields such as the humanities, new types of products and services are created, and new values that help people's lives can be created. AI can be selected as a representative technology in such convergence ICT. However, applying AI technology to actual production requires ensuring data security. Federated learning (FL) can achieve secure sharing of data, where all parties participate in model training locally and upload it to the server for aggregation. The data never leaves the parties involved, thus solving the problems of data privacy and data silos. However, FL faces issues such as high communication cost, imbalanced performance distribution among participants, and low privacy protection. To achieve a balance between model accuracy, communication cost, fairness, and privacy, this paper proposes a multi‐objective optimization‐based FL algorithm (M‐FedAvg). The multi‐objective optimization problem of maximising the accuracy of the global model, minimising the communication cost, minimising the variance of the accuracy, and minimising the privacy budget is solved by NSGA‐III. The experimental results show that the algorithm proposed can effectively reduce the communication cost of FL and achieve privacy protection for participants without affecting the accuracy of the global model.

An Application Programming Interface (API) Sensitive Data Identification Method Based on the Federated Large Language Model

The traditional methods for identifying sensitive data in APIs mainly encompass rule-based and machine learning-based approaches. However, these methods suffer from inadequacies in terms of security and robustness, exhibit high false positive rates, and struggle to cope with evolving threat landscapes. This paper proposes a method for detecting sensitive data in APIs based on the Federated Large Language Model (FedAPILLM). This method applies the large language model Qwen2.5 and the LoRA instruction tuning technique within the framework of federated learning (FL) to the field of data security. Under the premise of protecting data privacy, a domain-specific corpus and knowledge base are constructed for pre-training and fine-tuning, resulting in a large language model specifically designed for identifying sensitive data in APIs. This paper conducts comparative experiments involving Llama3 8B, Llama3.1 8B, and Qwen2.5 14B. The results demonstrate that Qwen2.5 14B can achieve similar or better performance levels compared to the Llama3.1 8B model with fewer training iterations.

Federated learning with differential privacy via fast Fourier transform for tighter-efficient combining.

Spurred by the simultaneous need for data privacy protection and data sharing, federated learning (FL) has been proposed. However, it still poses a risk of privacy leakage in it. This paper, an improved Differential Privacy (DP) algorithm to protect the federated learning model. Additionally, the Fast Fourier Transform (FFT) is used in the computation of the privacy budget [Formula: see text], to minimize the impact of limited arithmetic resources and numerous users on the effectiveness of training model. Moreover, instead of direct analyses of the privacy budget ε through various methods, Privacy Loss Distribution (PLD) and privacy curves are adopted, while the number of artificial assignments hyperparameters is reduced, and the grid parameters delineated for FFT use are improved. The improved algorithm tightens parameter bounds and minimizes human factors' influence with minimal efficiency impact. It decreases the errors caused by truncation and discreteness of PLDs while expanding the discreteness interval to reduce the calculation workload. Furthermore, an improved activation function using a temper sigmoid with only one parameter [Formula: see text], smooths the accuracy curve and mitigates drastically fluctuating scenarios during model training. Finally, simulation results on real datasets show that our improved DP algorithm, which accounts for long trailing, facilitates a better balance between privacy and utility in federated learning models.

Privacy-Preserving Techniques in Generative AI and Large Language Models: A Narrative Review

Generative AI, including large language models (LLMs), has transformed the paradigm of data generation and creative content, but this progress raises critical privacy concerns, especially when models are trained on sensitive data. This review provides a comprehensive overview of privacy-preserving techniques aimed at safeguarding data privacy in generative AI, such as differential privacy (DP), federated learning (FL), homomorphic encryption (HE), and secure multi-party computation (SMPC). These techniques mitigate risks like model inversion, data leakage, and membership inference attacks, which are particularly relevant to LLMs. Additionally, the review explores emerging solutions, including privacy-enhancing technologies and post-quantum cryptography, as future directions for enhancing privacy in generative AI systems. Recognizing that achieving absolute privacy is mathematically impossible, the review emphasizes the necessity of aligning technical safeguards with legal and regulatory frameworks to ensure compliance with data protection laws. By discussing the ethical and legal implications of privacy risks in generative AI, the review underscores the need for a balanced approach that considers performance, scalability, and privacy preservation. The findings highlight the need for ongoing research and innovation to develop privacy-preserving techniques that keep pace with the scaling of generative AI, especially in large language models, while adhering to regulatory and ethical standards.

PriPrune: Quantifying and Preserving Privacy in Pruned Federated Learning

Model pruning has been proposed as a technique for reducing the size and complexity of Federated learning (FL) models. By making local models coarser, pruning is intuitively expected to improve protection against privacy attacks. However, the level of this expected privacy protection has not been previously characterized, or optimized jointly with utility. In this paper, we first characterize the privacy offered by pruning. We establish information-theoretic upper bounds on the information leakage from pruned FL and we experimentally validate them under state-of-the-art privacy attacks across different FL pruning schemes. Second, we introduce PriPrune – a privacy-aware algorithm for pruning in FL. PriPrune uses defense pruning masks, which can be applied locally after any pruning algorithm, and adapts the defense pruning rate to jointly optimize privacy and accuracy. Another key idea in the design of PriPrune is Pseudo-Pruning : it undergoes defense pruning within the local model and only sends the pruned model to the server; while the weights pruned out by defense mask are withheld locally for future local training rather than being removed. We show that PriPrune significantly improves the privacy-accuracy tradeoff compared to state-of-the-art pruned FL schemes. For example, on the FEMNIST dataset, PriPrune improves the privacy of PruneFL by 45.5% without reducing accuracy.

DP-Poison: Poisoning Federated Learning under the Cover of Differential Privacy

Federated learning (FL) enables resource-constrained node devices to learn a shared model while keeping the training data local. Since recent research has demonstrated multiple privacy leakage attacks in FL, e.g., gradient inference attacks and membership inference attacks, differential privacy (DP) is applied to serve as one of the most effective privacy protection mechanisms. Despite the benefit DP brings, we observe that the introduction of DP also brings random changes to client updates, which will affect the robust aggregation algorithms. We reveal a novel poisoning attack under the cover of DP, named the DP-Poison attack in FL. Specifically, the DP-Poison attack is designed to achieve four goals: 1) maintaining the main task performance; 2) launching a successful attack; 3) escaping the robust aggregation algorithms in FL; and 4) keeping the effectiveness of DP privacy protection. To achieve these goals, we design multiple optimization goals to generate DP noise through a genetic algorithm. The optimization ensures that while the benign updates change randomly, the malicious updates can change towards the global model after adding the DP noise, so that it is easier to be accepted by the robust aggregation algorithms. Extensive experiments show that DP-Poison achieves a nearly 100% attack success rate while maintaining the proposed four goals.

SAMFL: Secure Aggregation Mechanism for Federated Learning with Byzantine-robustness by functional encryption

Federated learning (FL) enables collaborative model training without sharing private data, thereby potentially meeting the growing demand for data privacy protection. Despite its potentials, FL also poses challenges in achieving privacy-preservation and Byzantine-robustness when handling sensitive data. To address these challenges, we present a novel Secure Aggregation Mechanism for Federated Learning with Byzantine-Robustness by Functional Encryption (SAMFL). Our approach designs a novel dual-decryption multi-input functional encryption (DD-MIFE) scheme, which enables efficient computation of cosine similarities and aggregation of encrypted gradients through a single ciphertext. This innovative scheme allows for dual decryption, producing distinct results based on different keys, while maintaining high efficiency. We further propose TF-Init, integrating DD-MIFE with Truth Discovery (TD) to eliminate the reliance on a root dataset. Additionally, we devise a secure cosine similarity calculation aggregation protocol (SC2AP) using DD-MIFE, ensuring privacy-preserving and Byzantine-robust FL secure aggregation. To enhance FL efficiency, we employ single instruction multiple data (SIMD) to parallelize encryption and decryption processes. Concurrently, to preserve accuracy, we incorporate differential privacy (DP) with selective clipping of model layers within the FL framework. Finally, we integrate TF-Init, SC2AP, SIMD, and DP to construct SAMFL. Extensive experiments demonstrate that SAMFL successfully defends against both inference attacks and poisoning attacks, while improving efficiency and accuracy compared to existing methods. SAMFL provides a comprehensive integrated solution for FL with efficiency, accuracy, privacy-preservation, and robustness.

SecDefender: Detecting low-quality models in multidomain federated learning systems

Federated learning (FL) is an innovative distributed learning paradigm that permits multiple parties to train models collaboratively while protecting individual privacy. However, it encounters security challenges, making it vulnerable to several adversarial attacks and leading to compromising model performance. Existing research on FL poisoning attacks and defense techniques tends to be application-specific, primarily emphasizing attack capabilities. However, it fails to consider inherent vulnerabilities in FL and the impact of attack intensity. To our knowledge, no existing work has delved into these issues within a multi-domain FL environment. This paper addresses these concerns by investigating the consequences of targeted label-flipping attacks within FL systems and comprehensively examining the effects of the attacks in single-label, double-label, and triple-label scenarios with different levels of poisoning intensities. Additionally, we investigate the influence of a temporal label-flipping attack, where we study the impact of adversaries available only for specific federated training rounds. Moreover, we propose a novel server-based defense mechanism called SecDefender to detect low-quality models in both IID and non-IID settings of multi-domain environments. Our approach is rigorously evaluated against state-of-the-art alternatives using six benchmark datasets: CIC-Darknet2020, Fashion-MNIST, FEDMNIST, GTSR, HAR, and MNIST. Extensive experiments demonstrate that our proposed SecDefender significantly enhances its performance by over 65% in terms of source class recall, maintaining a low attack success rate. Consequently, there is a 1 to 2% enhancement in global model accuracy compared to existing approaches.

FedART: A neural model integrating federated learning and adaptive resonance theory

Federated Learning (FL) has emerged as a promising paradigm for collaborative model training across distributed clients while preserving data privacy. However, prevailing FL approaches aggregate the clients’ local models into a global model through multi-round iterative parameter averaging. This leads to the undesirable bias of the aggregated model towards certain clients in the presence of heterogeneous data distributions among the clients. Moreover, such approaches are restricted to supervised classification tasks and do not support unsupervised clustering. To address these limitations, we propose a novel one-shot FL approach called Federated Adaptive Resonance Theory (FedART) which leverages self-organizing Adaptive Resonance Theory (ART) models to learn category codes, where each code represents a cluster of similar data samples. In FedART, the clients learn to associate their private data with various local category codes. Under heterogeneity, the local codes across different clients represent heterogeneous data. In turn, a global model takes these local codes as inputs and aggregates them into global category codes, wherein heterogeneous client data is indirectly represented by distinctly encoded global codes, in contrast to the averaging out of parameters in the existing approaches. This enables the learned global model to handle heterogeneous data. In addition, FedART employs a universal learning mechanism to support both federated classification and clustering tasks. Our experiments conducted on various federated classification and clustering tasks show that FedART consistently outperforms state-of-the-art FL methods on data with heterogeneous distribution across clients.