Federated Learning Systems Research Articles

BFL-SA: Blockchain-based federated learning via enhanced secure aggregation

Federated learning, involving a central server and multiple clients, aims to keep data local but raises privacy concerns like data exposure and participation privacy. Secure aggregation, especially with pairwise masking, preserves privacy without accuracy loss. Yet, issues persist like security against malicious models, central server fault tolerance, and trust in decryption keys. Resolving these challenges is vital for advancing secure federated learning systems. In this paper, we present BFL-SA, a blockchain-based federated learning scheme via enhanced secure aggregation, which addresses key challenges by integrating blockchain consensus, publicly verifiable secret sharing, and an overdue gradients aggregation module. These enhancements significantly boost security and fault tolerance while improving the efficiency of data utilization in the secure aggregation process. After security analysis, we have demonstrated that BFL-SA achieves secure aggregation even in malicious models. Through experimental comparative analysis, BFL-SA exhibits rapid secure aggregation speed and achieves 100% model aggregation accuracy.

The Impact of Adversarial Attacks on Federated Learning: A Survey.

Federated learning (FL) has emerged as a powerful machine learning technique that enables the development of models from decentralized data sources. However, the decentralized nature of FL makes it vulnerable to adversarial attacks. In this survey, we provide a comprehensive overview of the impact of malicious attacks on FL by covering various aspects such as attack budget, visibility, and generalizability, among others. Previous surveys have primarily focused on the multiple types of attacks and defenses but failed to consider the impact of these attacks in terms of their budget, visibility, and generalizability. This survey aims to fill this gap by providing a comprehensive understanding of the attacks' effect by identifying FL attacks with low budgets, low visibility, and high impact. Additionally, we address the recent advancements in the field of adversarial defenses in FL and highlight the challenges in securing FL. The contribution of this survey is threefold: first, it provides a comprehensive and up-to-date overview of the current state of FL attacks and defenses. Second, it highlights the critical importance of considering the impact, budget, and visibility of FL attacks. Finally, we provide ten case studies and potential future directions towards improving the security and privacy of FL systems.

Constructing Executing and Overcoming Challenges in Distributed AI Systems: A Study of Federated Learning Framework

Federated learning stands out as a promising approach within the realm of distributed artificial intelligence (AI) systems, facilitating collaborative model training across decentralized devices while safeguarding data privacy. This study presents a thorough investigation into federated learning architecture, covering its foundational design principles, implementation methodologies, and the significant challenges encountered in distributed AI systems. We delve into the fundamental mechanisms underpinning federated learning, elucidating its merits in diverse environments and its prospective applications across various domains. Additionally, we scrutinize the technical complexities associated with deploying federated learning systems, including considerations such as communication efficiency, model aggregation techniques, and security protocols. By amalgamating insights gleaned from recent research endeavors and practical deployments, this study furnishes valuable guidance for both researchers and practitioners aiming to harness federated learning for the development of scalable and privacy-preserving AI solutions.

Constructing Executing and Overcoming Challenges in Distributed AI Systems: A Study of Federated Learning Framework

Federated learning stands out as a promising approach within the realm of distributed artificial intelligence (AI) systems, facilitating collaborative model training across decentralized devices while safeguarding data privacy. This study presents a thorough investigation into federated learning architecture, covering its foundational design principles, implementation methodologies, and the significant challenges encountered in distributed AI systems. We delve into the fundamental mechanisms underpinning federated learning, elucidating its merits in diverse environments and its prospective applications across various domains. Additionally, we scrutinize the technical complexities associated with deploying federated learning systems, including considerations such as communication efficiency, model aggregation techniques, and security protocols. By amalgamating insights gleaned from recent research endeavors and practical deployments, this study furnishes valuable guidance for both researchers and practitioners aiming to harness federated learning for the development of scalable and privacy-preserving AI solutions.

OpenFL: A scalable and secure decentralized federated learning system on the Ethereum blockchain

Decentralized Federated Learning (FL) offers a paradigm where independent entities collaboratively train a machine learning model while preserving the privacy of their datasets. Integrating blockchain technology into decentralized FL frameworks is critical to establishing the trust necessary for user participation. However, existing FL systems using blockchain often struggle with scalability, latency, and privacy issues, particularly in permissionless blockchain contexts. This paper proposes OpenFL, a novel, collateral-backed reputation system implemented on the Ethereum blockchain. This system aims to foster trust among participants in a decentralized FL environment. We present a fully autonomous smart contract platform specifically tailored to facilitate FL processes among anonymous users. Furthermore, we address potential security concerns by detailing our strategies to mitigate various attack vectors. To validate our system’s efficacy, we conducted experiments on the Ethereum Ropsten testnet using the MNIST and CIFAR-10 datasets. Our findings demonstrate OpenFL’s capability to overcome the inherent limitations of permissionless blockchains while highlighting the significance of open-access protocols in this context. OpenFL can potentially broaden the participant base in trust-sensitive applications by reducing entry barriers, thus substantially contributing to decentralized machine learning.

Marvel: Towards Efficient Federated Learning on IoT Devices

Federated learning has gained significant attention as a distributed machine learning paradigm, particularly due to its ability to preserve privacy by keeping data on IoT devices and only transmitting model parameters to the central server. However, existing federated learning systems face several challenges in their deployment. These challenges can be categorized into three main areas: data relation obliviousness, model characteristic obliviousness, and low communication bandwidth. To address these issues, we propose Marvel, a novel consolidated federated learning system designed specifically for efficient model training on IoT devices. Marvel tackles the challenges by employing a series of innovative techniques. Firstly, it clusters devices based on their data similarities, leveraging data fingerprints and the Hamming distance between them. This clustering process allows Marvel to group devices with similar data, enabling more efficient model training. Once the device clustering is complete, Marvel initiates the formal model training process. During this process, Marvel dynamically selects one device from each cluster to upload model parameters based on the predicted model convergence speed. This prediction is achieved using a lightweight LSTM-based prediction model, which helps determine the most suitable devices for parameter updates. Furthermore, Marvel employs a Tucker decomposition approach to decompose the parameters before transmission in each communication round. This decomposition reduces the size of the transmitted data and enables faster communication between devices and the central server. The parameters are then reconstructed on the central server, and vice versa when dispatching model parameters to devices. We evaluate Marvel using a 180-device testbed and implement a prototype using PyTorch. The results demonstrate the effectiveness of Marvel, as it achieves an average model training speedup of 59.81% and reduces communication rounds by up to 66.67% compared to existing schemes.

A Huber Loss Minimization Approach to Byzantine Robust Federated Learning

Federated learning systems are susceptible to adversarial attacks. To combat this, we introduce a novel aggregator based on Huber loss minimization, and provide a comprehensive theoretical analysis. Under independent and identically distributed (i.i.d) assumption, our approach has several advantages compared to existing methods. Firstly, it has optimal dependence on epsilon, which stands for the ratio of attacked clients. Secondly, our approach does not need precise knowledge of epsilon. Thirdly, it allows different clients to have unequal data sizes. We then broaden our analysis to include non-i.i.d data, such that clients have slightly different distributions.

Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity

Motivated by high resource costs of centralized machine learning schemes as well as data privacy concerns, federated learning (FL) emerged as an efficient alternative that relies on aggregating locally trained models rather than collecting clients' potentially private data. In practice, available resources and data distributions vary from one client to another, creating an inherent system heterogeneity that leads to deterioration of the performance of conventional FL algorithms. In this work, we present a federated quantization-based self-supervised learning scheme (Fed-QSSL) designed to address heterogeneity in FL systems. At clients' side, to tackle data heterogeneity we leverage distributed self-supervised learning while utilizing low-bit quantization to satisfy constraints imposed by local infrastructure and limited communication resources. At server's side, Fed-QSSL deploys de-quantization, weighted aggregation and re-quantization, ultimately creating models personalized to both data distribution as well as specific infrastructure of each client's device. We validated the proposed algorithm on real world datasets, demonstrating its efficacy, and theoretically analyzed impact of low-bit training on the convergence and robustness of the learned models.

RoPE: Defending against backdoor attacks in federated learning systems

Federated learning (FL) is vulnerable to backdoor attacks, which aim to cause the misclassification on samples with a specific backdoor. Most existing algorithms are restricted by some conditions, such as the data distribution across the joint clients, the number of attackers and some auxiliary information, thereby being limited in the practical FL. In this paper, we propose RoPE containing three parts: using principal component analysis to extract the important features of model gradients; leveraging expectation–maximization to separate malicious clients from benign ones in accordance to the important features; removing the potential malicious gradients within the selected cluster with Isolated Forest. RoPE requires no restricted assumptions during the training process. We evaluate the performance of RoPE on three image classification tasks under non-independent and identically distributed scenario (non-iid) against centralized backdoor attacks with various ratios of attackers and distributed backdoor attacks, respectively. We also evaluate the performance of RoPE against other backdoor attack scenarios, including independent and identically distributed (iid) scheme, elaborately designed attack schemes. The results show that RoPE can defend against these backdoor attacks and outperform the existing algorithms. In addition, we also explore the impact of different numbers of features on RoPE’s performance and conduct ablation experiments.

Defending against label-flipping attacks in federated learning systems using uniform manifold approximation and projection

<span lang="EN-US">The user experience can be greatly improved by using learning models that have been trained using data from mobile devices and other internet of things (IoT) devices. Numerous efforts have been made to implement federated learning (FL) algorithms in order to facilitate the success of machine learning models. Researchers have been working on various privacy-preserving methodologies, such as deep neural networks (DNN), support vector machines (SVM), logistic regression, and gradient boosted decision trees, to support a wider range of machine learning models. The capacity for computing and storage has increased over time, emphasizing the growing significance of data mining in engineering. Artificial intelligence and machine learning have recently achieved remarkable progress. We carried out research on data poisoning attacks in the FL system and proposed defence technique using uniform manifold approximation and projection (UMAP). We compare the efficiency by using UMAP, principal component analysis (PCA), Kernel principal component analysis (KPCA) and k-mean clustering algorithm. We make clear in the paper that UMAP performs better than PCA, KPCA and k-mean, and gives excellent performance in detection and mitigating against data-poisoning attacks.</span>

Gradient Leakage Attacks in Federated Learning: Research Frontiers, Taxonomy, and Future Directions

Federated learning (FL) is a distributed deep learning framework that has become increasingly popular in recent years. Essentially, FL supports numerous participants and the parameter server to co-train a deep learning model through shared gradients without revealing the private training data. Recent studies, however, have shown that a potential adversary (either the parameter server or participants) can recover private training data from the shared gradients, and such behavior is called gradient leakage attacks (GLAs). In this study, we first present an overview of FL systems and outline the GLA philosophy. We classify the existing GLAs into two paradigms: optimizationbased and analytics-based attacks. In particular, the optimizationbased approach defines the attack process as an optimization problem, whereas the analytics-based approach defines the attack as a problem of solving multiple linear equations. We present a comprehensive review of the state-of-the-art GLA algorithms followed by a detailed comparison. Based on the observations of the shortcomings of the existing optimization-based and analyticsbased methods, we devise a new generation-based GLA paradigm. We demonstrate the superiority of the proposed GLA in terms of data reconstruction performance and efficiency, thus posing a greater potential threat to federated learning protocols. Finally, we pinpoint a variety of promising future directions for GLA.

CCSFLF: Cloud‐edge‐terminal collaborative self‐adaptive federated learning framework

SummaryThis article addresses the issue of ensuring model accuracy and training efficiency in a constrained federated learning environment. In an actual federated learning environment, each device's software, hardware, and network conditions are heterogeneous. Some terminal devices may not be able to undertake the work assigned by the server, resulting in poor model accuracy and slower convergence speed. However, existing research cannot ensure that each terminal device participating in training can handle the workload allocated by the system without collecting too much equipment information. This article proposes the cloud‐edge‐terminal collaborative self‐adaptive federated learning framework (CCSFLF) to solve this problem. This framework combines the advantages of federated learning and edge computing, reduces the probability that devices cannot handle the workload of system allocation, solves the system heterogeneity, and improves the efficiency of federated learning. CCSFLF can adaptively adjust the number of training tasks for terminal devices and select valuable training participants using a terminal device selection strategy. Multiple edge servers can simultaneously aggregate local models. Cloud servers are responsible for the aggregation and task distribution of global models. The above strategy enables the framework to have a faster convergence rate and higher model accuracy. The experimental results confirm that this framework can reduce the dropout rate of terminal devices by more than 5% in heterogeneous federated learning systems, improve the model accuracy by about 2%, and reduce the training time by 1/3 compared with similar methods, with better performance and applicability.

FlexibleFL: Mitigating poisoning attacks with contributions in cloud-edge federated learning systems

Cloud-edge architecture is an emerging technology that aims to meet the growing demands of intelligent applications. To address the issues of machine learning privacy leakages and benefiting from imbalanced data distribution, federated learning has been widely applied. Nevertheless, they present inherent vulnerabilities that make them vulnerable to poisoning attacks. Existing defense techniques are largely attack-rigid: they are designed to recognize client properties or model updates directly, aimed at specific attack scenarios or rules, but may not work well for critical feature patterns or flexible attack methods, mainly due to the potential influence of redundant features and model performance on defense. Yet few flexible defense methods have been developed. In this paper, we propose FlexibleFL, a flexible defense method against poisoning attacks in cloud-edge federated learning system (CEFL). The key idea of FlexibleFL is to evaluate the quality of uploaded model parameters and further determine the contribution of participants through an optimal threshold selection strategy. Based on these differences, FlexibleFL can thus implement penalties to potential attackers in a way that involves assigning the updated federated model. Extensive results demonstrate that our method has significant advantages in countering poisoning attacks in IID and Non-IID scenarios, and can effectively protect CEFL systems.

Research advanced in FL systems based on blockchain

Federated learning (FL), as an excellent distributed machine learning paradigm, has gradually entered the public eye. FLs purpose is to solve the difficulty of centralized management of distributed data in real life and the prominent issues of data privacy, as well as data security. In FL, the central server distributes training tasks to clients to facilitate the training process. The client trains data locally and only uploads the updated model of local training to the central server, which is able to protect local data security effectively. In spite of that, FL still have disadvantages over high single-point failure, as well as lacking incentive mechanism. Thus, due to excellent decentralized nature of blockchain, a positive and feasible solution appears because of the technology combined with blockchain. In this article, we will introduce FL systems based on blockchain (BFL) and the current status of this field in detail. Specifically, we will first discuss the system coupled structure of BFL. Then, we will provide more details on challenges in BFL and corresponding solutions and explain main applications of BFL in various industries. Finally, we will discuss the difficult problems faced by BFL and give promising research directions in the future.

VFL-Chain: Bulletproofing Federated Learning in the V2X environments

Federated Learning (FL) has gained significant traction as a promising approach to enable collaborative machine learning (ML) while safeguarding data privacy across diverse applications, with the Vehicle-to-Everything (V2X) environment being a notable use case. However, conventional FL systems remain susceptible to model poisoning attacks, wherein malicious participants can introduce inaccurate or deliberately misleading local model updates to the central aggregator. In the context of V2X, such attacks could spawn catastrophic outcomes and jeopardize safety-critical applications. To tackle this pressing concern, we introduce a privacy-preserving and verifiable FL framework, dubbed VFL-Chain, designed to facilitate secure and efficient collaboration among intelligent connected vehicles (ICVs). VFL-Chain aspires to deliver a privacy-centric, verifiable FL experience complemented by a robust incentive mechanism tailored for ICVs. Our proposed solution leverages Bulletproofs to enable efficient verification of model update integrity, which are assimilated within smart contracts on an underlying permissioned blockchain. Furthermore, VFL-Chain presents a fair incentive mechanism that rewards honest participation, fostering a resilient and efficient FL implementation. We conduct an exhaustive security analysis and performance assessment of our proposed system, which underscores its efficacy in countering data poisoning attacks and augmenting the accuracy of FL.

Rec-Def: A Recommendation-Based Defence Mechanism for Privacy Preservation in Federated Learning Systems

Rec-Def: A Recommendation-Based Defence Mechanism for Privacy Preservation in Federated Learning Systems

Enhancing Communication Efficiency and Training Time Uniformity in Federated Learning through Multi-Branch Networks and the Oort Algorithm

In this study, we present a federated learning approach that combines a multi-branch network and the Oort client selection algorithm to improve the performance of federated learning systems. This method successfully addresses the significant issue of non-iid data, a challenge not adequately tackled by the commonly used MFedAvg method. Additionally, one of the key innovations of this research is the introduction of uniformity, a metric that quantifies the disparity in training time amongst participants in a federated learning setup. This novel concept not only aids in identifying stragglers but also provides valuable insights into assessing the fairness and efficiency of the system. The experimental results underscore the merits of the integrated multi-branch network with the Oort client selection algorithm and highlight the crucial role of uniformity in designing and evaluating federated learning systems.

Fair Federated Learning with Opposite GAN

Federated learning has surfaced as a proficient strategy for developing distributed machine learning models, leveraging decentralized data sources. This approach negates the requirement for centralizing raw data, thereby maintaining privacy and harnessing the strength of local data. Nevertheless, non-uniform performance and biases in the algorithm across different clients can adversely affect the results. To tackle these challenges, we introduce a method named Fair Federated Learning with Opposite Generative Adversarial Networks (FFL-OppoGAN), a method that leverages Opposite Generative Adversarial Networks (OppoGAN) to generate synthetic tabular datasets and incorporate them into federated learning to improve fairness and consistency. By adding synthetic data that minimizes algorithmic discrimination and adjusting the learning process to promote uniform performance among clients, our method ensures a more equitable learning process. We evaluated the effectiveness of FFL-OppoGAN on the Adult and Dutch datasets, chosen for their relevance to our study. The results demonstrate that our method successfully enhances algorithmic fairness and performance consistency. It achieves superior results compared to baseline methods. In conclusion, FFL-OppoGAN offers a robust solution for fair and consistent federated learning, setting a promising precedent for future federated learning systems.

A survey on vulnerability of federated learning: A learning algorithm perspective

Federated Learning (FL) has emerged as a powerful paradigm for training Machine Learning (ML), particularly Deep Learning (DL) models on multiple devices or servers while maintaining data localized at owners’ sites. Without centralizing data, FL holds promise for scenarios where data integrity, privacy and security and are critical. However, this decentralized training process also opens up new avenues for opponents to launch unique attacks, where it has been becoming an urgent need to understand the vulnerabilities and corresponding defense mechanisms from a learning algorithm perspective. This review paper takes a comprehensive look at malicious attacks against FL, categorizing them from new perspectives on attack origins and targets, and providing insights into their methodology and impact. In this survey, we focus on threat models targeting the learning process of FL systems. Based on the source and target of the attack, we categorize existing threat models into four types, Data to Model (D2M), Model to Data (M2D), Model to Model (M2M) and composite attacks. For each attack type, we discuss the defense strategies proposed, highlighting their effectiveness, assumptions and potential areas for improvement. Defense strategies have evolved from using a singular metric to excluding malicious clients, to employing a multifaceted approach examining client models at various phases. In this survey paper, our research indicates that the to-learn data, the learning gradients, and the learned model at different stages all can be manipulated to initiate malicious attacks that range from undermining model performance, reconstructing private local data, and to inserting backdoors. We have also seen these threat are becoming more insidious. While earlier studies typically amplified malicious gradients, recent endeavors subtly alter the least significant weights in local models to bypass defense measures. This literature review provides a holistic understanding of the current FL threat landscape and highlights the importance of developing robust, efficient, and privacy-preserving defenses to ensure the safe and trusted adoption of FL in real-world applications. The categorized bibliography can be found at: https://github.com/Rand2AI/Awesome-Vulnerability-of-Federated-Learning.

Federated learning for green and sustainable 6G IIoT applications

The 6th generation mobile network (6G) is expected to be launched in the early 2030s. The architecture of 6G will be the convergence of space, air, ground, and undersea networks. The power and intelligence of 6G based on advanced AI techniques will realize the concept of the Industrial Internet of Things (IIoT). In this study, we conduct a comprehensive survey of 6G IIoT applications based on Federated Learning (FL), starting from introducing recent advances in FL and IIoT systems to discussing how to integrate them. In particular, we highlight the potential of FL for supporting a range of IIoT systems such as smart medical, intelligent transportation, smart cities, unmanned aerial vehicles, and smart industry. The important discussions to drive FL into IIoT applications are emphasized. Finally, we present challenges and open issues for future research to realize green and suitable FL-based IIoT applications. Index TermsFederated Learning, Artificial Intelligence, Industrial Internet of Things, Smart IoT Applications