Benign Clients Research Articles

BR-FEEL: A backdoor resilient approach for federated edge learning with fragment-sharing

In the resource-constrained federated edge learning (FEEL) systems, fragment-sharing is an efficient approach for multiple clients to cooperatively train a giant model with billions of parameters. Compared with the classical federated learning schemes where the local model is fully trained and exchanged by each client, the fragment-sharing only requires each client to optionally choose a parameter-fragment to train and share, according to its storage, computing, and networking abilities. However, when the full model is no longer delivered in fragment-sharing, the backdoor attacks hidden behind the fragments become harder to be detected, which introduces formidable challenge for the security of FEEL systems. In this paper, we firstly show that the existing fragment-sharing works suffer a lot from the backdoor attacks. Then, a Backdoor-Resilient approach, named BR-FEEL, is introduced to defend against the potential backdoor attacks. Specifically, a twin model is built by each benign client to integrate the parameter-fragments from others. A knowledge distillation process is designed on each client to transfer the clean knowledge from its twin model to local model. With the twin model and knowledge distillation process, our BR-FEEL approach makes sure that the local models of the benign clients will not be backdoored. Experiments on CIFAR-10 and GTSRB datasets with MobileNetV2 and ResNet-34 are conducted. The numerical results demonstrate the efficacy of BR-FEEL on reducing attack success rates by over 90% compared to other baselines under various attack methods.

BapFL: You can Backdoor Personalized Federated Learning

In federated learning (FL), malicious clients could manipulate the predictions of the trained model through backdoor attacks, posing a significant threat to the security of FL systems. Existing research primarily focuses on backdoor attacks and defenses within the generic federated learning scenario, where all clients collaborate to train a single global model. A recent study conducted by Qin et al. [ 24 ] marks the initial exploration of backdoor attacks within the personalized federated learning (pFL) scenario, where each client constructs a personalized model based on its local data. Notably, the study demonstrates that pFL methods with parameter decoupling can significantly enhance robustness against backdoor attacks. However, in this article, we whistleblow that pFL methods with parameter decoupling are still vulnerable to backdoor attacks. The resistance of pFL methods with parameter decoupling is attributed to the heterogeneous classifiers between malicious clients and benign counterparts. We analyze two direct causes of the heterogeneous classifiers: (1) data heterogeneity inherently exists among clients and (2) poisoning by malicious clients further exacerbates the data heterogeneity. To address these issues, we propose a two-pronged attack method, BapFL, which comprises two simple yet effective strategies: (1) poisoning only the feature encoder while keeping the classifier fixed and (2) diversifying the classifier through noise introduction to simulate that of the benign clients. Extensive experiments on three benchmark datasets under varying conditions demonstrate the effectiveness of our proposed attack. Additionally, we evaluate the effectiveness of six widely used defense methods and find that BapFL still poses a significant threat even in the presence of the best defense, Multi-Krum. We hope to inspire further research on attack and defense strategies in pFL scenarios. The code is available at: https://github.com/BapFL/code

Rethinking the defense against free-rider attack from the perspective of model weight evolving frequency

Federated learning (FL) is a distributed machine learning approach where multiple clients collaboratively train a joint model without exchanging their own data. Despite FL's unprecedented success in data privacy-preserving, its vulnerability towards free-rider attacks. Numerous defense methods have been proposed, however, they fail to resist highly camouflaged free-riders. To address these challenges, we reconsider the defense from a novel perspective, i.e., model weight evolving frequency. Empirically, we gain a novel insight that during the FL's training, the model weight evolving frequency of free-riders and that of benign clients are significantly different. Inspired by this insight, we propose a novel defense method based on the model Weight Evolving Frequency, referred to as WEF-Defense. Specifically, we first collect the weight evolving frequency (defined as WEF-Matrix) during local training. Each client uploads the WEF-Matrix of the local model as well as the model weights to the server. The server then separates free-riders from benign clients based on the difference in the WEF-Matrix. At last, the server provides different global models for the corresponding clients using a personalization algorithm, which prevents free-riders from gaining high-quality models. Comprehensive experiments conducted on five datasets and five models demonstrate that WEF-Defense achieves better defense effectiveness (∼×1.4) than the state-of-the-art baselines and identifies free-riders at an earlier stage of training. Besides, we verify the effectiveness of WEF-Defense against an adaptive attack and visualize the WEF-Matrix during the training to interpret its effectiveness. The data and code of WEF-Defense are available at: https://github.com/research-limingjun/WEF-Defense.git.

Adversarial Attacks on Federated-Learned Adaptive Bitrate Algorithms

Learning-based adaptive bitrate (ABR) algorithms have revolutionized video streaming solutions. With the growing demand for data privacy and the rapid development of mobile devices, federated learning (FL) has emerged as a popular training method for neural ABR algorithms in both academia and industry. However, we have discovered that FL-based ABR models are vulnerable to model-poisoning attacks as local updates remain unseen during global aggregation. In response, we propose MAFL (Malicious ABR model based on Federated Learning) to prove that backdooring the learning-based ABR model via FL is practical. Instead of attacking the global policy, MAFL only targets a single ``target client''. Moreover, the unique challenges brought by deep reinforcement learning (DRL) make the attack even more challenging. To address these challenges, MAFL is designed with a two-stage attacking mechanism. Using two representative attack cases with real-world traces, we show that MAFL significantly degrades the model performance on the target client (i.e., increasing rebuffering penalty by 2x and 5x) with a minimal negative impact on benign clients.

RFVIR: A robust federated algorithm defending against Byzantine attacks

Federated learning (FL) is susceptible to Byzantine attacks due to its inherently distributed and privacy-preserving nature. Most model parameters-based defense methods become utterly ineffective under intense non-independent and identically distributed (Non-IID) scenarios. In this paper, we shift our focus from model’s parameters to the specific behavior of the model on a dataset, and propose a robust algorithm named RFVIR to defend against Byzantine attack under FL setting. RFVIR primarily tests the feature representations of each local model on a virtual dataset, and then computes the gram feature matrix to capture the difference between Byzantine attackers and benign clients, and removes suspicious local models based on Median Absolute Deviation (MAD), and finally leverage clipping operation to further mitigate the effect of potential Byzantine attackers. Since RFVIR focuses on the specific behavior of the model rather than the model parameters, it applies to intense Non-IID scenarios. We conduct experiments on CIFAR-10, MNIST, and GTSRB datasets, considering five typical Byzantine attacks and various attack scenarios. The experimental results demonstrate that RFVIR can successfully defend against various Byzantine attacks and outperform the existing robust algorithms.

Privacy-Preserving Federated Learning Against Label-Flipping Attacks on Non-IID Data

Federated learning (FL) has attracted widespread attention in the Internet of Things domain recently. With FL, multiple distributed devices can cooperatively train a global model by transmitting model updates without disclosing the original data. However, the distributed nature of FL makes it vulnerable to data poisoning attacks. In practice, malicious clients can launch the label-flipping attack by simply tampering with the labels of local data, thus causing the global model to misclassify the samples of a selected class as the target class. Although some defense mechanisms have been proposed, they rely on specific assumptions about data distribution, and their performance degrades significantly when the data on clients are non-IID. Besides, most existing methods require clients to upload model updates in plaintext so that the server can identify and remove the malicious updates. But, direct transmission of model updates may still reveal private information. Considering these issues, we develop a label-flipping-robust and privacy-preserving FL (LFR-PPFL) algorithm, which is applicable to both IID and non-IID data. We first propose a detection method based on temporal analysis on cosine similarity to distinguish malicious clients from benign clients. Then, we propose a privacy-preserving computation protocol based on homomorphic encryption to implement this detection method and perform federated aggregation while protecting the privacy of clients. Besides, a detailed theoretical analysis is given to demonstrate the privacy guarantee of the proposed protocol. Experimental results on real-world datasets show that the proposed algorithm can effectively defend against label-flipping attacks under various data distributions.

SCFL: Mitigating backdoor attacks in federated learning based on SVD and clustering

Federated learning (FL) is a distributed machine learning paradigm that enables scattered clients to collaboratively train a shared global model. FL is suitable for privacy-preserving applications due to keeping the training data decentralized. However, FL is susceptible to backdoor attacks which attempt to embed backdoor triggers into the global model during the training process, and later activate them to cause a desired misclassification. In this paper, to effectively defend against backdoor attacks in the FL system, we propose SCFL including three parts: first, the Singular Value Decomposition (SVD) technique is used to extract the significant features of model updates; second, the k-means clustering algorithm is used to cluster the significant features; finally, cosine similarity is used to measure the distance between two model updates and the optimal clients are selected to aggregate the global model after clipping. Unlike most robust algorithms, SCFL does not limit the number of attackers to be less than that of benign clients, nor does restrict the data distribution among all clients to be independent and identically distributed (IID). Moreover, SCFL does not require any auxiliary information outside of the learning process. We conduct extensive experiments including various types of backdoor attacks. Experimental results demonstrate that SCFL can effectively defend against these backdoor attacks and outperform the existing state-of-the-art algorithms.

DeMAC: Towards detecting model poisoning attacks in federated learning system

Federated learning (FL) is an efficient distributed machine learning paradigm for the collaborative training of neural network models by many clients with the assistance of a central server. Currently, the main challenging issue is that malicious clients can send poisoned model updates to the central server, making FL vulnerable to model poisoning attacks. In this paper, we propose a new system named DeMAC to improve the detection and defence against model poisoning attacks by malicious clients. The main idea behind the new system is based on an observation that, as malicious clients need to reduce the poisoning task learning loss, there will be obvious increases in the norm of gradients. We define a metric called GradScore to measure this norm of clients. It is shown through experiments that the GradScores of malicious and benign clients are distinguishable in all training stages. Therefore, DeMAC can detect malicious clients by measuring the GradScore of clients. Furthermore, a historical record of the contributed global model updates is utilized to enhance the DeMAC system, which can spontaneously detect malicious behaviours without requiring manual settings. Experiment results over two benchmark datasets show that DeMAC can reduce the attack success rate under various attack strategies. In addition, DeMAC can eliminate model poisoning attacks under heterogeneous environments.

How to cope with malicious federated learning clients: An unsupervised learning-based approach

With the advent of data-driven and Artificial Intelligence solutions, data providers such as Internet of Things (IoT)-enabled devices and sensors have become essential for intelligent services. In conventional machine learning, data collected from sensors and edge devices are transmitted to a centralized server for model training. Although it achieves state-of-the-art results, this process has drawbacks, such as privacy concerns, data leakage, etc. Specifically, some companies or services like hospitals want to keep their data themselves. Since they do not share data on a server, training on the centralized server cannot take the overall benefit of all data. Therefore a new training method is proposed, namely federated learning. In this learning, users keep their data to themselves and train their local model with their data. After training, users send their model weights to the server instead of data. When the centralized server takes all weights of users, it starts federated aggregations and generates a new global model. This new model is distributed among clients. This method provides privacy and security over clients’ data. However, In the malicious environment, where malicious clients trained with corrupted data send their weights to a centralized server, global model aggregation degrades because of harmful weights. Also, in a collaboration environment, where clients can send and receive data from other clients, malicious clients can send their corrupted data to benign clients, negatively affecting their local training. In the end, those clients might turn into partially malicious clients. In this work, we improve our previous research, score-based aggregation (SBA), and develop a malicious-client elimination algorithm before aggregation. With this algorithm, global model training can be converged higher scores according to unseen validation data scores even if there are 50% malicious rate among clients.

ARMOR: Differential Model Distribution for Adversarially Robust Federated Learning

In this work, we formalize the concept of differential model robustness (DMR), a new property for ensuring model security in federated learning (FL) systems. For most conventional FL frameworks, all clients receive the same global model. If there exists a Byzantine client who maliciously generates adversarial samples against the global model, the attack will be immediately transferred to all other benign clients. To address the attack transferability concern and improve the DMR of FL systems, we propose the notion of differential model distribution (DMD) where the server distributes different models to different clients. As a concrete instantiation of DMD, we propose the ARMOR framework utilizing differential adversarial training to prevent a corrupted client from launching white-box adversarial attack against other clients, for the local model received by the corrupted client is different from that of benign clients. Through extensive experiments, we demonstrate that ARMOR can significantly reduce both the attack success rate (ASR) and average adversarial transfer rate (AATR) across different FL settings. For instance, for a 35-client FL system, the ASR and AATR can be reduced by as much as 85% and 80% over the MNIST dataset.

Privacy-Preserving Federated Learning of Remote Sensing Image Classification With Dishonest Majority

The classification of remote sensing images can give valuable data for various practical applications for smart cities, including urban planning, construction, and water resource management. The federated learning (FL) solution is often adopted to resolve the problems of limited resources and the confidentiality of data in remote sensing image classification. Privacy-preserving federated learning (PPFL) is a state-of-art FL scheme tailored for the privacy-constrained situation. It is required to address safeguarding data privacy and optimizing model accuracy effectively. However, existing PPFL methods usually suffer from model poisoning attacks, especially in the case of dishonest-majority scenarios. To address this challenge, in this work, we propose a blockchain-empowered PPFL for remote sensing image classification framework with the poisonous dishonest majority, which is able to defend against encrypted model poisoning attacks without compromising users' privacy. Specifically, we first propose the method of proof of accuracy (PoA) aiming to evaluate the encrypted models in an authentic way. Then, we design the secure aggregation framework using PoA, which can achieve robustness in a majority proportion of adversary settings. The experimental results show that our scheme can reach 92.5%, 90.61%, 87.48%, and 81.84% accuracy when the attacker accounts for 20%, 40%, 60%, and 80%, respectively. This is consistent with the FedAvg accuracy when only benign clients own the corresponding proportion of data. The experiment results demonstrate the proposed scheme's superiority in defending against model poisoning attacks.

Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning

Data heterogeneity has been identified as one of the key features in federated learning but often overlooked in the lens of robustness to adversarial attacks. This paper focuses on characterizing and understanding its impact on backdooring attacks in federated learning through comprehensive experiments using synthetic and the LEAF benchmarks. The initial impression driven by our experimental results suggests that data heterogeneity is the dominant factor in the effectiveness of attacks and it may be a redemption for defending against backdooring as it makes the attack less efficient, more challenging to design effective attack strategies, and the attack result also becomes less predictable. However, with further investigations, we found data heterogeneity is more of a curse than a redemption as the attack effectiveness can be significantly boosted by simply adjusting the client-side backdooring timing. More importantly, data heterogeneity may result in overfitting at the local training of benign clients, which can be utilized by attackers to disguise themselves and fool skewed-feature based defenses. In addition, effective attack strategies can be made by adjusting attack data distribution. Finally, we discuss the potential directions of defending the curses brought by data heterogeneity. The results and lessons learned from our extensive experiments and analysis offer new insights for designing robust federated learning methods and systems.

A game-theoretic analysis to defend against remote operating system fingerprinting

Remote Operating System (OS) Fingerprinting is a precursory step for launching attacks on the Internet. As a precaution against potential attacks, a remote machine can take a proactive counter-strategy to deceive fingerprinters. This is done by normalizing or mystifying the distinguishing behaviors in the packets. However, the unified modification causes a significant performance degradation to benign clients. Using a game-theoretic approach, we propose a selective and dynamic mechanism for counter-fingerprinting. We analyze the interaction between a fingerprinter and a target by modeling the problem as a signaling game and name it Defense-Game. Here, the defender observes the attacker’s actions and takes moves accordingly. We derive the Nash equilibrium strategy profiles based on the information gain analysis and propose a defense mechanism named Strategic-Defender. Our game-theoretic approach appropriately distinguishes a fingerprinter from a benign client and mystifies packets to confuse the fingerprinter, while minimizing the side effects on benign clients. The performance analysis shows that the defense mechanism can reduce the probability of success of the fingerprinter significantly, without deteriorating the overall performance of other clients. Later, we restructure the game as a screening game where the attacker plays as the follower so that it can recognize the type of the target and act accordingly. We name the game Attack-Game and solve it for the equilibrium strategies. We evaluate the performance of Strategic-Defender when the attacker takes actions according to the optimal results of Attack-Game. The evaluation results show that the attacker’s payoff may increase a bit, however, it is still too low to succeed.

A framework for generating realistic traffic for Distributed Denial-of-Service attacks and Flash Events

An intrinsic challenge associated with evaluating proposed techniques for detecting Distributed Denial-of-Service (DDoS) attacks and distinguishing them from Flash Events (FEs) is the extreme scarcity of publicly available real-word traffic traces. Those available are either heavily anonymised or too old to accurately reflect the current trends in DDoS attacks and FEs. This paper proposes a traffic generation and testbed framework for synthetically generating different types of realistic DDoS attacks, FEs and other benign traffic traces, and monitoring their effects on the target. Using only modest hardware resources, the proposed framework, consisting of a customised software traffic generator, ‘Botloader’, is capable of generating a configurable mix of two-way traffic, for emulating either large-scale DDoS attacks, FEs or benign traffic traces that are experimentally reproducible. Botloader uses IP-aliasing, a well-known technique available on most computing platforms, to create thousands of interactive UDP/TCP endpoints on a single computer, each bound to a unique IP-address, to emulate large numbers of simultaneous attackers or benign clients.