Neuron-level defense against backdoor attacks in federated learning

Backdoor attacks and adversarial attacks are two major security threats to deep neural networks (DNNs), with the former one is a training-time data poisoning attack that aims to implant backdoor triggers into models by injecting trigger patterns into training samples, and the latter one is a testing-time attack trying to generate adversarial examples (AEs) from benign images to mislead a well-trained model. While previous works generally treat these two attacks separately, the inherent connection between these two attacks is rarely explored. In this paper, we focus on bridging backdoor and adversarial attacks and observe two intriguing phenomena when applying adversarial attacks on an infected model implanted with backdoors: 1) the sample is harder to be turned into an AE when the trigger is presented; 2) the AEs generated from backdoor samples are highly likely to be predicted as its true labels. Inspired by these observations, we proposed a novel backdoor defense method, dubbed Adversarial-Inspired Backdoor Defense (AIBD), to isolate the backdoor samples by leveraging a progressive top-q scheme and break the correlation between backdoor samples and their target labels using adversarial labels. Through extensive experiments on various datasets against six state-of-the-art backdoor attacks, the AIBD-trained models on poisoned data demonstrate superior performance over the existing defense methods.

This paper highlights vulnerabilities of deep learning-driven semantic communications to backdoor (Trojan) attacks. Semantic communications aims to convey a desired meaning while transferring information from a transmitter to its receiver. The encoder-decoder pair of an autoencoder that is represented by deep neural networks (DNNs) is trained to reconstruct signals such as images at the receiver by transmitting latent features of small size over a limited number of channel uses. In the meantime, the DNN of a semantic task classifier at the receiver is jointly trained with the autoencoder to check the meaning conveyed to the receiver. The complex decision space of the DNNs makes semantic communications susceptible to adversarial manipulations. In a backdoor (Trojan) attack, the adversary adds triggers to a small portion of training samples and changes the label to a target label. When the transfer of images is considered, the triggers can be added to the images or equivalently to the corresponding transmitted or received signals. In test time, the adversary activates these triggers by providing poisoned samples as input to the encoder (or decoder) of semantic communications. The backdoor attack can effectively change the semantic information transferred for the poisoned input samples to a target meaning. As the performance of semantic communications improves with the signal-to-noise ratio and the number of channel uses, the success of the backdoor attack increases as well. Also, increasing the Trojan ratio in training data makes the attack more successful. On the other hand, the attack is selective and its effect on the unpoisoned input samples remains small. Overall, this paper shows that the backdoor attack poses a serious threat to semantic communications and presents novel design guidelines to preserve the meaning of transferred information in the presence of backdoor attacks.

Graph Neural Networks (GNNs) have achieved remarkable success in various applications, while still exhibiting high vulnerability to backdoor attacks when applied to node classification. Existing single-category attack methods typically rely on adaptive triggers that force victim nodes to be misclassified into a fixed target label, but they often neglect the inherent structural and feature priors associated with the target category. In this work, we propose a novel and effective backdoor attack framework Category-Specific Trigger Backdoor Attacks (CSTBA), employing category-specific information to generate more natural and unnoticeable triggers. Specifically, we introduce a Category-Specific Subgraph Triggers Pool (CS-STP) to capture representative patterns of the target category, along with a Match-and-Attach Strategy (MAS) to unnoticeably attach triggers to victim nodes, thereby ensuring that the modifications remain effective and unnoticeable within the graph. Extensive experiments on multiple benchmark datasets demonstrate that our method significantly enhances both the attack success rate (ASRs) and the unnoticeability of the attack compared with existing single-category approaches, highlighting the critical importance of category-aware trigger design in GNN backdoor attacks.

Lack of transparency in deep learning models makes them vulnerable to backdoor attack, which can cause severe security consequences. For a backdoored model, the specific inputs can trigger misclassification rules while it performs normal behaviors on clean data. Existing backdoor attacks usually generate poisoned data by adding an obvious trigger to the original data and mislabeling them, which suffers from poor invisibility and hence can be easily detected. In this paper, we propose Stand-in Backdoor, a more stealthy and powerful backdoor attack, which can completely hide the trigger while maintaining correct labels of poisoned data. Specifically, we design a novel optimization strategy to transform triggers into imperceptible perturbation in the feature space. Furthermore, utilizing the transferability of feature perturbation, we fine-tune the victim model with well-constructed poisoned data that are correctly labeled. Extensive experiments conducted on various image classification tasks demonstrate that our attack outperforms the state-of-the-art work in terms of backdoor stealth and attack performance, without sacrificing the model's utility.

Universal adversarial backdoor attacks to fool vertical federated learning

Backdoor (Trojan) attacks are emerging threats against deep neural networks (DNN). A DNN being attacked will predict to an attacker-desired target class whenever a test sample from any source class is embedded with a backdoor pattern, while correctly classifying clean (attack-free) test samples. Existing backdoor defenses have shown success in detecting whether a DNN is attacked and in reverse-engineering the backdoor pattern in a "post-training" scenario: the defender has access to the DNN to be inspected and a small, clean dataset collected independently, but has no access to the (possibly poisoned) training set of the DNN. However, these defenses neither catch culprits in the act of triggering the backdoor mapping, nor mitigate the backdoor attack at test-time. In this paper, we propose an "in-flight" unsupervised defense against backdoor attacks on image classification that 1) detects use of a backdoor trigger at test-time; and 2) infers the class of origin (source class) for a detected trigger example. The effectiveness of our defense is demonstrated experimentally for a wide variety of DNN architectures, datasets, and backdoor attack configurations.

Deep Learning (DL) has become a key technology that assists radio frequency (RF) signal classification applications, such as modulation classification. However, the DL models are vulnerable to adversarial machine learning threats, such as data manipulation attacks. We study a physical backdoor (Trojan) attack that targets a DL-based modulation classifier. In contrast to digital backdoor attacks, where digital triggers are injected into the training dataset, we use power amplifier (PA) non-linear distortions to create physical triggers before the dataset is formed. During training, the adversary manipulates amplitudes of RF signals and changes their labels to a target modulation scheme, training a backdoored model. At inference, the adversary aims to keep the backdoor attack inactive such that the backdoored model maintains high accuracy on test signals. However, if they apply the same manipulation used during training on these test signals, the backdoor is activated, and the model misclassifies these signals. We demonstrate that our proposed attack achieves high attack success rates with few manipulated RD signals for different noise levels. Furthermore, we test the resilience of the proposed attack to multiple defense techniques, and the results show that these techniques fail to mitigate the attack.

In this paper, we propose a resilient control plane based on endogenous security for Software-Defined Networking (SDN) named SDN-ESRC to prevent vulnerability backdoor attacks. SDN-ESRC uses a set of heterogeneous controllers (e.g., RYU, OpenDayLight, ONOS) to compose the control plane and dynamically and adaptively selects several heterogeneous controller instances from the controller set to detect and correct the malicious control messages. The design of SDN-ESRC faces two challenges: (1) increasing network update delay due to multi-controller comparison and (2) maintaining high controllable security. To address the first challenge, SDN-ESRC adopts the master modification mode to reduce the network update delay and identify malicious control messages. To address the second challenge, SDN-ESRC introduces the comparison modification mode to ensure high availability in real time. We propose an evaluation model for SDN-ESRC and theoretically analyze the SDN-ESRC’s endogenous security performance under three typical backdoor attack scenarios. We implement SDN-ESRC in a prototype system and conduct simulations and experiments. The results show that SDN-ESRC can improve the backdoor damage attack security up to 98.3%, the backdoor random attack security up to 99.99%, and the backdoor coordinated attack security up to 82% at the cost of increasing network update delay less than 8.3%.

Federated Learning (FL), a burgeoning technology, has received increasing attention due to its privacy protection capability. However, the base algorithm FedAvg is vulnerable when it suffers from so-called backdoor attacks. Former researchers proposed several robust aggregation methods. Unfortunately, due to the hidden characteristic of backdoor attacks, many of these aggregation methods are unable to defend against backdoor attacks. What's more, the attackers recently have proposed some hiding methods that further improve backdoor attacks' stealthiness, making all the existing robust aggregation methods fail.To tackle the threat of backdoor attacks, we propose a new aggregation method, X-raying Models with A Matrix (XMAM), to reveal the malicious local model updates submitted by the backdoor attackers. Since we observe that the output of the Softmax layer exhibits distinguishable patterns between malicious and benign updates, unlike the existing aggregation algorithms, we focus on the Softmax layer's output in which the backdoor attackers are difficult to hide their malicious behavior. Specifically, like medical X-ray examinations, we investigate the collected local model updates by using a matrix as an input to get their Softmax layer's outputs. Then, we preclude updates whose outputs are abnormal by clustering. Without any training dataset in the server, the extensive evaluations show that our XMAM can effectively distinguish malicious local model updates from benign ones. For instance, when other methods fail to defend against the backdoor attacks at no more than 20% malicious clients, our method can tolerate 45% malicious clients in the black-box mode and about 30% in Projected Gradient Descent (PGD) mode. Besides, under adaptive attacks, the results demonstrate that XMAM can still complete the global model training task even when there are 40% malicious clients. Finally, we analyze our method's screening complexity and compare the real screening time with other methods. The results show that XMAM is about 10–10000 times faster than the existing methods.

Backdoor Attacks with Wavelet Embedding: Revealing and enhancing the insights of vulnerabilities in visual object detection models on transformers within digital twin systems

Since the emergence of security concerns in artificial intelligence (AI), there has been significant attention devoted to the examination of backdoor attacks. Attackers can utilize backdoor attacks to manipulate model predictions, leading to significant potential harm. However, current research on backdoor attacks and defenses in both theoretical and practical fields still has many shortcomings. To systematically analyze these shortcomings and address the lack of comprehensive reviews, this article presents a comprehensive and systematic summary of both backdoor attacks and defenses targeting multi-domain AI models. Simultaneously, based on the design principles and shared characteristics of triggers in different domains and the implementation stages of backdoor defense, this article proposes a new classification method for backdoor attacks and defenses. We use this method to extensively review backdoor attacks in the fields of computer vision and natural language processing, and we also examine the current applications of backdoor attacks in audio recognition, video action recognition, multimodal tasks, time series tasks, generative learning, and reinforcement learning, while critically analyzing the open problems of various backdoor attack techniques and defense strategies. Finally, this article builds upon the analysis of the current state of AI security to further explore potential future research directions for backdoor attacks and defenses.

PTB: Robust physical backdoor attacks against deep neural networks in real world

Industrial software technology: R Mitchell (ed) Peter Peregrinus Ltd (on behalf of the Institution of Electrical Engineers), London, UK (1987) 289pp £37.00

The development of the Internet of Things (IoT) combined with the emergence of federated learning (FL) makes it possible for mobile edge computing (MEC) to gain insight from physically separated data without violating privacy or burdening communication. Due to the distributed nature of MEC devices, researchers have uncovered that the FL is vulnerable to backdoor attacks, which aim at injecting a subtask into the FL without corrupting the performance of the main task. The backdoor attack achieves high accuracy on both the main task and the backdoor subtask when injected at FL model convergence. However, the effectiveness of the backdoor is weak when injected in early training stage. In this article, we strengthen the early-injected backdoor attack by using information leakage. We show that FL convergence can be expedited if the client's data set mimics the distribution and gradients of the whole population. Based on this observation, we propose a two-phase backdoor attack, which includes a preliminary phase for the subsequent backdoor attack. Taking advantage of the preliminary phase, the later injected backdoor achieves better effectiveness, as the backdoor effect is less likely to be diluted by normal model updates. Extensive experiments are conducted on the MNIST data set under various data heterogeneity settings to evaluate the effectiveness of the proposed backdoor attack. The results show that the proposed backdoor outperforms existing backdoor attacks in both success rate and longevity, even when defense mechanisms are in place.

Denial of Service (DoS) backdoor attacks are a significant threat to computer systems. Backdoor attacks involve the installation of unauthorized software on a target system, which is then used to launch various attacks against other systems. In this paper, we explore several techniques that can be used to defeat DoS backdoor attacks, including access control, encryption, network monitoring, patch management, incident response planning, and employee education. Denial of Service (DoS) and Backdoor attacks are common types of cyber-attacks that can cause severe damage to computer systems and networks. DoS attacks aim to disrupt the availability of a system by overwhelming it with traffic or exploiting vulnerabilities, while Backdoor attacks provide attackers with unauthorized access to a system. This paper proposes a framework for defeating both types of attacks by implementing security measures at multiple levels, including the application layer, network layer, and host layer. The proposed framework involves techniques such as traffic filtering, load balancing, network segmentation, access control, and anomaly detection. At the application layer, the framework proposes implementing security measures such as input validation, error handling, and resource allocation to prevent DoS attacks. At the network layer, the framework recommends deploying firewalls, routers, and intrusion detection systems to filter traffic and detect anomalies. At the host layer, the framework proposes implementing security measures such as operating system hardening, patch management, and antivirus software to prevent Backdoor attacks. The proposed framework aims to provide a comprehensive approach to defeating DoS and Backdoor attacks by combining multiple security measures to provide a layered defense. By implementing the proposed framework, organizations can enhance the security of their computer systems and networks and prevent these types of attacks from causing damage.