Universal Adversarial Perturbations Research Articles

Transferable universal adversarial perturbations against speaker recognition systems

Transferable universal adversarial perturbations against speaker recognition systems

Enhancing image steganography security via universal adversarial perturbations

Enhancing image steganography security via universal adversarial perturbations

Generating Universal Adversarial Perturbations for Quantum Classifiers

Quantum Machine Learning (QML) has emerged as a promising field of research, aiming to leverage the capabilities of quantum computing to enhance existing machine learning methodologies. Recent studies have revealed that, like their classical counterparts, QML models based on Parametrized Quantum Circuits (PQCs) are also vulnerable to adversarial attacks. Moreover, the existence of Universal Adversarial Perturbations (UAPs) in the quantum domain has been demonstrated theoretically in the context of quantum classifiers. In this work, we introduce QuGAP: a novel framework for generating UAPs for quantum classifiers. We conceptualize the notion of additive UAPs for PQC-based classifiers and theoretically demonstrate their existence. We then utilize generative models (QuGAP-A) to craft additive UAPs and experimentally show that quantum classifiers are susceptible to such attacks. Moreover, we formulate a new method for generating unitary UAPs (QuGAP-U) using quantum generative models and a novel loss function based on fidelity constraints. We evaluate the performance of the proposed framework and show that our method achieves state-of-the-art misclassification rates, while maintaining high fidelity between legitimate and adversarial samples.

On Credibility of Adversarial Examples Against Learning-Based Grid Voltage Stability Assessment

Voltage stability assessment is essential for maintaining reliable power grid operations. Stability assessment approaches using deep learning address the shortfalls of the traditional time-domain simulation-based approaches caused by increased system complexity. However, deep learning models are shown to be vulnerable to adversarial examples in the field of computer vision. While this vulnerability has been noticed by the power grid cybersecurity research, the domain-specific analysis on the requirements imposed upon effective attack implementation is still lacking. Although these attack requirements are usually reasonable in computer vision tasks, they can be stringent in the context of power grids. In this paper, we conduct a systematic investigation on the attack requirements and credibility of six representative adversarial example attacks based on a voltage stability assessment application for the New England 10-machine 39-bus power system. We show that (1) compromising about half the transmission system buses' voltage traces is a rule-of-thumb attack requirement; (2) the universal adversarial perturbations regardless of the original clean voltage trajectory possess the same credibility as the widely studied false data injection attacks on power grid state estimation, while the input-specific adversarial perturbations are less credible; (3) the prevailing strong adversarial training thwarts the universal perturbations but fails in defending certain input-specific perturbations. To advance defense to cope with both universal and input-specific adversarial examples, we propose a new approach that simultaneously estimates the predictive uncertainty of any given input of voltage trajectory and thwarts the attacks effectively.

IG-Based Method for Voiceprint Universal Adversarial Perturbation Generation

In this paper, we propose an Iterative Greedy-Universal Adversarial Perturbations (IGUAP) approach based on an iterative greedy algorithm to create universal adversarial perturbations for acoustic prints. A thorough, objective account of the IG-UAP method is provided, outlining its framework and approach. The method leverages a greedy iteration approach to formulate an optimization problem for solving acoustic universal adversarial perturbations, with a new objective function designed to ensure that the attack has higher accuracy in terms of minimizing the perceptibility of adversarial perturbations and increasing the accuracy of successful attacks. The perturbation generation process is described in detail, and the resulting acoustic universal adversarial perturbation is evaluated in both target-attack and no-target-attack scenarios. Experimental analysis and testing were carried out using comparable techniques and dissimilar target models. The findings reveal that the acoustic generality adversarial perturbation produced by the IG-UAP method can obtain effective attack results even when the audio training data sample size is minimal, i.e., one for each category. Moreover, the human ear finds it difficult to detect the loss of original data information and the addition of adversarial perturbation (for the case of a target attack, the ASR values range from 82.4% to 90.2% for the small sample data set). The success rates for untargeted and targeted attacks average 85.8% and 84.9%, respectively.

Improving Transferability of Universal Adversarial Perturbation with Feature Disruption.

Deep neural networks (DNNs) are shown to be vulnerable to universal adversarial perturbations (UAP), a single quasi-imperceptible perturbation that deceives the DNNs on most input images. The current UAP methods can be divided into data-dependent and data-independent methods. The former exhibits weak transferability in black-box models due to overly relying on model-specific features. The latter shows inferior attack performance in white-box models as it fails to exploit the model's response information to benign images. To address the above issues, this paper proposes a novel universal adversarial attack to generate UAP with strong transferability by disrupting the model-agnostic features (e.g., edges or simple texture), which are invariant to the models. Specifically, we first devise an objective function to weaken the significant channel-wise features and strengthen the less significant channel-wise features, which are partitioned by the designed strategy. Furthermore, the proposed objective function eliminates the dependency on labeled samples, allowing us to utilize out-of-distribution (OOD) data to train UAP. To enhance the attack performance with limited training samples, we exploit the average gradient of the mini-batch input to update the UAP iteratively, which encourages the UAP to capture the local information inside the mini-batch input. In addition, we introduce the momentum term to accumulate the gradient information at each iterative step for the purpose of perceiving the global information over the training set. Finally, extensive experimental results demonstrate that the proposed methods outperform the existing UAP approaches. Additionally, we exhaustively investigate the transferability of the UAP across models, datasets, and tasks.

UniAP: Protecting Speech Privacy With Non-Targeted Universal Adversarial Perturbations

Ubiquitous microphones on smart devices considerably raise users' concerns about speech privacy. Since the microphones are primarily controlled by hardware/software developers, profit-driven organizations can easily collect and analyze individuals' daily conversations on a large scale with deep learning models, and users have no means to stop such privacy-violating behavior. In this paper, we propose UniAP to empower users with the capability of protecting their speech privacy from the large-scale analysis without affecting their routine voice activities. Based on our observation of the recognition model, we utilize adversarial learning to generate quasi-imperceptible perturbations to disturb speech signals captured by nearby microphones, thus obfuscating the recognition results of recordings into meaningless contents. As validated in experiments, our perturbations can protect user privacy regardless of what users speak and when they speak. The jamming performance stability is further improved by training optimization. Additionally, the perturbations are robust against noise removal techniques. Extensive evaluations show that our perturbations achieve successful jamming rates of more than 87% in the digital domain and at least 90% and 70% for common and challenging settings, respectively, in the real-life chatting scenario. Moreover, our perturbations, solely trained on DeepSpeech, exhibit good transferability over other models based on similar architecture.

Learning transferable targeted universal adversarial perturbations by sequential meta-learning

Recently, the transferability of adversarial perturbations in non-targeted scenarios has been extensively studied. However, changing the predictions of an unknown model to a pre-defined ‘targeted’ class still remains challenging. In this study, we aim to learn the targeted universal adversarial perturbations (UAPs) with higher transferability by the ensemble of multiple models. First, we observe the phenomenon that the logit of the target class will bias to a specific white-box model in existing ensemble-based attacks. To deal with the issue, we propose a normalized logit loss to narrow the margin of the targeted class's logits among different models. Besides, we introduce a novel sequential meta-learning optimization strategy to further increase transferability, consisting of the inner loop and the outer loop. In the inner loop, we sequentially learn task-specific targeted UAPs for each source model by jointly considering the perturbation from the previous model. In the outer loop, we optimize the task-agnostic targeted UAP by combining the targeted UAPs from the inner loop. Experimental results demonstrate the mutual benefits of the normalized logit loss and the sequential meta-learning optimization strategy for learning targeted adversarial perturbations, outperforming existing ensemble attacks in both white-box and black-box settings. The source code of this study is available at: Link.

Universal adversarial perturbations for multiple classification tasks with quantum classifiers

Abstract Quantum adversarial machine learning is an emerging field that studies the vulnerability of quantum learning systems against adversarial perturbations and develops possible defense strategies. Quantum universal adversarial perturbations are small perturbations, which can make different input samples into adversarial examples that may deceive a given quantum classifier. This is a field that was rarely looked into but worthwhile investigating because universal perturbations might simplify malicious attacks to a large extent, causing unexpected devastation to quantum machine learning models. In this paper, we take a step forward and explore the quantum universal perturbations in the context of heterogeneous classification tasks. In particular, we find that quantum classifiers that achieve almost state-of-the-art accuracy on two different classification tasks can be both conclusively deceived by one carefully-crafted universal perturbation. This result is explicitly demonstrated with well-designed quantum continual learning models with elastic weight consolidation method to avoid catastrophic forgetting, as well as real-life heterogeneous datasets from hand-written digits and medical MRI images. Our results provide a simple and efficient way to generate universal perturbations on heterogeneous classification tasks and thus would provide valuable guidance for future quantum learning technologies.

Detection of backdoor attacks using targeted universal adversarial perturbations for deep neural networks

Backdoor attacks on deep neural networks (DNNs) using targeted universal adversarial perturbations (TUAPs) do not require training datasets and model tampering, and triggers based on TUAPs can make DNNs output any class the adversary wants. Retraining DNNs using adversarial training for security is time-consuming and does not apply to DNNs in runtime. We want to detect backdoors using a black-box testing approach. We observe that after superimposing random noise on the input of a backdoor attack, the output still tends to remain the same, so we propose Sequential Analysis method based on the Metamorphosis Testing (SAMT). We designed two metamorphic relations for test case generation. Using sequential sampling, we calculate the label stability rate (LSR) and infer whether the image to be verified contains a trigger based on the sequential probability ratio change. The experimental results show that our method has a higher backdoor detection success rate (dsr) than the state-of-the-art detection algorithms. Moreover, our method does not need to use the model structure of DNNs, which has more adaptability and generalization ability. Based on our proposed method, we can simply add a backdoor detection layer to detect backdoors as early as possible, which can eventually alleviate the harm of such backdoors.

Exploring aesthetic procedural noise for crafting model-agnostic universal adversarial perturbations

The research on universal adversarial perturbations (UAPs) is significant to trustworthy deep learning. To disentangle the UAPs with the training data dependency and the target model dependency, the exploration of procedural noise functions is a feasible method. However, the current procedural adversarial noise attack method has several characteristics like visually significant anisotropy and gradient artifacts that may impact the stealthiness of adversarial examples. This study proposes a novel model-free and data-free UAP method based on the procedural noise functions with two variants: Simplex noise attack and Worley noise attack. The attack method can achieve deceit on the neural networks with a more aesthetic rendering effect. A detailed empirical study is provided to validate the effectiveness of the proposed attack method. The extensive experiments show that the UAPs generated by the proposed method achieve considerable attack performance on the ImageNet dataset and the CIFAR-10 dataset. Moreover, this study implements the performance evaluation and robustness analysis of existing defense methods against the proposed UAPs. It has the potential to enhance research on the robustness of neural networks in real applications. The code is available at https://github.com/momo1986/adversarial_example_simplex_worley.

Enhanced covertness class discriminative universal adversarial perturbations

The main aim of class discriminative universal adversarial perturbations (CD-UAPs) is that the adversary can flexibly control the targeted class and influence remaining classes limitedly. CD-UAPs generated by the existing attack strategies suffer from a high fooling ratio of non-targeted source classes under non-targeted and targeted attacks, and face the increasing risk of discovery. In this paper, we propose a training framework for generating enhanced covertness CD-UAPs. It trains the targeted source class set and the non-targeted source classes set alternately to update the perturbation and introduces logit pairing to mitigate the influence of perturbation on the non-targeted source classes set. Further, we extend CD-UAPs on the targeted (one-targeted) attack to the multi-targeted attack, which perturbs a targeted source class to multiple targeted sink classes that seriously threaten the current scenario. It can not only provide the adversary with freedom of precise attack but reduce the risk of being detected. This attack poses a strong threat to security-sensitive applications. Extensive experiments on the CIFAR-10, CIFAR-100 and ImageNet datasets show our method can generate more deceptive perturbations and enhance the covertness of CD-UAPs. For example, our method improves the absolute fooling ratio gaps of ResNet-20 and VGG-16 by 9.46% and 6.94% compared with the baseline method, respectively. We achieve the multi-targeted attack with a high fooling ratio on the GTSRB dataset. The average absolute target fooling ratio gaps of ResNet-20 and VGG-16 are 81.89% and 76.33%, respectively.

Generative UAP attacks against deep‐learning based modulation classification

AbstractThanks for its powerful feature extraction capability and unprecedented success in computer vision and language processing tasks, deep learning (DL) has also been applied in recent years to wireless communications. However, DL models are proved to be inherently vulnerable to adversarial perturbations: carefully crafted perturbations that appear imperceptible but can fool the models to induce misclassification. Existing researches in wireless communications employ the gradient‐based or optimization based methods, for example, FGSM, PGD and iterative FGSM, to craft input‐dependent adversarial perturbations. However, these methods have a high computational complexity and the perturbations they craft cannot be readily applied to fool DL‐based algorithms in wireless communications. In this paper, the efficient generation of universal (input‐agnostic) adversarial perturbations (UAPs) used to attack DL‐based modulation classification algorithms is studied. Communication signals exhibit unique characteristics, for example, received modulated signals after transmitting through noisy channels are densely distributed around the modulation constellation points. In this work, the authors have proposed, by making use of this unique characteristics, a generative network to model the distribution of adversarial perturbations for high‐efficiency generation of UAPs, which can fool the DL‐based modulation classification algorithms for most inputs. Instead of attacking all regions of modulated signals, attention mechanisms are introduced in generative network design that enables the perturbations to be concentrated around the constellation points. In this way, a higher attack efficiency and a lower perceptibility are achieved by crafting UAPs with smooth variations and more focused attack. In addition, a diversity term is included in loss function to help capture a wide range distribution of adversary perturbations that will help the generation of more diverse adversary perturbations. Once trained, the generative network can construct with ease a large number of UAPs with some favorable features, which will benefit more efficient adversarial attack and faster adversarial training. Experimental results show the superiority of our proposed method in terms of attack efficiency, imperceptibility and diversity over existing methods.

Generalizing universal adversarial perturbations for deep neural networks

Generalizing universal adversarial perturbations for deep neural networks

Consistent attack: Universal adversarial perturbation on embodied vision navigation

Embodied agents in vision navigation coupled with deep neural networks have attracted increasing attention. However, deep neural networks have been shown vulnerable to malicious adversarial noises, which may potentially cause catastrophic failures in Embodied Vision Navigation. Among different adversarial noises, universal adversarial perturbations (UAP), i.e., a constant image-agnostic perturbation applied on every input frame of the agent, play a critical role in Embodied Vision Navigation since they are computation-efficient and application-practical during the attack. However, existing UAP methods ignore the system dynamics of Embodied Vision Navigation and might be sub-optimal. In order to extend UAP to the sequential decision setting, we formulate the disturbed environment under the universal noise δ, as a δ-disturbed Markov Decision Process (δ-MDP). Based on the formulation, we analyze the properties of δ-MDP and propose two novel Consistent Attack methods, named Reward UAP and Trajectory UAP, for attacking Embodied agents, which consider the dynamic of the MDP and calculate universal noises by estimating the disturbed distribution and the disturbed Q function. For various victim models, our Consistent Attack can cause a significant drop in their performance in the PointGoal task in Habitat with different datasets and different scenes. Extensive experimental results indicate that there exist serious potential risks for applying Embodied Vision Navigation methods to the real world.

AdvDDoS: Zero-Query Adversarial Attacks Against Commercial Speech Recognition Systems

Automatic speech recognition (ASR) has been widely and commercially employed in health care, autonomous vehicles, and finance. Yet, recent studies have shown that universal adversarial perturbations (UAPs) pose a serious threat to white-box ASR systems, when the adversary has access to the target model. Until now, the impacts of such a threat on commercial systems are still open since their models are not publicly available. To understand the security weakness in the practical black-box setting, this paper introduces the first <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">zero-query</i> UAP attacks, called AdvDDoS, with black-box access to ASR systems: we do not need to pay any query expense to estimate UAPs. Specifically, we craft targeted UAPs under a popular feature extractor and a local ASR model by reversing the robust target-category features, in which adversarial perturbations containing robust features are believed to have better transferability. Compared with vanilla UAPs, our UAPs incorporated with target-category features lead to better attacks against commercial ASR systems. We validate the efficacy of our AdvDDoS by launching attacks against a range of commercial ASR systems, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., three API services (Alibaba, Tencent, and Baidu), and three personal assistants (Apple Siri, iFlytek, and Google). Extensive experimental results demonstrate the superiority of AdvDDoS. For example, AdvDDoS achieves 83.26% word error rate (WER) and 53.25% success rates of attacks (SRoA) for the universal attack against Tencent ASR API, which outperforms the vanilla UAPs by up to 61.56% on WER and 11.6% on SRoA. The success of our attack sheds light on zero-query UAP attacks against Commercial ASR systems.

Cover Enhancement Method for Audio Steganography Based on Universal Adversarial Perturbations with Sample Diversification

Cover Enhancement Method for Audio Steganography Based on Universal Adversarial Perturbations with Sample Diversification

Crafting Targeted Universal Adversarial Perturbations: Considering Images as Noise

Crafting Targeted Universal Adversarial Perturbations: Considering Images as Noise

ULAN: A Universal Local Adversarial Network for SAR Target Recognition Based on Layer-Wise Relevance Propagation

Recent studies have proven that synthetic aperture radar (SAR) automatic target recognition (ATR) models based on deep neural networks (DNN) are vulnerable to adversarial examples. However, existing attacks easily fail in the case where adversarial perturbations cannot be fully fed to victim models. We call this situation perturbation offset. Moreover, since background clutter takes up most of the area in SAR images and has low relevance to recognition results, fooling models with global perturbations is quite inefficient. This paper proposes a semi-white-box attack network called Universal Local Adversarial Network (ULAN) to generate universal adversarial perturbations (UAP) for the target regions of SAR images. In the proposed method, we calculate the model’s attention heatmaps through layer-wise relevance propagation (LRP), which is used to locate the target regions of SAR images that have high relevance to recognition results. In particular, we utilize a generator based on U-Net to learn the mapping from noise to UAPs and craft adversarial examples by adding the generated local perturbations to target regions. Experiments indicate that the proposed method effectively prevents perturbation offset and achieves comparable attack performance to conventional global UAPs by perturbing only a quarter or less of SAR image areas.

Detecting the universal adversarial perturbations on high-density sEMG signals

Myoelectric pattern recognition is a promising approach for upper limb neuroprosthetic control. Convolutional neural networks (CNN) are increasingly used in dealing with the electromyography (EMG) signal collected by high-density electrodes due to its capacity to take full advantage of spatial information about muscle activity. However, it has been found that CNN models are very vulnerable to well-designed and tiny perturbations, such like universal adversarial perturbation (UAP). As shown in this work, the CNN-based myoelectric pattern recognition method can achieve a classification accuracy of more than 90%, but can only achieve a classification accuracy of less than 20% after the attack. This type of attack poses a big security concern to prosthetic control. To the best of our knowledge, there is no study on the detection of adversarial attacks to the myoelectric control system. In this paper, a correlation feature based on Chebyshev distance between the adjacent channels is proposed to detect the attack for EMG signals, which will serve as early warning and defense against the adversarial attacks. The performance of the detection framework is assessed with two high-density EMG datasets. The results show that our method has a detection rate of 91.39% and 93.87% for the attacks on both datasets with a latency of no more than 2 ms, which will facilitate the security of muscle–computer interfaces.