Adversarial Attack Algorithms Research Articles

A Multi-task Adversarial Attack Against Face Authentication

Deep-learning-based identity management systems, such as face authentication systems, are vulnerable to adversarial attacks. However, existing attacks are typically designed for single-task purposes, which means they are tailored to exploit vulnerabilities unique to the individual target rather than being adaptable for multiple users or systems. This limitation makes them unsuitable for certain attack scenarios, such as morphing, universal, transferable, and counter attacks. In this paper, we propose a multi-task adversarial attack algorithm called MTADV that are adaptable for multiple users or systems. By interpreting these scenarios as multi-task attacks, MTADV is applicable to both single- and multi-task attacks, and feasible in the white- and gray-box settings. Furthermore, MTADV is effective against various face datasets, including LFW, CelebA, and CelebA-HQ, and can work with different deep learning models, such as FaceNet, InsightFace, and CurricularFace. Importantly, MTADV retains its feasibility as a single-task attack targeting a single user/system. To the best of our knowledge, MTADV is the first adversarial attack method that can target all of the aforementioned scenarios in one algorithm.

Sample-analysis based adversarial attack with saliency map

With the widespread application of deep learning, the vulnerability of neural networks has attracted considerable attention, raising reliability and security concerns. Therefore, research on the robustness of neural networks has become increasingly critical. In this paper, we propose a novel sample-analysis based robustness evaluation method that overcomes the drawbacks of existing techniques, such as solving difficulty, single strategy, and loose radius. Our algorithm comprises two parts: robustness evaluation and adversarial attacks. Specifically, we introduce formal definitions of multiple sample types and a general solution to the problem of adversarial samples. We formulate a disturbance model-based description of adversarial samples in the adversarial attack algorithm and utilize saliency map to solve them. Our experimental results demonstrate that our adversarial attack algorithm not only achieves a high attack success rate in a relatively small disturbance range but also generates multiple adversarial examples for each clean example. Our algorithm can evaluate the robustness of complex datasets and models, overcome the lack of a single strategy in solving adversarial examples, and provide a more accurate radius of robustness.

Attacking Robot Vision Models Efficiently Based on Improved Fast Gradient Sign Method

The robot vision model is the basis for the robot to perceive and understand the environment and make correct decisions. However, the security and stability of robot vision models are seriously threatened by adversarial examples. In this study, we propose an adversarial attack algorithm, RMS-FGSM, for robot vision models based on root-mean-square propagation (RMSProp). RMS-FGSM uses an exponentially weighted moving average (EWMA) to reduce the weight of the historical cumulative squared gradient. Additionally, it can suppress the gradient growth based on an adaptive learning rate. By integrating with the RMSProp, RMS-FGSM is more likely to generate optimal adversarial examples, and a high attack success rate can be achieved. Experiments on two datasets (MNIST and CIFAR-100) and several models (LeNet, Alexnet, and Resnet-101) show that the attack success rate of RMS-FGSM is higher than the state-of-the-art methods. Above all, our generated adversarial examples have a smaller perturbation than those generated by existing methods under the same attack success rate.

DefenseFea: An Input Transformation Feature Searching Algorithm Based Latent Space for Adversarial Defense

Abstract Deep neural networks based image classification systems could suffer from adversarial attack algorithms, which generate input examples by adding deliberately crafted yet imperceptible noise to original input images. These crafted examples can fool systems and further threaten their security. In this paper, we propose to use latent space protect image classification. Specifically, we train a feature searching network to make up the difference between adversarial examples and clean examples with label guided loss function. We name it DefenseFea(input transformation based defense with label guided loss function), experimental result shows that DefenseFea can improve the rate of adversarial examples that achieved a success rate of about 99% on a specific set of 5000 images from ILSVRC 2012. This study plays a positive role in the further investigation of the relationship between adversarial examples and clean examples.

DE-JSMA: a sparse adversarial attack algorithm for SAR-ATR models

The vulnerability of DNN makes the SAR-ATR system that uses an intelligent algorithm for recognition also somewhat vulnerable. In order to verify the vulnerability, this paper proposes DE-JSMA, a novel sparse adversarial attack algorithm based on a salient map's adversarial attack algorithm and differential evolution algorithm, with the synthetic aperture radar (SAR) image feature sparsity considered. After accurately screening out the salient features that have a great impact on the model inference results, the DE-JSMA algorithm optimizes the appropriate feature values for the salient features. In order to verify its effectiveness more comprehensively, a new metric that combines the attack success rate with the average confidence interval of adversarial examples is proposed. The experimental results show that DE-JSMA extends JSMA, which can be used only for targeted attack scenario, to untargeted attack scenario without increasing too much time consumption but ensuring a high attack success rate, thus achieving sparse adversarial attack with higher reliability and better sparsity in both attack scenarios. The pixel perturbations of only 0.31% and 0.85% can achieve the untargeted and targeted attack success rates up to 100% and 78.79% respectively.

MixCam-attack: Boosting the transferability of adversarial examples with targeted data augmentation

Many black-box adversarial attack algorithms perform attacks on machine learning models based on the transferability of adversarial examples, and the input transformation-based attack is one of the most effective methods. However, existing input transformation methods ignore that each pixel contributes differently to the output of the model, and the focus regions of different models on the same images are similar. Therefore, this paper proposes a targeted data augmentation-based adversarial attack algorithm named MixCam, which augments the input data based on the contribution of each pixel to the prediction result. This is done to enhance the transferability of the crafted adversarial example from the perspective of shifting the regions to which the model pays most of its attention. In addition, this paper proposes further boosting the transferability of the crafted adversarial examples by fusing the class activation maps of multiple models for the input image. Furthermore, the MixCam can integrate other input transformation methods to further boost the transferability of crafted adversarial examples. Extensive experiments on ImageNet demonstrate that MixCam outperforms other state-of-the-art methods in black-box attacks against considered adversarially trained models, with an average increase of 11.7% and 10.7% in attack success rates for single and ensemble attack settings, respectively.

Adversarial attack and training for deep neural network based power quality disturbance classification

Power quality disturbance (PQD) can significantly affect the normal operation of the power system. Deep neural network (DNN) can classify PQD with extremely high accuracy. However, adversarial attacks can drastically degrade the performance of DNN, which raises security issues for DNN-based PQD classification. At present, there are few researches on adversarial attacks and their defenses in PQD classification. In this study, we first adopt convolutional neural network-long short-term memory (CNN-LSTM) to classify PQD signals. Then we propose an adversarial attack algorithm for PQD classification, i.e., momentum iterative-fast gradient sign method (MI-FGSM). MI-FGSM generates adversarial perturbations along the gradient direction of the loss function by incorporating momentum during iterations. Finally, we propose a defense algorithm to defend against adversarial attacks, i.e., iterative adversarial training (IAT). IAT can improve the robustness of the classification model by making it learn to minimize the worst loss caused by multi-class adversarial perturbation strengths. The experimental results demonstrate that compared with the fast gradient sign method, the MI-FGSM produces smaller perturbations and can further reduce the classification accuracy of CNN-LSTM. In addition, compared with the most advanced adversarial training defense algorithm in PQD classification, IAT can effectively enhance the robustness of the CNN-LSTM against adversarial attacks.

Exploring the Physical-World Adversarial Robustness of Vehicle Detection

Adversarial attacks can compromise the robustness of real-world detection models. However, evaluating these models under real-world conditions poses challenges due to resource-intensive experiments. Virtual simulations offer an alternative, but the absence of standardized benchmarks hampers progress. Addressing this, we propose an innovative instant-level data generation pipeline using the CARLA simulator. Through this pipeline, we establish the Discrete and Continuous Instant-level (DCI) dataset, enabling comprehensive experiments involving three detection models and three physical adversarial attacks. Our findings highlight diverse model performances under adversarial conditions. YOLO v6 demonstrates remarkable resilience, exhibiting just a marginal 6.59% average drop in average precision (AP). In contrast, the ASA attack yields a substantial 14.51% average AP reduction, twice the effect of other algorithms. We also note that static scenes yield higher recognition AP values, and outcomes remain relatively consistent across varying weather conditions. Intriguingly, our study suggests that advancements in adversarial attack algorithms may be approaching its “limitation”. In summary, our work underscores the significance of adversarial attacks in real-world contexts and introduces the DCI dataset as a versatile benchmark. Our findings provide valuable insights for enhancing the robustness of detection models and offer guidance for future research endeavors in the realm of adversarial attacks.

Adversarial Attack and Defense in Breast Cancer Deep Learning Systems.

Deep-learning-assisted medical diagnosis has brought revolutionary innovations to medicine. Breast cancer is a great threat to women's health, and deep-learning-assisted diagnosis of breast cancer pathology images can save manpower and improve diagnostic accuracy. However, researchers have found that deep learning systems based on natural images are vulnerable to attacks that can lead to errors in recognition and classification, raising security concerns about deep systems based on medical images. We used the adversarial attack algorithm FGSM to reveal that breast cancer deep learning systems are vulnerable to attacks and thus misclassify breast cancer pathology images. To address this problem, we built a deep learning system for breast cancer pathology image recognition with better defense performance. Accurate diagnosis of medical images is related to the health status of patients. Therefore, it is very important and meaningful to improve the security and reliability of medical deep learning systems before they are actually deployed.

The Study and Implementation on Adversarial Attack Algorithm Based on Deep Convolutional Generative Adversarial Network

The field of machine learning has been growing rapidly in recent years, with significant advancements in computer vision. However, this progress has also led to increased concerns over the safety and security of machine learning systems. Despite previous research efforts in this area, traditional adversarial algorithms continue to fall short in defending against attacks in most circumstances. As a response to this challenge, this study seeks to develop an adversarial algorithm based on Deep Convolutional Generative Adversarial Network (DCGAN), which aims to reconstruct the data distribution of input data and generate new data to improve model robustness. To evaluate the efficacy of the proposed method, a Deep Neural Network (DNN) classifier is employed to perform classification on the generated dataset. The experimental results suggest the potential feasibility of the proposed hypothesis, but further improvement is required to strengthen the defence mechanism. Overall, this study contributes to the ongoing efforts to enhance machine learning safety and security in practical applications.

Rethinking Label Flipping Attack: From Sample Masking to Sample Thresholding.

Nowadays, machine learning (ML) and deep learning (DL) methods have become fundamental building blocks for a wide range of AI applications. The popularity of these methods also makes them widely exposed to malicious attacks, which may cause severe security concerns. To understand the security properties of the ML/DL methods, researchers have recently started to turn their focus to adversarial attack algorithms that could successfully corrupt the model or clean data owned by the victim with imperceptible perturbations. In this paper, we study the Label Flipping Attack (LFA) problem, where the attacker expects to corrupt an ML/DL model's performance by flipping a small fraction of the labels in the training data. Prior art along this direction adopts combinatorial optimization problems, leading to limited scalability toward deep learning models. To this end, we propose a novel minimax problem which provides an efficient reformulation of the sample selection process in LFA. In the new optimization problem, the sample selection operation could be implemented with a single thresholding parameter. This leads to a novel training algorithm called Sample Thresholding. Since the objective function is differentiable and the model complexity does not depend on the sample size, we can apply Sample Thresholding to attack deep learning models. Moreover, since the victim's behavior is not predictable in a poisonous attack setting, we have to employ surrogate models to simulate the true model employed by the victim model. Seeing the problem, we provide a theoretical analysis of such a surrogate paradigm. Specifically, we show that the performance gap between the true model employed by the victim and the surrogate model is small under mild conditions. On top of this paradigm, we extend Sample Thresholding to the crowdsourced ranking task, where labels collected from the annotators are vulnerable to adversarial attacks. Finally, experimental analyses on three real-world datasets speak to the efficacy of our method.

Highly transferable adversarial attack against deep‐reinforcement‐learning‐based frequency control

AbstractWith the increase in inverter‐based renewable energy resources, the complexity and uncertainty of low‐carbon power systems have increased significantly. Deep reinforcement learning (DRL)–based approaches have been extensively studied for frequency control to overcome the limitations of traditional model‐based approaches. The goal of DRL‐based methods for primary frequency control is to minimise load shedding while satisfying frequency safety requirements, thereby reducing control costs. However, the vulnerabilities of DRL models pose new security threats to power systems. These threats have not been identified and addressed in the existing literature. Therefore, in this paper, a series of vulnerability assessment methods are proposed for DRL‐based frequency control with a focus on the under‐frequency load shedding (UFLS) problem. Three adversarial sample production methods are designed with different optimisation directions: Q‐value‐based FGSM (Q‐FGSM), action‐based JSMA (A‐JSMA), and state‐action‐based CW (SA‐CW). Furthermore, combining the advantages of the above three attack methods, a hybrid adversarial attack algorithm is designed, Q‐value‐state‐action‐based mix (QSA‐MIX), to significantly affect the decision process of the DRL model. In case studies of the IEEE39 bus system, the proposed attack methods had a severe impact on system operation and control. In particular, the high attack transferability of the proposed attack algorithms in a black‐box setting provides further evidence that the vulnerability of current DRL‐based control schemes is prevalent.

Efficient text-based evolution algorithm to hard-label adversarial attacks on text

Deep neural networks that play a pivotal role in fields such as images, text, and audio are vulnerable to adversarial attacks. In current textual adversarial attacks, the vast majority are configured with a black-box soft-label which is achieved by the gradient information or confidence of the model. Therefore, it becomes challenging and realistic to implement adversarial attacks using only the predicted top labels of the hard-label model. Existing methods to implement hard-label adversarial attacks use population-based genetic optimization algorithms. However, this approach requires significant query consumption, which is a considerable shortcoming. To solve this problem, we propose a new textual black-box hard-label adversarial attack algorithm based on the idea of differential evolution of populations, called the text-based differential evolution (TDE) algorithm. First, the method will judge the importance of the words of the initial rough adversarial examples, according to which only the keywords in the text sentence will be operated, and the rest of the words will be gradually replaced with the original words so as to reduce the words in the sentence in which the replacement occurs. Our method judges the quality of semantic similarity of the adversarial examples in the replacement process and deposits high-quality adversarial example individuals into the population. Secondly, the optimization process of adversarial examples is combined and optimized according to the word importance. Compared with existing methods based on genetic algorithm guidance, our method avoids a large number of meaningless repetitive queries and significantly improves the overall attack efficiency of the algorithm and the semantic quality of the generated adversarial examples. We experimented with multiple datasets on three text tasks of sentiment classification, natural language inference, and toxic comment, and also perform experimental comparisons on models and APIs in realistic scenarios. For example, in the Google Cloud commercial API adversarial attack experiment, compared to the existing hard-label method, our method reduces the average number of queries required for the attack from 6986 to 176, and increases semantic similarity from 0.844 to 0.876. It is shown through extensive experimental data that our approach not only significantly reduces the number of queries, but also significantly outperforms existing methods in terms of the quality of adversarial examples.

Generating watermarked adversarial texts

Adversarial example generation (AEG) has been a hot spot in recent years because it can cause deep neural networks (DNNs) to misclassify the generated adversarial examples, which reveals the vulnerability of DNNs, motivating us to find good solutions to improve the robustness of DNN models. Due to the extensiveness and high liquidity of natural language over the social networks, various natural language-based adversarial attack algorithms have been proposed in the literature. These algorithms generate adversarial text examples with high semantic quality. However, the generated adversarial text examples and the corresponding attack models may be maliciously or illegally used. To tackle this problem, we present a general framework encapsulated in the cloud application programming interfaces (APIs) for generating watermarked adversarial text examples to protect adversarial text examples and corresponding adversarial text attack models. For each word in a given text, a set of candidate words are determined to ensure that all the words in the set can be used to carry secret bits or facilitate the construction of adversarial example. By applying a word-level adversarial text generation algorithm, the watermarked adversarial text example can be finally generated. Experiment results show that the adversarial text examples generated by the proposed method not only successfully fool advanced DNN models, but also carry watermarks that can effectively verify the ownership and trace the source of the adversarial examples and the corresponding attack models. Moreover, the watermark can still survive after attacked with AEG algorithms, which has shown the applicability and superiority.

ADSAttack: An Adversarial Attack Algorithm via Searching Adversarial Distribution in Latent Space

Deep neural networks are susceptible to interference from deliberately crafted noise, which can lead to incorrect classification results. Existing approaches make less use of latent space information and conduct pixel-domain modification in the input space instead, which increases the computational cost and decreases the transferability. In this work, we propose an effective adversarial distribution searching-driven attack (ADSAttack) algorithm to generate adversarial examples against deep neural networks. ADSAttack introduces an affiliated network to search for potential distributions in image latent space for synthesizing adversarial examples. ADSAttack uses an edge-detection algorithm to locate low-level feature mapping in input space to sketch the minimum effective disturbed area. Experimental results demonstrate that ADSAttack achieves higher transferability, better imperceptible visualization, and faster generation speed compared to traditional algorithms. To generate 1000 adversarial examples, ADSAttack takes 11.08s and, on average, achieves a success rate of 98.01%.

The Threat of Adversarial Attack on a COVID-19 CT Image-Based Deep Learning System

The coronavirus disease 2019 (COVID-19) rapidly spread around the world, and resulted in a global pandemic. Applying artificial intelligence to COVID-19 research can produce very exciting results. However, most research has focused on applying AI techniques in the study of COVID-19, but has ignored the security and reliability of AI systems. In this paper, we explore adversarial attacks on a deep learning system based on COVID-19 CT images with the aim of helping to address this problem. Firstly, we built a deep learning system that could identify COVID-19 CT images and non-COVID-19 CT images with an average accuracy of 76.27%. Secondly, we attacked the pretrained model with an adversarial attack algorithm, i.e., FGSM, to cause the COVID-19 deep learning system to misclassify the CT images, and the classification accuracy of non-COVID-19 CT images dropped from 80% to 0%. Finally, in response to this attack, we proposed how a more secure and reliable deep learning model based on COVID-19 medical images could be built. This research is based on a COVID-19 CT image recognition system, which studies the security of a COVID-19 CT image-based deep learning system. We hope to draw more researchers' attention to the security and reliability of medical deep learning systems.

EIFDAA: Evaluation of an IDS with function-discarding adversarial attacks in the IIoT

The complexity of the Industrial Internet of Things (IIoT) presents higher requirements for intrusion detection systems (IDSs). An adversarial attack is a threat to the security of machine learning-based IDSs. For such a complex situation, this paper analyses adversarial attackers’ ability to deceive IDSs used in the IIoT and proposes the evaluation of an IDS with function-discarding adversarial attacks in the IIoT (EIFDAA), a framework that can evaluate the defence performance of machine learning-based IDSs against various adversarial attack algorithms. This framework is composed of two main processes: adversarial evaluation and adversarial training. Adversarial evaluation can diagnose IDS that is unfitting in adversarial environments. Then, adversarial training is used to treat the weak IDS. In this framework, five well-known adversarial attacks, the fast-gradient sign method (FGSM), basic iterative method (BIM), projected gradient descent (PGD), DeepFool and Wasserstein generative adversarial network with gradient penalty (WGAN-GP) are used to convert attack samples into adversarial samples to simulate the adversarial environment. This study evaluates the capability of mainstream machine learning techniques as intrusion detection models to defend against adversarial attacks, and retrains these detectors to improve the robustness of IDSs through adversarial training. In addition, the framework includes an adversarial attack model that discards the attack function of the attack samples in the IIoT. Through the experimental results on the X-IIoTID dataset, the dropped adversarial detection rate of these detectors to nearly zero demonstrates that an adversarial attack has black-box attack capabilities for these IDSs. Additionally, the improved IDSs retrained with adversarial samples can effectively defend against adversarial attackers while maintaining the original detection rate for the attack samples. EIFDAA is expected to be a solution that can be applied to IDS for improving the robustness in the IIoT.

Plant Disease Classification and Adversarial Attack Using SimAM-EfficientNet and GP-MI-FGSM

Plant diseases have received common attention, and deep learning has also been applied to plant diseases. Deep neural networks (DNNs) have achieved outstanding results in plant diseases. Furthermore, DNNs are very fragile, and adversarial attacks in image classification deserve much attention. It is important to detect the robustness of DNNs through adversarial attacks. The paper firstly improves the EfficientNet by adding the SimAM attention module. The SimAM-EfficientNet is proposed in this paper. The experimental results show that the accuracy of the improved model on PlantVillage reaches 99.31%. The accuracy of ResNet50 is 98.33%. The accuracy of ResNet18 is 98.31%. The accuracy of DenseNet is 98.90%. In addition, the GP-MI-FGSM adversarial attack algorithm improved by gamma correction and image pyramid in this paper can increase the success rate of attack. The model proposed in this paper has an error rate of 87.6% whenattacked by the GP-MI-FGSM adversarial attack algorithm. The success rate of GP-MI-FGSM proposed in this paper is higher than other adversarial attack algorithms, including FGSM, I-FGSM, and MI-FGSM.

Research on a Method of Defense Adversarial Samples for Target Detection Model of Driverless Cars

The adversarial examples make the object detection model make a wrong judgment, which threatens the security of driverless cars. In this paper, by improving the Momentum Iterative Fast Gradient Sign Method (MI-FGSM), based on ensemble learning, combined with L∞ perturbation and spatial transformation, a strong transferable black-box adversarial attack algorithm for object detection model of driverless cars is proposed. Through a large number of experiments on the nuScenes driverless dataset, it is proved that the adversarial attack algorithm proposed in this paper have strong transferability, and successfully make the mainstream object detection models such as FasterRcnn, SSD, YOLOv3 make wrong judgments. Based on the adversarial attack algorithm proposed in this paper, the parametric noise injection with adversarial training is performed to generate a defense model with strong robustness. The defense model proposed in this paper significantly improves the robustness of the object detection model. It can effectively alleviate various adversarial attacks against the object detection model of driverless cars, and does not affect the accuracy of clean samples. This is of great significance for studying the application of object detection model of driverless cars in the real physical world.

Plant Disease Classification and Adversarial Attack based CL-CondenseNetV2 and WT-MI-FGSM

In recent years, deep learning has been increasingly used to the detection of pests and diseases. Unfortunately, deep neural networks are particularly vulnerable when attacked by adversarial examples. Hence it is vital to explore the creation of intensely aggressive adversarial examples to increase neural network robustness. This paper proposes a wavelet transform and histogram equalization-based adversarial attack algorithm: WT-MI-FGSM. In order to verify the performance of the WT-MI-FGSM, we propose a plant pests and diseases identification method based on the coordinate attention mechanism and CondenseNetV2: CL-CondenseNetV2. The accuracy of CL- CondenseNetV2 on the PlantVillage dataset is 99.45%, which indicates that the improved CondenseNetV2 model has a more significant classification performance. In adversarial sample experiments using WT-MI-FGSM and CL-CondenseNetV2, experimental results show that when CL-CondenseNetV2 is attacked by the adversarial algorithm WT-MI-FGSM, the error rate reaches 89.8%, with a higher attack success rate than existing adversarial attack algorithms. In addition, the accuracy of CL-CondenseNetV2 is improved to 99.71% by adding the adversarial samples generated by WT-MI-FGSM to the training set and performing adversarial training. The experiments demonstrate that the adversarial examples caused by WT-MI-FGSM can improve the model's performance.