Adversarial Attack Methods Research Articles

Attacking Click-through Rate Predictors via Generating Realistic Fake Samples

How to construct imperceptible (realistic) fake samples is critical in adversarial attacks. Due to the sample feature diversity of a recommender system (containing both discrete and continuous features), traditional gradient-based adversarial attack methods may fail to construct realistic fake samples. Meanwhile, most recommendation models adopt click-through rate (CTR) predictors, which usually utilize black-box deep models with discrete features as input. Thus, how to efficiently construct realistic fake samples for black-box recommender systems is still full of challenges. In this article, we propose a hierarchical adversarial attack method against black-box CTR models via generating realistic fake samples, named CTRAttack. To better train the generation network, the weights of its embedding layer are shared with those of the substitute model, with both the similarity loss and classification loss used to update the generation network. To ensure that the discrete features of the generated fake samples are all real, we first adopt the similarity loss to ensure that the distribution of the generated perturbed samples is sufficiently close to the distribution of the real features, and then the nearest neighbor algorithm is used to retrieve the most appropriate features for non-existent discrete features from the candidate instance set. Extensive experiments demonstrate that CTRAttack can not only effectively attack the black-box recommender systems but also improve the robustness of these models while maintaining prediction accuracy.

Adversarial patch-based false positive creation attacks against aerial imagery object detectors

Although adversarial attacks have revealed weaknesses in Deep Neural Networks (DNNs)-based aerial detectors, they present a new paradigm for concealing vulnerable assets from autonomous detection systems onboard satellites. Among them, adversarial patches were widely applied due to their physical realizability. Nonetheless, most existing adversarial patch-based attack methods are developed to hide objects from detectors to achieve a vanishing attack. This paper proposes a novel Adversarial Patch False Positive Creation Attack (APFP-CA) framework that enables aerial detectors to recognize non-existent objects, thereby realizing a creation attack. Concretely, the APFP-CA models the creation attack as a two-stage optimization problem. The first stage uses a well-designed efficient loss function to increase the objectness score of patches placed anywhere in the scene to achieve untargeted attacks. The second stage involves elaborating two category-dependent loss functions for fulfilling targeted attacks. Experiments conducted on diverse datasets and detectors demonstrate the effectiveness and universality of our method. Transfer attacks across different datasets and models validate the generalizability of the optimized adversarial patches. Finally, we perform several proportionally scaled experiments physically to illustrate that the optimized adversarial patches can successfully deceive aerial detectors in the physical world. The proposed approach offers a novel contribution to adversarial attacks against aerial imagery objectors and holds potential for practical applications in high-security systems.

DeepInsight: Topology Changes Assisting Detection of Adversarial Samples on Graphs

With the rapid development of artificial intelligence, a number of machine learning algorithms, such as graph neural networks (GNNs), have been proposed to facilitate network analysis or graph data mining. Although effective, recent studies show that these advanced methods may suffer from adversarial attacks, i.e., they may lose effectiveness when only a small fraction of links are unexpectedly changed. This article investigates three well-known adversarial attack methods, i.e., Nettack, Meta Attack, and GradArgmax. It is found that different attack methods have their specific attack preferences on changing the target network structures. Such attack patterns are further verified by experimental results on some real-world networks, revealing that, generally, the top-4 most important network attributes on detecting adversarial samples suffice to explain the preference of an attack method. Based on these findings, the network attributes are utilized to design machine learning models for adversarial sample detection and attack method recognition with outstanding performance.

A Word-Level Adversarial Attack Method Based on Sememes and an Improved Quantum-Behaved Particle Swarm Optimization.

The goal of textual adversarial attack methods is to replace some words in an input text in order to make the victim model misbehave. This article proposes an effective word-level adversarial attack method based on sememes and an improved quantum-behaved particle swarm optimization (QPSO) algorithm. The sememe-based substitute method, which uses the words sharing the same sememes as the substitutes of the original words, is first employed to form the reduced search space. Then, an improved QPSO algorithm, called historical information-guided QPSO with random drift local attractor (HIQPSO-RD), is proposed to search the reduced search space for adversarial examples. The HIQPSO-RD introduces historical information into the current mean best position of the QPSO, for the purpose of improving the convergence speed of the algorithm, by enhancing its exploration ability and preventing the premature convergence of the swarm. The proposed algorithm uses the random drift local attractor technique to make a good balance between its exploration and exploitation, so that the algorithm can find a better adversarial attack example with low grammaticality and perplexity (PPL). In addition, it employs a two-stage diversity control strategy to enhance the search performance of the algorithm. Experiments on three natural language processing (NLP) datasets, with three commonly used nature language processing models as victim models, show that our method achieves higher attack success rates but lower modification rates than the state-of-the-art adversarial attack methods. Moreover, the results of human evaluations show that adversarial examples generated by our method can better maintain the semantic similarity and grammatical correctness of the original input.

Optimizing Latent Variables in Integrating Transfer and Query Based Attack Framework.

Black-box adversarial attacks can be categorized into transfer-based and query-based attacks. The former usually has poor transfer performance due to the mismatch between the architectures of models, while the query-based attacks require massive queries and high dimensional optimization variables. In order to solve the above problems, we propose a novel attack framework integrating the advantages of transfer- and query-based attacks, where the framework is divided into two phases: training the adversarial generator and executing the black-box attacks. In the first stage, a generator is trained by the adversarial loss function so that it can output adversarial perturbation, where the latent variables are designed as the input of the generator to reduce the dimension of the optimization variables. In the second stage, based on the trained generator, we further employ a particle swarm optimization algorithm to optimize the latent variables so that the generator can output the perturbation that can achieve a successful attack. Extensive experiments are performed on the ImageNet dataset, and the results demonstrate that the proposed framework can obtain better attack performance compared with a number of the state-of-the-art black-box adversarial attack methods. In addition, we show the flexibility of the proposed framework by extending the experiment for few-pixel attacks.

Robust object detection for autonomous driving based on semi-supervised learning

Deep learning based on labeled data has brought massive success in computer vision, speech recognition, and natural language processing. Nevertheless, labeled data is just a drop in the ocean compared with unlabeled data. How can people utilize the unlabeled data effectively? Research has focused on unsupervised and semi-supervised learning to solve such a problem. Some theoretical and empirical studies have proved that unlabeled data can help boost the generalization ability and robustness under adversarial attacks. However, current theoretical research on the relationship between robustness and unlabeled data limits its scope to toy datasets. Meanwhile, the visual models in autonomous driving need a significant improvement in robustness to guarantee security and safety. This paper proposes a semi-supervised learning framework for object detection in autonomous vehicles, improving the robustness with unlabeled data. Firstly, we build a baseline with the transfer learning of an unsupervised contrastive learning method—Momentum Contrast (MoCo). Secondly, we propose a semi-supervised co-training method to label the unlabeled data for retraining, which improves generalization on the autonomous driving dataset. Thirdly, we apply the unsupervised Bounding Box data augmentation (BBAug) method based on a search algorithm, which uses reinforcement learning to improve the robustness of object detection for autonomous driving. We present an empirical study on the KITTI dataset with diverse adversarial attack methods. Our proposed method realizes the state-of-the-art generalization and robustness under white-box attacks (DPatch and Contextual Patch) and black-box attacks (Gaussian noise, Rain, Fog, and so on). Our proposed method and empirical study show that using more unlabeled data benefits the robustness of perception systems in autonomous driving.

Physical adversarial attack in artificial intelligence of things

AbstractWith the continuous development of wireless communication and artificial intelligence technology, Internet of Things (IoT) technology has made great progress. Deep learning methods are currently used in IoT technology, but deep neural networks (DNNs) are notoriously susceptible to adversarial examples, and subtle pixel changes to images can result in incorrect recognition results from DNNs. In the real‐world application, the patches generated by the recent physical attack methods are larger or less realistic and easily detectable. To address this problem, a Generative Adversarial Network based on Visual attention model and Style transfer network (GAN‐VS) is proposed, which reduces the patch area and makes the patch more natural and less noticeable. A visual attention model combined with generative adversarial network is introduced to detect the critical regions of image recognition, and only generate patches within the critical regions to reduce patch area and improve attack efficiency. For any type of seed patch, an adversarial patch can be generated with a high degree of stylistic and content similarity to the attacked image by generative adversarial network and style transfer network. Experimental evaluation shows that the proposed GAN‐VS has good camouflage and outperforms state‐of‐the‐art adversarial patch attack methods.

Exploring Adversarial Robustness of LiDAR Semantic Segmentation in Autonomous Driving

Deep learning networks have demonstrated outstanding performance in 2D and 3D vision tasks. However, recent research demonstrated that these networks result in failures when imperceptible perturbations are added to the input known as adversarial attacks. This phenomenon has recently received increased interest in the field of autonomous vehicles and has been extensively researched on 2D image-based perception tasks and 3D object detection. However, the adversarial robustness of 3D LiDAR semantic segmentation in autonomous vehicles is a relatively unexplored topic. This study expands the adversarial examples to LiDAR-based 3D semantic segmentation. We developed and analyzed three LiDAR point-based adversarial attack methods on different networks developed on the SemanticKITTI dataset. The findings illustrate that the Cylinder3D network has the highest adversarial susceptibility to the analyzed attacks. We investigated how the class-wise point distribution influences the adversarial robustness of each class in the SemanticKITTI dataset and discovered that ground-level points are extremely vulnerable to point perturbation attacks. Further, the transferability of each attack strategy was assessed, and we found that networks relying on point data representation demonstrate a notable level of resistance. Our findings will enable future research in developing more complex and specific adversarial attacks against LiDAR segmentation and countermeasures against adversarial attacks.

Adversarial attack driven data augmentation for medical images

An important stage in medical image analysis is segmentation, which aids in focusing on the required area of an image and speeds up findings. Fortunately, deep learning models have taken over with their high-performing capabilities, making this process simpler. The deep learning model’s reliance on vast data, however, makes it difficult to utilize for medical image analysis due to the scarcity of data samples. Too far, a number of data augmentations techniques have been employed to address the issue of data unavailability. Here, we present a novel method of augmentation that enabled the UNet model to segment the input dataset with about 90% accuracy in just 30 epochs. We describe the us- age of fast gradient sign method (FGSM) as an augmentation tool for adversarial machine learning attack methods. Besides, we have developed the method of Inverse FGSM, which im- proves performance by operating in the opposite way from FGSM adversarial attacks. In comparison to the conventional FGSM methodology, our strategy boosted performance up to 6% to 7% on average. The model became more resilient to hostile attacks because to these two strategies. An innovative implementation of adversarial machine learning and resilience augmentation is revealed by the overall analysis of this study.

Federated Adversarial Training Strategies for Achieving Privacy and Security in Sustainable Smart City Applications

Smart city applications that request sensitive user information necessitate a comprehensive data privacy solution. Federated learning (FL), also known as privacy by design, is a new paradigm in machine learning (ML). However, FL models are susceptible to adversarial attacks, similar to other AI models. In this paper, we propose federated adversarial training (FAT) strategies to generate robust global models that are resistant to adversarial attacks. We apply two adversarial attack methods, projected gradient descent (PGD) and the fast gradient sign method (FGSM), to our air pollution dataset to generate adversarial samples. We then evaluate the effectiveness of our FAT strategies in defending against these attacks. Our experiments show that FGSM-based adversarial attacks have a negligible impact on the accuracy of global models, while PGD-based attacks are more effective. However, we also show that our FAT strategies can make global models robust enough to withstand even PGD-based attacks. For example, the accuracy of our FAT-PGD and FL-mixed-PGD models is 81.13% and 82.60%, respectively, compared to 91.34% for the baseline FL model. This represents a reduction in accuracy of 10%, but this could be potentially mitigated by using a more complex and larger model. Our results demonstrate that FAT can enhance the security and privacy of sustainable smart city applications. We also show that it is possible to train robust global models from modest datasets per client, which challenges the conventional wisdom that adversarial training requires massive datasets.

Adversarial Attacks in Underwater Acoustic Target Recognition with Deep Learning Models

Deep learning models can produce unstable results by introducing imperceptible perturbations that are difficult for humans to recognize. This can have a significant impact on the accuracy and security of deep learning applications due to their poorly understood interpretability. As a field critical to security research, this problem clearly exists in underwater acoustic target recognition for ocean sensing. To address this issue, this article investigates the reliability of state-of-the-art deep learning models by exploring adversarial attack methods that add small, exquisite perturbations on acoustic Mel-spectrograms to generate adversarial spectrograms. Experimental results based on real-world datasets reveal that these models can be forced to learn unexpected features when subjected to adversarial spectrograms, resulting in significant accuracy drops. Specifically, when employing the iterative attack method, the overall accuracy of all models experiences a significant decrease of approximately 70% for two datasets under stronger perturbations.

Towards evaluating the robustness of deep neural semantic segmentation networks with Feature-Guided Method

Although deep neural networks (DNNs) demonstrated its superior performance in computer vision, recent works proved that the vulnerability of deep neural task systems to carefully crafted human-imperceptible perturbations. We observe that the majority of adversarial attack methods for generating perturbations are based on rotation or modification of the input-diagnostic. Regardless of the characteristics completed by additional sources, such approaches yield comparable properties to enrich initial details. It motivates us to consider views about using various information apart from original inputs to deliver adversarial features. For such needs, we induce a simple yet adaptable adversarial attack strategy with a Feature-Guided Method (FGM) for crafting adversarial examples (AEs) in segmentation domain. FGM first induces the multi-source patterns that are apart from the original inputs. Then, producing feature diversities generated with the original data to deliver the perturbed components. Finally, FGM blends original inputs with created features in a defined norm constraint to form the adversarial examples. In such way, it preserves the original positive class-general characteristics and enriches the new positive class-specific diversities when performing adversarial attacks. Moreover, FGM employs the adaptive gradient-based strategy on such generated information, which lowers the risk of falling into the local optimum when searching for the decision boundary of the source and target models in latent space. We conduct detailed experiments to evaluate the performance of the proposed method compared to baselines on public segmentation models. The experimental results reveal better performance of FGMs in fooling source and target segmentation systems leading large margins over 5% on mIoU, mRec, and mAcc. We also deploy the adversarial training with proposed work and PGD on widely used models. Our approach improves the robust quality of adversarially trained models on FCN, PSPNet, and DeepLabv3 with various backbones by significant margins with over 13% improvement on mIoU and 12% on mRec, which indicates a better impact of deploying such mechanisms on robust deep neural segmentation models.

Robustness-Eva-MRC: Assessing and analyzing the robustness of neural models in extractive machine reading comprehension

Deep neural networks, despite their remarkable success in various language understanding tasks, have been found vulnerable to adversarial attacks and subtle input perturbations, revealing a robustness shortfall. To explore this, this paper presents Robustness-Eva-MRC, an interactive platform designed to assess and analyze the robustness of pre-trained and large-scale language models in extractive machine reading comprehension (MRC) tasks. The platform integrates eight adversarial attack methods across character-, word-, and sentence-levels, and applies them to five MRC datasets, thereby fabricating challenging adversarial testing sets. Then it evaluates the MRC models on both original and adversarial sets, yielding insights into their robustness through performance gaps. Moreover, Robustness-Eva-MRC provides comprehensive visualizations and detailed case studies, enhancing the understanding of model robustness. A screencast video and additional material are available at https://github.com/distantJing/Robustness-Eva-MRC.

FeConDefense: Reversing adversarial attacks via feature consistency loss

Existing adversarial defense methods often employ adversarial training or data pre-processing techniques to defend against adversarial attacks. However, adversarial training is burdensome as it requires to find a single representation that works for all attack possibilities, and may decrease the network model’s classification accuracy after excessive training. While data pre-processing methods focus on eliminating adversarial perturbations by modifying the input samples, they do not consider the internal relationships between reverse perturbations and adversarial examples, resulting in weak specificity of the generated modifications. In this paper, we propose a novel adversarial defense method, named FeConDefense, which aims to reverse adversarial attacks via analyzing the intrinsic features of images. Specifically, we first extract two different features of adversarial examples by respectively using two different network models. Then, we design a novel feature consistency loss to measure the distance between these two features. Finally, we integrate the feature consistency into the contrastive learning to generate reverse perturbation for each adversarial example. Comprehensive experiments on different adversarial attack methods demonstrate that our FeConDefense achieves state-of-the-art results in reversing adversarial perturbations and improving robustness of image classifiers.

Attention-Enhanced One-Shot Attack against Single Object Tracking for Unmanned Aerial Vehicle Remote Sensing Images

Recent studies have shown that deep-learning-based models for processing Unmanned Aerial Vehicle (UAV) remote sensing images are vulnerable to artificially designed adversarial examples, which can lead to incorrect predictions of deep models when facing adversarial examples. Previous adversarial attack methods have mainly focused on the classification and detection of UAV remote sensing images, and there is still a lack of research on adversarial attacks for object tracking in UAV video. To address this challenge, we propose an attention-enhanced one-shot adversarial attack method for UAV remote sensing object tracking, which perturbs only the template frame and generates adversarial samples offline. First, we employ an attention feature loss to make the original frame’s features dissimilar to those of the adversarial frame, and an attention confidence loss to either suppress or enhance different confidence scores. Additionally, by forcing the tracker to concentrate on the background information near the target, a background distraction loss is used to mismatch templates with subsequent frames. Finally, we add total variation loss to generate adversarial examples that appear natural to humans. We validate the effectiveness of our method against popular trackers such as SiamRPN, DaSiamRPN, and SiamRPN++ on the UAV123 remote sensing dataset. Experimental results verify the superior attack performance of our proposed method.

Face Detection Using a Capsule Network for Driver Monitoring Application

Bus driver distraction and cognitive load lead to higher accident risk. Driver distraction sources and complex physical and psychological effects must be recognized and analyzed in real-world driving conditions to reduce risk and enhance overall road safety. The implementation of a camera-based system utilizing computer vision for face recognition emerges as a highly viable and effective driver monitoring approach applicable in public transport. Reliable, accurate, and unnoticeable software solutions need to be developed to reach the appropriate robustness of the system. The reliability of data recording depends mainly on external factors, such as vibration, camera lens contamination, lighting conditions, and other optical performance degradations. The current study introduces Capsule Networks (CapsNets) for image processing and face detection tasks. The authors’ goal is to create a fast and accurate system compared to state-of-the-art Neural Network (NN) algorithms. Based on the seven tests completed, the authors’ solution outperformed the other networks in terms of performance degradation in six out of seven cases. The results show that the applied capsule-based solution performs well, and the degradation in efficiency is noticeably smaller than for the presented convolutional neural networks when adversarial attack methods are used. From an application standpoint, ensuring the security and effectiveness of an image-based driver monitoring system relies heavily on the mitigation of disruptive occurrences, commonly referred to as “image distractions,” which represent attacks on the neural network.

Imperceptible graph injection attack on graph neural networks

In recent years, Graph Neural Networks (GNNs) have achieved excellent applications in classification or prediction tasks. Recent studies have demonstrated that GNNs are vulnerable to adversarial attacks. Graph Modification Attack (GMA) and Graph Injection Attack (GIA) are commonly attack strategies. Most graph adversarial attack methods are based on GMA, which has a clear drawback: the attacker needs high privileges to modify the original graph, making it difficult to execute in practice. GIA can perform attacks without modifying the original graph. However, many GIA models fail to take care of attack invisibility, i.e., fake nodes can be easily distinguished from the original nodes. To solve the above issue, we propose an imperceptible graph injection attack, named IMGIA. Specifically, IMGIA uses the normal distribution sampling and mask learning to generate fake node features and links respectively, and then uses the homophily unnoticeability constraint to improve the camouflage of the attack. Our extensive experiments on three benchmark datasets demonstrate that IMGIA performs better than the existing state-of-the-art GIA methods. As an example, IMGIA shows an improvement in performance with an average increase in effectiveness of 2%.

Adaptive local adversarial attacks on 3D point clouds

Modern artificial intelligence systems rely heavily on deep learning techniques. However, deep neural networks are easily disturbed by adversarial objects. Adversarial examples are beneficial to improve the robustness of the 3D neural network model and enhance the stability of the artificial intelligence system. At present, most 3D adversarial attack methods perturb the entire point cloud to generate adversarial examples, which results in high perturbation costs and low operability in the physical world. In this paper, we propose an adaptive local adversarial attack method (AL-Adv) on 3D point clouds to generate adversarial point clouds. First, we analyze the vulnerability of the 3D network model and extract the salient regions of the input point cloud, namely the vulnerable regions. Second, we propose an adaptive gradient attack algorithm that targets salient regions. The proposed attack algorithm adaptively assigns different disturbances in different directions of the three-dimensional coordinates of the point cloud. Experimental results show that our proposed AL-Adv method achieves a higher attack success rate than the global attack method. Specifically, the adversarial examples generated by AL-Adv demonstrate good imperceptibility and small generation costs.

Frequency constraint-based adversarial attack on deep neural networks for medical image classification

The security of AI systems has gained significant attention in recent years, particularly in the medical diagnosis field. To develop a secure medical image classification system based on deep neural networks, it is crucial to design effective adversarial attacks that can embed hidden, malicious behaviors into the system. However, designing a unified attack method that can generate imperceptible attack samples with high content similarity and be applied to diverse medical image classification systems is challenging due to the diversity of medical imaging modalities and dimensionalities. Most existing attack methods are designed to attack natural image classification models, which inevitably corrupt the semantics of pixels by applying spatial perturbations. To address this issue, we propose a novel frequency constraint-based adversarial attack method capable of delivering attacks in various medical image classification tasks. Specially, our method introduces a frequency constraint to inject perturbation into high-frequency information while preserving low-frequency information to ensure content similarity. Our experiments include four public medical image datasets, including a 3D CT dataset, a 2D chest X-Ray image dataset, a 2D breast ultrasound dataset, and a 2D thyroid ultrasound dataset, which contain different imaging modalities and dimensionalities. The results demonstrate the superior performance of our model over other state-of-the-art adversarial attack methods for attacking medical image classification tasks on different imaging modalities and dimensionalities.

Boosting Adversarial Transferability with Shallow-Feature Attack on SAR Images

Adversarial example generation on Synthetic Aperture Radar (SAR) images is an important research area that could have significant impacts on security and environmental monitoring. However, most current adversarial attack methods on SAR images are designed for white-box situations by end-to-end means, which are often difficult to achieve in real-world situations. This article proposes a novel black-box targeted attack method, called Shallow-Feature Attack (SFA). Specifically, SFA assumes that the shallow features of the model are more capable of reflecting spatial and semantic information such as target contours and textures in the image. The proposed SFA generates ghost data packages for input images and generates critical features by extracting gradients and feature maps at shallow layers of the model. The feature-level loss is then constructed using the critical features from both clean images and target images, which is combined with the end-to-end loss to form a hybrid loss function. By fitting the critical features of the input image at specific shallow layers of the neural network to the target critical features, our attack method generates more powerful and transferable adversarial examples. Experimental results show that the adversarial examples generated by the SFA attack method improved the success rate of single-model attack under a black-box scenario by an average of 3.73%, and 4.61% after combining them with ensemble-model attack without victim models.