Transferable Adversarial Examples Research Articles

Boosting the Transferability of Ensemble Adversarial Attack via Stochastic Average Variance Descent

Adversarial examples have the property of transferring across models, which has created a great threat for deep learning models. To reveal the shortcomings in the existing deep learning models, the method of the ensemble has been introduced to the generating of transferable adversarial examples. However, most of the model ensemble attacks directly combine the different models’ output but ignore the large differences in optimization direction of them, which severely limits the transfer attack ability. In this work, we propose a new kind of ensemble attack method called stochastic average ensemble attack. Unlike the existing approach of averaging the outputs of each model as an integrated output, we continuously optimize the ensemble gradient in an internal loop using the model history gradient and the average gradient of different models. In this way, the adversarial examples can be updated in a more appropriate direction and make the crafted adversarial examples more transferable. Experimental results on ImageNet show that our method generates highly transferable adversarial examples and outperforms existing methods.

Improving the transferability of adversarial examples through neighborhood attribution

Adversarial examples, which add carefully planned perturbations to images, pose a serious threat to neural network applications. Transferable adversarial attacks, in which adversarial examples generated on the source model can successfully attack the target model, provide a realistic and undetectable method. Existing transfer-based attacks tend to improve the transferability of adversarial examples by destroying their intrinsic features. They destabilized features differentially by assessing their importance, thus rendering the model incapable of inference. However, the existing methods generate feature-importance assessments that are overly dependent on the source model, leading to inaccurate importance guidance and insufficient feature destruction. In this paper, we propose neighborhood expectancy attribution attacks (NEAA) that accurately guide the destruction of deep features, leading to highly transferable adversarial examples. First, we design a highly versatile attribution tool called neighborhood attribution to represent the importance of features that attribute highly similar results to various source models. Specifically, we discard the imputation of a single baseline and adopt the imputed expectation of a baseline within the neighborhood of the image. Subsequently, we generalize the neighborhood attribution to the middle layer of the model and simplify the computation by assuming linear independence. Finally, the attribution result guides the attack to destroy the intrinsic features of the image and obtain highly transferable adversarial examples. Numerous experiments demonstrate the effectiveness of the proposed method. Code is available at Github: https://github.com/KWPCCC/NEAA.

A Theoretical View of Linear Backpropagation and its Convergence.

Backpropagation (BP) is widely used for calculating gradients in deep neural networks (DNNs). Applied often along with stochastic gradient descent (SGD) or its variants, BP is considered as a de-facto choice in a variety of machine learning tasks including DNN training and adversarial attack/defense. Recently, a linear variant of BP named LinBP was introduced for generating more transferable adversarial examples for performing black-box attacks, by (Guo et al. 2020). Although it has been shown empirically effective in black-box attacks, theoretical studies and convergence analyses of such a method is lacking. This paper serves as a complement and somewhat an extension to Guo et al. (2020) paper, by providing theoretical analyses on LinBP in neural-network-involved learning tasks, including adversarial attack and model training. We demonstrate that, somewhat surprisingly, LinBP can lead to faster convergence in these tasks in the same hyper-parameter settings, compared to BP. We confirm our theoretical results with extensive experiments.

CFOA: Exploring transferable adversarial examples by content feature optimization with attention guidance

Deep neural networks are known to be highly susceptible to adversarial examples, created by adding small and carefully-designed perturbations on original images. In black-box scenarios, attacking neural networks is challenging since the information about models is not available. Generating transferable adversarial examples on surrogate models is a viable way. However, current mainstream transfer-based attacks seldomly consider the relationship between transferability and content features of the image. To effectively enhance the attack success rate at the feature level, we newly propose an attack method called Content Feature Optimization Attack (CFOA). This novel CFOA approach operates on white-box models, aiming to generate transferable adversarial examples by targeting the content feature representations of images in the feature space. The method involves the extraction and parameterization of content features, followed by the iterative generation of feature-level perturbations guided by attention loss and adversarial loss. Ultimately, the adversarial examples generated by CFOA exhibit strong transferability on other black-box models. Experimental results show that our method produces adversarial examples with higher success rates in transferability compared to state-of-the-art methods, and importantly, they can effectively evade certain defense mechanisms, presenting a challenge for adversarial defense.

Towards transferable adversarial attacks on vision transformers for image classification

The deployment of high-performance Vision Transformer (ViT) models has garnered attention from both industry and academia. However, their vulnerability to adversarial examples highlights security risks for scenarios such as intelligent surveillance, autonomous driving, and fintech regulation. As a black-box attack technique, transfer attacks leverage a surrogate model to generate transferable adversarial examples to attack a target victim model, which mainly focuses on a forward (input diversification) and a backward (gradient modification) approach. However, both approaches are currently implemented straightforwardly and limit the transferability of surrogate models. In this paper, we propose a Forward-Backward Transferable Adversarial Attack framework (FBTA) that can generate highly transferable adversarial examples against different models by fully leveraging ViT’s distinctive intermediate layer structures. In the forward inference process of FBTA, we propose a Dropout-based Transferable Attack (DTA) approach to diversify the intermediate states of ViT models, simulating an ensemble learning effect; in the backward process, a Backpropagation Gradient Clipping (BGC) method is designed to refine the gradients within intermediate layers of ViT models intricately. Extensive experiments on state-of-the-art ViTs and robust CNNs demonstrate that our FBTA framework achieves an average performance improvement of 2.79% compared to state-of-the-art transfer-based attacks, offering insights for the comprehension and defense against transfer attacks.

Cascade & allocate: A cross-structure adversarial attack against models fusing vision and language

The data fusion systems between multiple modals have attracted a wide range of concerns about their adversarial robustness. Recent image captioning attacks lack cross-structure transferability against models with diverse structures. Therefore, the attackers demonstrate satisfactory performance only when they gain full or partial knowledge of target image captioning models. So far, the cross-structure transferability of multi-modal adversarial examples has not been thoroughly investigated. In this paper, a theorem is proposed to analyze the upper bound captioning adversarial transferability. We design a new transfer-based adversarial attack method (Cascade & Allocate) against image captioning models. Our method can be divided into two steps. The first step randomly selects a set of candidate models with diverse structures, and we use a momentum-like strategy to generate perturbations. This ‘model cascade’ step narrows the gap of the gradient directions between different image captioning models, therefore enhancing the cross-model transferability and generating model-agnostic adversarial perturbations. The second step utilizes the upper bound transferability theorem to allocate the weight of perturbations per model. This ‘weight allocate’ step enhances the weight of model with the most consistent gradient in all candidate models, which generates more transferable adversarial examples. In the experiments, we compare our approach with other captioning and ensemble-based attacks against five black-box models with different structures on MS COCO and Flickr-30k datasets. The adversarial examples exhibit state-of-the-art adversarial transferability against five black-box models with different structures. The results demonstrate that our approach is practical and generalizes well to a wide range of captioning models.

LRS: Enhancing Adversarial Transferability through Lipschitz Regularized Surrogate

The transferability of adversarial examples is of central importance to transfer-based black-box adversarial attacks. Previous works for generating transferable adversarial examples focus on attacking given pretrained surrogate models while the connections between surrogate models and adversarial trasferability have been overlooked. In this paper, we propose Lipschitz Regularized Surrogate (LRS) for transfer-based black-box attacks, a novel approach that transforms surrogate models towards favorable adversarial transferability. Using such transformed surrogate models, any existing transfer-based black-box attack can run without any change, yet achieving much better performance. Specifically, we impose Lipschitz regularization on the loss landscape of surrogate models to enable a smoother and more controlled optimization process for generating more transferable adversarial examples. In addition, this paper also sheds light on the connection between the inner properties of surrogate models and adversarial transferability, where three factors are identified: smaller local Lipschitz constant, smoother loss landscape, and stronger adversarial robustness. We evaluate our proposed LRS approach by attacking state-of-the-art standard deep neural networks and defense models. The results demonstrate significant improvement on the attack success rates and transferability. Our code is available at https://github.com/TrustAIoT/LRS.

Enhancing adversarial attacks with resize-invariant and logical ensemble

In black-box scenarios, most transfer-based attacks usually improve the transferability of adversarial examples by optimizing the gradient calculation of the input image. Unfortunately, since the gradient information is only calculated and optimized for each pixel point in the image individually, the generated adversarial examples tend to overfit the local model and have poor transferability to the target model. To tackle the issue, we propose a resize-invariant method (RIM) and a logical ensemble transformation method (LETM) to enhance the transferability of adversarial examples. Specifically, RIM is inspired by the resize-invariant property of Deep Neural Networks (DNNs). The range of resizable pixel is first divided into multiple intervals, and then the input image is randomly resized and padded within each interval. Finally, LETM performs logical ensemble of multiple images after RIM transformation to calculate the final gradient update direction. The proposed method adequately considers the information of each pixel in the image and the surrounding pixels. The probability of duplication of image transformations is minimized and the overfitting effect of adversarial examples is effectively mitigated. Numerous experiments on the ImageNet dataset show that our approach outperforms other advanced methods and is capable of generating more transferable adversarial examples.

Query Attack by Multi-Identity Surrogates

Deep Neural Networks (DNNs) are acknowledged as vulnerable to adversarial attacks, while the existing black-box attacks require extensive queries on the victim DNN to achieve high success rates. For query-efficiency, surrogate models of the victim are used to generate transferable Adversarial Examples (AEs) because of their <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Gradient Similarity (GS)</i> , i.e., surrogates' attack gradients are similar to the victim's ones. However, it is generally neglected to exploit their similarity on outputs, namely the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Prediction Similarity (PS)</i> , to filter out inefficient queries by surrogates without querying the victim. To jointly utilize and also optimize surrogates' GS and PS, we develop QueryNet, a unified attack framework that can significantly reduce queries. QueryNet creatively attacks by multi-identity surrogates, i.e., crafts several AEs for one sample by different surrogates, and also uses surrogates to decide on the most promising AE for the query. After that, the victim's query feedback is accumulated to optimize not only surrogates' parameters but also their architectures, enhancing both the GS and the PS. Although QueryNet has no access to pre-trained surrogates' prior, it reduces queries by averagely about an order of magnitude compared to alternatives within an acceptable time, according to our comprehensive experiments: 11 victims (including two commercial models) on MNIST/CIFAR10/ImageNet, allowing only 8-bit image queries, and no access to the victim's training data. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Sizhe-Chen/QueryNet</uri> .

Crafting Transferable Adversarial Examples Against Face Recognition via Gradient Eroding

In recent years, deep neural networks (DNNs) have made significant progress on face recognition. However, DNNs have been found to be vulnerable to adversarial examples, leading to fatal consequences in real-world applications. This paper focuses on improving the transferability of adversarial examples against face recognition models. We propose Gradient Eroding (GE) to make the gradient of the residual blocks more diverse, by eroding the back-propagation dynamically. We also propose a novel black-box adversarial attack named Corrasion Attack based on GE. Extensive experiments demonstrate that our approach can effectively improve the transferability of adversarial attacks against face recognition models. Our approach over-performs 29.35% in fooling rate than state-of-the-art black-box attacks. Leveraging adversarial training with adversarial examples generated by us, the robustness of models can be improved by up to 43.2%. Besides, Corrasion Attack successfully breaks two online face recognition systems, achieving a highest fooling rate of 89.8%.

UCG: A Universal Cross-Domain Generator for Transferable Adversarial Examples

UCG: A Universal Cross-Domain Generator for Transferable Adversarial Examples

Image classification adversarial attack with improved resizing transformation and ensemble models.

Convolutional neural networks have achieved great success in computer vision, but incorrect predictions would be output when applying intended perturbations on original input. These human-indistinguishable replicas are called adversarial examples, which on this feature can be used to evaluate network robustness and security. White-box attack success rate is considerable, when already knowing network structure and parameters. But in a black-box attack, the adversarial examples success rate is relatively low and the transferability remains to be improved. This article refers to model augmentation which is derived from data augmentation in training generalizable neural networks, and proposes resizing invariance method. The proposed method introduces improved resizing transformation to achieve model augmentation. In addition, ensemble models are used to generate more transferable adversarial examples. Extensive experiments verify the better performance of this method in comparison to other baseline methods including the original model augmentation method, and the black-box attack success rate is improved on both the normal models and defense models.

Strengthening transferability of adversarial examples by adaptive inertia and amplitude spectrum dropout

Deep neural networks are sensitive to adversarial examples and would produce wrong results with high confidence. However, most existing attack methods exhibit weak transferability, especially for adversarially trained models and defense models. In this paper, two methods are proposed to generate highly transferable adversarial examples, namely Adaptive Inertia Iterative Fast Gradient Sign Method (AdaI2-FGSM) and Amplitude Spectrum Dropout Method (ASDM). Specifically, AdaI2-FGSM aims to integrate adaptive inertia into the gradient-based attack, and leverage the looking ahead property to search for a flatter maximum, which is essential to strengthen the transferability of adversarial examples. By introducing a loss-preserving transformation in the frequency domain, the proposed ASDM with the dropout invariance property can craft the copies of input images to overcome the poor generalization on the surrogate models. Furthermore, AdaI2-FGSM and ASDM can be naturally integrated as an efficient gradient-based attack method to yield more transferable adversarial examples. Extensive experimental results on the ImageNet-compatible dataset demonstrate that higher transferability is achieved by our method than some advanced gradient-based attacks.

Crafting transferable adversarial examples via contaminating the salient feature variance

Adversarial attacks play a vital role in the development of deep learning techniques, which can evaluate the robustness of deep neural networks (DNNs) as well as explore their decision mechanism. Recently, feature-level attacks have been proposed to contaminate the internal feature maps of the source model at each iteration, providing a new method to produce transferable adversarial examples. In this paper, we uncover two neglected problems behind current feature-level attacks and propose an ingenious Salient Feature Variance Attack (SFVA). Concretely, we first apply a Combined Feature Enhancement Transformation (CFET) on the copies of clean images to estimate the optimal feature weight. Then we construct an efficient objective based on the variance of salient features and adopt a classical attack MI-FGSM (MI) to add adversarial noises to the clean image along the direction of gradients. Moreover, we also make it possible to combine the ensemble strategy with feature-level attacks. Abundant experiments on the ImageNet dataset forcefully confirm the superiority of SFVA, which has become a state-of-the-art feature-level attack. Furthermore, we also evaluate the robustness of the practical online model with SFVA, where the 90% attack success rate reveals a worrying fact that the real-world deployed models are subject to serious security threats.

Deep Fusion: Crafting Transferable Adversarial Examples and Improving Robustness of Industrial Artificial Intelligence of Things

Industry 5.0 is aimed at merging the cognitive computing capabilities of Deep neural networks (DNNs) with human resourcefulness in collaborative operations. DNNs have been widely used in Industrial Artificial Intelligence of Things (Industrial AIoT) systems. However, DNNs are vulnerable to adversarial attacks, which bring a considerable risk to Industrial Artificial Intelligence of Things systems. The adversary uses adversarial examples crafted on the local ensemble model to attack black-box target of Industrial Artificial Intelligence of Things systems, resulting in catastrophic consequences. It is essential to study ensemble adversarial attack and defense strategies in black-box scenarios. Nevertheless, current ensemble attacks' performance is limited by the diversity of local models and ensemble strategies, and defensive strategies are inefficient. To solve these problems, we propose two novel Deep Fusion methods from both an attacker's and a defender's perspective. For initiating attacks, we propose Deep Fusion Attack. The erosion models are applied to compensate for local models' insufficiency in diversity. We fuse erosion models in the output space and the feature space simultaneously and continuously accumulate historical gradients to retain adversarial information, thereby improving transferability. Extensive experimental results show that our approach achieves superior performance in black-box attacks than state-of-the-art ensemble attacks. The average success rate of our targeted black-box attack reaches a compelling 87.4%. For constructing defenses, we propose Deep Fusion Defense, using a fusion of multiple predictions with erosion models as a novel approach.

Boosting Adversarial Transferability with Shallow-Feature Attack on SAR Images

Adversarial example generation on Synthetic Aperture Radar (SAR) images is an important research area that could have significant impacts on security and environmental monitoring. However, most current adversarial attack methods on SAR images are designed for white-box situations by end-to-end means, which are often difficult to achieve in real-world situations. This article proposes a novel black-box targeted attack method, called Shallow-Feature Attack (SFA). Specifically, SFA assumes that the shallow features of the model are more capable of reflecting spatial and semantic information such as target contours and textures in the image. The proposed SFA generates ghost data packages for input images and generates critical features by extracting gradients and feature maps at shallow layers of the model. The feature-level loss is then constructed using the critical features from both clean images and target images, which is combined with the end-to-end loss to form a hybrid loss function. By fitting the critical features of the input image at specific shallow layers of the neural network to the target critical features, our attack method generates more powerful and transferable adversarial examples. Experimental results show that the adversarial examples generated by the SFA attack method improved the success rate of single-model attack under a black-box scenario by an average of 3.73%, and 4.61% after combining them with ensemble-model attack without victim models.

TAN: A Transferable Adversarial Network for DNN-Based UAV SAR Automatic Target Recognition Models

Recently, the unmanned aerial vehicle (UAV) synthetic aperture radar (SAR) has become a highly sought-after topic for its wide applications in target recognition, detection, and tracking. However, SAR automatic target recognition (ATR) models based on deep neural networks (DNN) are suffering from adversarial examples. Generally, non-cooperators rarely disclose any SAR-ATR model information, making adversarial attacks challenging. To tackle this issue, we propose a novel attack method called Transferable Adversarial Network (TAN). It can craft highly transferable adversarial examples in real time and attack SAR-ATR models without any prior knowledge, which is of great significance for real-world black-box attacks. The proposed method improves the transferability via a two-player game, in which we simultaneously train two encoder–decoder models: a generator that crafts malicious samples through a one-step forward mapping from original data, and an attenuator that weakens the effectiveness of malicious samples by capturing the most harmful deformations. Particularly, compared to traditional iterative methods, the encoder–decoder model can one-step map original samples to adversarial examples, thus enabling real-time attacks. Experimental results indicate that our approach achieves state-of-the-art transferability with acceptable adversarial perturbations and minimum time costs compared to existing attack methods, making real-time black-box attacks without any prior knowledge a reality.

Erosion Attack: Harnessing Corruption To Improve Adversarial Examples.

Although adversarial examples pose a serious threat to deep neural networks, most transferable adversarial attacks are ineffective against black-box defense models. This may lead to the mistaken belief that adversarial examples are not truly threatening. In this paper, we propose a novel transferable attack that can defeat a wide range of black-box defenses and highlight their security limitations. We identify two intrinsic reasons why current attacks may fail, namely data-dependency and network-overfitting. They provide a different perspective on improving the transferability of attacks. To mitigate the data-dependency effect, we propose the Data Erosion method. It involves finding special augmentation data that behave similarly in both vanilla models and defenses, to help attackers fool robustified models with higher chances. In addition, we introduce the Network Erosion method to overcome the network-overfitting dilemma. The idea is conceptually simple: it extends a single surrogate model to an ensemble structure with high diversity, resulting in more transferable adversarial examples. Two proposed methods can be integrated to further enhance the transferability, referred to as Erosion Attack (EA). We evaluate the proposed EA under different defenses that empirical results demonstrate the superiority of EA over existing transferable attacks and reveal the underlying threat to current robust models. The source code is publicly available at https://github.com/mesunhlf/EA.

Generating Transferable Adversarial Examples for Speech Classification

Despite the success of deep neural networks, the existence of adversarial attacks has revealed the vulnerability of neural networks in terms of security. Adversarial attacks add subtle noise to the original example, resulting in a false prediction. Although adversarial attacks have been mainly studied in the image domain, a recent line of research has discovered that speech classification systems are also exposed to adversarial attacks. By adding inaudible noise, an adversary can deceive speech classification systems and cause fatal issues in various applications, such as speaker identification and command recognition tasks. However, research on the transferability of audio adversarial examples is still limited. Thus, in this study, we first investigate the transferability of audio adversarial examples with different structures and conditions. Through extensive experiments, we discover that the transferability of audio adversarial examples is related to their noise sensitivity. Based on the analyses, we present a new adversarial attack called noise injected attack that generates highly transferable audio adversarial examples by injecting additive noise during the gradient ascent process. Our experimental results demonstrate that the proposed method outperforms other adversarial attacks in terms of transferability.

Frequency domain regularization for iterative adversarial attacks

Adversarial examples have attracted more and more attentions with the prosperity of convolutional neural networks. The transferability of adversarial examples is an important property that makes black-box attacks possible in real-world applications. On the other side, many adversarial defense methods have been proposed to improve the robustness, leading to the requirement for more transferable adversarial examples. Inspired by the regularization term for network parameters at training process, we treat adversarial attacks as training process of inputs and propose regularization constraint for inputs to prevent adversarial examples from overfitting the white-box networks and enhance the transferability. Specifically, we find a universal attribute that the outputs of convolutional neural networks have consistency to the low frequencies of inputs, and based on this, we construct a frequency domain regularization to inputs for iterative attacks. In this way, our method is compatible with existing iterative attack methods and can learn more transferable adversarial examples. Extensive experiments on ImageNet validate the superiority of our method, and compared with several attacks, we achieve attack success rate improvements of 8.0% and 11.5% on average to normal models and defense methods respectively.