Adversarial Perturbations Research Articles

Defending Video Recognition Model Against Adversarial Perturbations via Defense Patterns

Defending Video Recognition Model Against Adversarial Perturbations via Defense Patterns

Perturbation Augmentation for Adversarial Training with Diverse Attacks

Adversarial Training (AT) aims to alleviate the vulnerability of deep neural networks to adversarial perturbations. However, the AT techniques struggle to maintain the performance on natural samples while improving the deep model’s robustness. The absence of perturbation diversity in generated during the adversarial training degrades the generalizability of the robust models, causing overfitting to particular perturbations and a decrease in natural performance. This study proposes an adversarial training framework that augments adversarial directions from a single-step attack to address the trade-off between robustness and generalization. Inspired by feature scattering adversarial training, the proposed framework computes a principal adversarial direction with a single-step attack that finds a perturbation disrupting the inter-sample relationships in the mini-batch during adversarial training. The principal direction obtained at each iteration is augmented by sampling new adversarial directions within a region spanning 45 degrees from the principal adversarial direction. The proposed adversarial training approach does not require extra backpropagation steps in adversarial direction augmentation. Therefore, generalization of the robust model is improved without posing an additional burden on the feature scattering adversarial training. Experiments on CIFAR-10, CIFAR-100, SVHN, Tiny-ImageNet, and The German Traffic Sign Recognition Benchmark consistently improve the accuracy on adversarial with an almost pristine natural performance.

Efficient Adversarial Attack Based on Moment Estimation and Lookahead Gradient

Adversarial example generation is a technique that involves perturbing inputs with imperceptible noise to induce misclassifications in neural networks, serving as a means to assess the robustness of such models. Among the adversarial attack algorithms, momentum iterative fast gradient sign Method (MI-FGSM) and its variants constitute a class of highly effective offensive strategies, achieving near-perfect attack success rates in white-box settings. However, these methods’ use of sign activation functions severely compromises gradient information, which leads to low success rates in black-box attacks and results in large adversarial perturbations. In this paper, we introduce a novel adversarial attack algorithm, NA-FGTM. Our method employs the Tanh activation function instead of the sign which can accurately preserve gradient information. In addition, it utilizes the Adam optimization algorithm as well as the Nesterov acceleration, which is able to stabilize gradient update directions and expedite gradient convergence. Above all, the transferability of adversarial examples can be enhanced. Through integration with data augmentation techniques such as DIM, TIM, and SIM, NA-FGTM can further improve the efficacy of black-box attacks. Extensive experiments on the ImageNet dataset demonstrate that our method outperforms the state-of-the-art approaches in terms of black-box attack success rate and generates adversarial examples with smaller perturbations.

DIB-UAP: enhancing the transferability of universal adversarial perturbation via deep information bottleneck

Significant structural differences in DNN-based object detectors hinders the transferability of adversarial attacks. Studies show that intermediate features extracted by the detector contain more model-independent information, and disrupting these features can enhance attack transferability across different detectors. However, the challenge lies in selecting crucial features that impact detection from redundant intermediate features. To address this issue, we introduce the Deep information bottleneck universal adversarial perturbation (DIB-UAP). DIB-UAP utilizes the deep information bottleneck to establish a link between intermediate features and model output, extracting crucial intermediate features and disrupting them to generate UAP with strong attack transferability. Additionally, we propose a data augmentation method, Scale & Tile, which effectively enhances the attack performance of UAP on medium and large-scale objects. Testing on two benchmark datasets with eight comparative methods across four black-box mainstream detectors has confirmed the attack transferability of DIB-UAP. Furthermore, practical utility validation of DIB-UAP has been conducted on a commercial object detection platform.

Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors Using Machine Learning

Existing literature on adversarial Machine Learning (ML) focuses either on showing attacks that break every ML model or defenses that withstand most attacks. Unfortunately, little consideration is given to the actual feasibility of the attack or the defense. Moreover, adversarial samples are often crafted in the “feature-space,” making the corresponding evaluations of questionable value. Simply put, the current situation does not allow one to estimate the actual threat posed by adversarial attacks, leading to a lack of secure ML systems. We aim to clarify such confusion in this article. By considering the application of ML for Phishing Website Detection (PWD), we formalize the “evasion-space,” in which an adversarial perturbation can be introduced to fool an ML-PWD—demonstrating that even perturbations in the “feature-space” are useful. Then, we propose a realistic threat model describing evasion attacks against ML-PWD that are cheap to stage, and hence intrinsically more attractive for real phishers. After that, we perform the first statistically validated assessment of state-of-the-art ML-PWD against 12 evasion attacks. Our evaluation shows (i) the true efficacy of evasion attempts that are more likely to occur; and (ii) the impact of perturbations crafted in different evasion-spaces; our realistic evasion attempts induce a statistically significant degradation (3–10% at p < 0.05), and their cheap cost makes them a subtle threat. Notably, however, some ML-PWD are immune to our most realistic attacks ( p = 0.22). Finally, as an additional contribution of this journal publication, we are the first to propose and empirically evaluate the intriguing case wherein an attacker introduces perturbations in multiple evasion-spaces at the same time . These new results show that simultaneously applying perturbations in the problem- and feature-space can cause a drop in the detection rate from 0.95 to 0. Our contribution paves the way for a much-needed re-assessment of adversarial attacks against ML systems for cybersecurity.

A Security Study of Multimodel Artificial Intelligence System: Adaptive Retention Attack for Object Detection System with Multifocus Image Fusion Model

Image preprocessing models are usually employed as the preceding operations of high‐level vision tasks to improve the performance. The adversarial attack technology makes both these models face severe challenges. Prior research is focused solely on attacking single object detection models, without considering the impact of the preprocessing models (multifocus image fusion) on adversarial perturbations within the object detection system. Multifocus image fusion models work in conjunction with the object detection models to enhance the quality of the images and improve the capability of object detection system. Herein, the problem of attacking object detection system that utilizes multifocus image fusion as its preprocessing models is addressed. To retain the attack capabilities of adversarial samples against as many perturbations as possible, new attack method called adaptive retention attack (ARA) is proposed. Additionally, adversarial perturbations concentration mechanism and image selection mechanism, which, respectively, enhance the transferability and attack capability of ARA‐generated adversarial samples. Extensive experiments have demonstrated the feasibility of the ARA. The results confirm that the ARA method can successfully bypass multifocus image fusion models to attack the object detection model.

Towards assessing the synthetic-to-measured adversarial vulnerability of SAR ATR

Recently, there has been increasing concern about the vulnerability of deep neural network (DNN)-based synthetic aperture radar (SAR) automatic target recognition (ATR) to adversarial attacks, where a DNN could be easily deceived by clean input with imperceptible but aggressive perturbations. This paper studies the synthetic-to-measured (S2M) transfer setting, where an attacker generates adversarial perturbation based solely on synthetic data and transfers it against victim models trained with measured data. Compared with the current measured-to-measured (M2M) transfer setting, our approach does not need direct access to the victim model or the measured SAR data. We also propose the transferability estimation attack (TEA) to uncover the adversarial risks in this more challenging and practical scenario. The TEA makes full use of the limited similarity between the synthetic and measured data pairs for blind estimation and optimization of S2M transferability, leading to feasible surrogate model enhancement without mastering the victim model and data. Comprehensive evaluations based on the publicly available synthetic and measured paired labeled experiment (SAMPLE) dataset demonstrate that the TEA outperforms state-of-the-art methods and can significantly enhance various attack algorithms in computer vision and remote sensing applications. Codes and data are available at https://github.com/scenarri/S2M-TEA.

A two-stage co-adversarial perturbation to mitigate out-of-distribution generalization of large-scale graph

In the realm of graph out-of-distribution (OOD), despite recent strides in advancing graph neural networks (GNNs) for the modeling of graph data, training GNNs on large-scale datasets presents a formidable hurdle due to the pervasive challenge of overfitting. To address these issues, researchers have explored adversarial training, a technique that enriches training data with worst-case adversarial examples. However, while prior work on adversarial training primarily focuses on safeguarding GNNs against malicious attacks, its potential to enhance the OOD generalization abilities of GNNs in the context of graph analytics remains less explored. In our research, we delve into the inner workings of GNNs by examining the landscapes of weight and feature losses, which respectively illustrate how the loss function changes concerning model weights and node features. Our investigation reveals a noteworthy phenomenon: GNNs are inclined to become trapped in sharp local minima within these loss landscapes, resulting in suboptimal OOD generalization performance. To address this challenge, we introduce the concept of co-adversarial perturbation optimization, which considers both model weights and node features, and we design an alternating adversarial perturbation algorithm for graph out-of-distribution generalization. This algorithm operates iteratively, smoothing the weight and feature loss landscapes alternately. Moreover, our training process unfolds in two distinct stages. The first stage centers on standard cross-entropy minimization, ensuring rapid convergence of GNN models. In the second stage, we employ our alternating adversarial training strategy to prevent the models from becoming ensnared in locally sharp minima. Our extensive experiments provide compelling evidence that our CAP approach can generally enhance the OOD generalization performance of GNNs across a diverse range of large-scale graphs.

CommanderUAP: a practical and transferable universal adversarial attacks on speech recognition models

Most of the adversarial attacks against speech recognition systems focus on specific adversarial perturbations, which are generated by adversaries for each normal example to achieve the attack. Universal adversarial perturbations (UAPs), which are independent of the examples, have recently received wide attention for their enhanced real-time applicability and expanded threat range. However, most of the UAP research concentrates on the image domain, and less on speech. In this paper, we propose a staged perturbation generation method that constructs CommanderUAP, which achieves a high success rate of universal adversarial attack against speech recognition models. Moreover, we apply some methods from model training to improve the generalization in attack and we control the imperceptibility of the perturbation in both time and frequency domains. In specific scenarios, CommanderUAP can also transfer attack some commercial speech recognition APIs.

Cube-Evo: A Query-Efficient Black-Box Attack on Video Classification System

The current progressive research in the domain of black-box adversarial attack enhances the reliability of deep neural network (DNN)-based video systems. Recent works mainly carry out black-box adversarial attacks on video systems by query-based parameter dimension reduction. However, the additional temporal dimension of video data leads to massive query consumption and low attack success rate. In this article, we embark on our efforts to design an effective adversarial attack on popular video classification systems. We deeply root the observations that the DNN-based systems are sensitive to adversarial perturbations with high frequency and reconstructed shape. Specifically, we propose a systematic attack pipeline Cube-Evo, aiming to reduce the search space dimension and obtain the effective adversarial perturbation via the optimal parameter group updating. We evaluate the proposed attack pipeline on two popular datasets: UCF101 and JESTER. Our attack pipeline reduces query consumption and achieves a high success rate on various DNN-based video classification systems. Compared with the state-of-the-art method Geo-Trap-Att, our pipeline averagely reduces 1.6× query consumption in untargeted attacks and 2.9× in targeted attacks. Besides, Cube-Evo improves 13% attack success rate on average, achieving new state-of-the-art results over diverse video classification systems.

Adversarial Learning for Constrained Image Splicing Detection and Localization

Abstract: In the era of digital media, the manipulation of images has become a significant concern, particularly on social media platforms. Adversarial Learning for Constrained Image Splicing Detection and Localization aims to develop a robust system capable of recognizing and localizing spliced or tampered images. This research proposes a novel approach that leverages Convolutional Neural Networks (CNNs) and adversarial learningtechniques to enhance the detection and localization of image splicing. The system is designed to aid in distinguishing between authentic and manipulated images, thereby promoting transparency and trust in digital media. With the rapid advancement of digital media and image editingtools, the manipulation of visual content has become increasingly prevalent, posing significant challenges to the credibility and trustworthiness of images shared on social media platforms. One pervasive form of image manipulation is image splicing, where portions of two or more images are seamlessly combined to create a composite image, often with the intent to mislead or deceive viewers. Adversarial Learning for Constrained Image Splicing Detection and Localization aims to address this issue by developing a robust system capable of accurately detecting and localizing spliced regions within tampered images. This research proposes a novel approach that synergistically integrates Convolutional Neural Networks (CNNs) and adversarial learning techniques to enhance the performance of image splicing detection and localization. The proposed system employs a two-stage process: first, a CNN-based classifier is trained on a large dataset of authentic and spliced images to learn discriminative features for distinguishing between the two classes; subsequently, an adversarial learning model is employed to generate adversarial examples that can deceive the CNN classifier, while the classifier is iteratively updated to become more robust against these adversarial perturbations. By accurately identifying and localizing spliced regions within images, this research contributes to promoting transparency and trust in digital media shared on social media platforms, ultimately empowering users to make informed decisions about the authenticity of visual content. The proposed system has potential applications in various domains, including journalism, law enforcement, and content moderation, where verifying the integrity of visual evidence is of paramount importance.

An adversarial example attack method based on predicted bounding box adaptive deformation in optical remote sensing images

Existing global adversarial attacks are not applicable to real-time optical remote sensing object detectors based on the YOLO series of deep neural networks, which makes it difficult to improve the adversarial robustness of single-stage detectors. The existing methods do not work well enough in optical remote sensing images, which may be due to the mechanism of adversarial perturbations is not suitable. Therefore, an adaptive deformation method (ADM) was proposed to fool the detector into generating wrong predicted bounding boxes. Building upon this, we introduce the Adaptive Deformation Method Iterative Fast Gradient Sign Method (ADM-I-FGSM) and Adaptive Deformation Mechanism Projected Gradient Descent (ADM-PGD) against YOLOv4 and YOLOv5. ADM method can obtain the deformation trend values based on the length-to-width ratio of the prediction box, and the adversarial perturbation trend generated based on these trend values has better adversarial effect. Through experiments, we validate that our approach exhibits a higher adversarial success rate compared to the state-of-the-art methods. We anticipate that our unveiled attack scheme will aid in the evaluation of adversarial resilience of these models.

Feature-aware transferable adversarial attacks against image classification

Compared to white-box adversarial attacks, black-box adversarial attacks are more applicable in practical scenarios and have received significant attention. However, most existing black-box attacks are optimized at the output layer, and the generated adversarial samples are difficult to transfer from the surrogate model to the target model. Existing adversarial attacks indiscriminately perturb features in intermediate layers, which are prone to fall into local optima of surrogate models and have limited transferability. In this paper, we propose a transferable targeted adversarial attack framework based on a feature-aware triplet, which achieves a better tradeoff between attack ability and transferability by perturbing salient features and constructing representative sample features. The importance-weighted optimization objective interferes with the salient object-aware features of images in a targeted manner and guides adversarial perturbations in the optimization process to pull the intermediate features of samples toward a target class while pushing them far from a source class to achieve targeted transferable attacks. Moreover, a construction method of a feature library with the weighted average of feature importance is built to obtain more expressive intermediate features of the target and source classes. The modified target features and source features are fed into the triplets to guide the optimization objectives to find more transferable adversarial samples. Extensive experiments on the ImageNet-compatible dataset verify the effectiveness of the proposed method, e.g., improving the untargeted success rate by 1.8% and the targeted success rate by 2.3% against normally trained models as compared to the existing methods.

Improving Adversarial Robustness of ECG Classification Based on Lipschitz Constraints and Channel Activation Suppression.

Deep neural networks (DNNs) are increasingly important in the medical diagnosis of electrocardiogram (ECG) signals. However, research has shown that DNNs are highly vulnerable to adversarial examples, which can be created by carefully crafted perturbations. This vulnerability can lead to potential medical accidents. This poses new challenges for the application of DNNs in the medical diagnosis of ECG signals. This paper proposes a novel network Channel Activation Suppression with Lipschitz Constraints Net (CASLCNet), which employs the Channel-wise Activation Suppressing (CAS) strategy to dynamically adjust the contribution of different channels to the class prediction and uses the 1-Lipschitz's ℓ∞ distance network as a robust classifier to reduce the impact of adversarial perturbations on the model itself in order to increase the adversarial robustness of the model. The experimental results demonstrate that CASLCNet achieves ACCrobust scores of 91.03% and 83.01% when subjected to PGD attacks on the MIT-BIH and CPSC2018 datasets, respectively, which proves that the proposed method in this paper enhances the model's adversarial robustness while maintaining a high accuracy rate.

Transferable universal adversarial perturbations against speaker recognition systems

Transferable universal adversarial perturbations against speaker recognition systems

Gamma-convergence of a nonlocal perimeter arising in adversarial machine learning

In this paper we prove Gamma-convergence of a nonlocal perimeter of Minkowski type to a local anisotropic perimeter. The nonlocal model describes the regularizing effect of adversarial training in binary classifications. The energy essentially depends on the interaction between two distributions modelling likelihoods for the associated classes. We overcome typical strict regularity assumptions for the distributions by only assuming that they have bounded BV densities. In the natural topology coming from compactness, we prove Gamma-convergence to a weighted perimeter with weight determined by an anisotropic function of the two densities. Despite being local, this sharp interface limit reflects classification stability with respect to adversarial perturbations. We further apply our results to deduce Gamma-convergence of the associated total variations, to study the asymptotics of adversarial training, and to prove Gamma-convergence of graph discretizations for the nonlocal perimeter.

Blinding and blurring the multi-object tracker with adversarial perturbations

Adversarial attack reveals a potential imperfection in deep models that they are susceptible to being tricked by imperceptible perturbations added to images. Recent deep multi-object trackers combine the functionalities of detection and association, rendering attacks on either the detector or the association component an effective means of deception. Existing attacks focus on increasing the frequency of ID switching, which greatly damages tracking stability, but is not enough to make the tracker completely ineffective. To fully explore the potential of adversarial attacks, we propose Blind-Blur Attack (BBA), a novel attack method based on spatio-temporal motion information to fool multi-object trackers. Specifically, a simple but efficient perturbation generator is trained with the blind-blur loss, simultaneously making the target invisible to the tracker and letting the background be regarded as moving targets. We take TraDeS as our main research tracker, and verify our attack method on other excellent algorithms (i.e., CenterTrack, FairMOT, and ByteTrack) on MOT-Challenge benchmark datasets (i.e., MOT16, MOT17, and MOT20). BBA attack reduced the MOTA of TraDeS and ByteTrack from 69.1 and 80.3 to −238.1 and −357.0, respectively, indicating that it is an efficient method with a high degrees of transferability.

A contrastive learning approach for enhanced robustness for strengthening federated intelligence in internet of visual things

As the Internet of Things (IoT) continues to grow, edge intelligence has developed as a favorable paradigm to enable effective and instantaneous processing of data at the network's edge. The deployment of an edge intelligence approach in untrusted environments exposes them to adversarial attacks, posing a substantial challenge to the reliability and confidentiality of the learned models especially in federated settings. Among these attacks, the white-box attacks are the most dangerous and challenging to defend in IoT systems. This study proposes a novel approach to robustify the federated edge intelligence technique by integrating the power of contrastive learning to enhance the model's ability to distinguish between clean and adversarial examples, thereby improving its robustness. Our robustification approach is based on two lines of defense. First, an elegant federated contrastive pretraining is introduced to robustify the model by adversarially training it using a label-free representation attack. Second, we introduce a novel Consistency Regularized Triplet (CRT) loss function that encourages the regularization of the federated edge intelligence model to learn discriminative representations while minimizing the influence of adversarial perturbations. The empirical evaluation of the Food-101 and CIFAR-100 datasets revealed that the proposed method outperformed the state-of-the-art robust methods on both standard accuracy and robustness accuracy.

Enhancing reinforcement learning based adversarial malware generation to evade static detection

The anti-detection capabilities of adversarial malware examples have drawn the attention of antivirus vendors and researchers. In black-box scenarios where internal information of the target model cannot be accessed, with the ability of the reinforcement learning (RL) agents to adjust strategies based on the feedback from the environment, existing RL-based methods enable evasion of Windows PE malware detectors. However, obtaining evasion rewards as positive feedback in the black-box setting proves challenging, resulting in low training efficiency. To address this issue, we introduce the intrinsic curiosity reward into the framework to motivate the agent to explore unknown state spaces and learn effective evasion strategies. Additionally, we employ the generative adversarial network (GAN) to obtain varying synthetic data, which replaces random or benign bytes as adversarial payloads for the agent's action content, improving attack capabilities and reducing the risk of hard-coded adversarial perturbations being anchored. We compare the attack performance with other RL-based baseline methods, and experimental results show that our framework is more flexible and effective, achieving a 63%-85% attack success rate against EMBER, FireEye and MalConv. Even if defensive measures are taken, the proposed method still has a certain attack capability, and the success rate remains between 48%-67%.

Instance-Specific Model Perturbation Improves Generalized Zero-Shot Learning.

Zero-shot learning (ZSL) refers to the design of predictive functions on new classes (unseen classes) of data that have never been seen during training. In a more practical scenario, generalized zero-shot learning (GZSL) requires predicting both seen and unseen classes accurately. In the absence of target samples, many GZSL models may overfit training data and are inclined to predict individuals as categories that have been seen in training. To alleviate this problem, we develop a parameter-wise adversarial training process that promotes robust recognition of seen classes while designing during the test a novel model perturbation mechanism to ensure sufficient sensitivity to unseen classes. Concretely, adversarial perturbation is conducted on the model to obtain instance-specific parameters so that predictions can be biased to unseen classes in the test. Meanwhile, the robust training encourages the model robustness, leading to nearly unaffected prediction for seen classes. Moreover, perturbations in the parameter space, computed from multiple individuals simultaneously, can be used to avoid the effect of perturbations that are too extreme and ruin the predictions. Comparison results on four benchmark ZSL data sets show the effective improvement that the proposed framework made on zero-shot methods with learned metrics.