White-box Attacks Research Articles

Adversarial artifact detection in EEG-based brain-computer interfaces.

Machine learning has achieved significant success in electroencephalogram (EEG) based brain-computer interfaces (BCIs), with most existing research focusing on improving the decoding accuracy. However, recent studies have shown that EEG-based BCIs are vulnerable to adversarial attacks, where small perturbations added to the input can cause misclassification. Detecting adversarial examples is crucial for both understanding this phenomenon and developing effective defense strategies. This paper, for the first time, explores adversarial detection in EEG-based BCIs. We extend several popular adversarial detection approaches from computer vision to BCIs. Two new Mahalanobis distance based adversarial detection approaches, and three cosine distance based adversarial detection approaches, are also proposed, which showed promising performance in detecting three kinds of white-box attacks. We evaluated the performance of eight adversarial detection approaches on three EEG datasets, three neural networks, and four types of adversarial attacks. Our approach achieved an area under the curve (AUC) score of up to 99.99% in detecting white-box attacks. Additionally, we assessed the transferability of different adversarial detectors to unknown attacks. Through extensive experiments, we found that white-box attacks may be easily detected, and differences exist in the distributions of different types of adversarial examples. Our work should facilitate understanding the vulnerability of existing BCI models and developing more secure BCIs in the future.

Adversarial sample attack method based on loss smoothing

Deep neural networks (DNNs) are vulnerable to adversarial examples.Although the existing momentum-based adversarial example generation method can achieve a close 100%white-box attack success rate, it is still not ideal when attacking other models, and the black- box attack success rate is low. To address this, an adversarial example attack method based on loss smoothing is proposed to improve the transferability of adversarial examples. In the iterative process of calculating the gradient at each step, the current gradient is not used directly, but the local average gradient is used to accumulate momentum, so as to suppress the local oscillation phenomenon on the loss function surface, thereby stabilizing the update direction and escaping the local extreme point. A large number of experimental results on the ImageNet dataset show that compared with the existing momentum-based method, the average black-box attack success rate of the proposed method in single model attack experiments is improved by 38.07%and 27.77%, and the average black-box attack success rate in integrated model attack experiments is improved by and.Rising32.50%and28.63%

Mitigating Adversarial Attacks in Object Detection through Conditional Diffusion Models

The field of object detection has witnessed significant advancements in recent years, thanks to the remarkable progress in artificial intelligence and deep learning. These breakthroughs have significantly enhanced the accuracy and efficiency of detecting and categorizing objects in digital images. Nonetheless, contemporary object detection technologies have certain limitations, such as their inability to counter white-box attacks, insufficient denoising, suboptimal reconstruction, and gradient confusion. To overcome these hurdles, this study proposes an innovative approach that uses conditional diffusion models to perturb adversarial examples. The process begins with the application of a random chessboard mask to the adversarial example, followed by the addition of a slight noise to fill the masked area during the forward process. The adversarial image is then restored to its original form through a reverse generative process that only considers the masked pixels, not the entire image. Next, we use the complement of the initial mask as the mask for the second stage to reconstruct the image once more. This two-stage masking process allows for the complete removal of global disturbances and aids in image reconstruction. In particular, we employ a conditional diffusion model based on a class-conditional U-Net architecture, with the source image further conditioned through concatenation. Our method outperforms the recently introduced HARP method by 5% and 6.5% in mAP on the COCO2017 and PASCAL VOC datasets, respectively, under non-APT PGD attacks. Comprehensive experimental results confirm that our method can effectively restore adversarial examples, demonstrating its practical utility.

An auxiliary correction model implemented by the Correction Property of intermediate layer against adversarial examples

In powerful adversarial attacks against deep neural networks (DNN), the generated adversarial example will mislead the DNN-implemented classifier by destroying the features of the last layer. To enhance the robustness of the classifier, in our paper, a Feature Analysis and Conditional Matching prediction distribution (FACM) model is proposed to utilize the features of intermediate layers to correct the misclassification. Specifically, we first prove that the intermediate layers of the classifier still retain effective features for the original category when the classifier is subjected to adversarial attacks, which is defined as the Correction Property in our paper. According to this, we propose the FACM model consisting of Feature Analysis (FA) correction module, Conditional Matching Prediction Distribution (CMPD) correction module and decision module. Specifically, the FA correction module is comprised of fully connected layers, which takes the features of the intermediate layers as the input to correct the misclassification of the classifier. The CMPD correction module is based on a conditional autoencoder, which not only uses the features of intermediate layers as the condition to accelerate convergence but also mitigates the negative effect of adversarial examples, trained with the Kullback–Leibler loss to match prediction distribution. Through the empirically verified Diversity Property among the individual correction modules, the decision module is proposed to integrate the proposed correction modules to enhance the DNN-implemented classifier’s robustness by reducing the dimensionality of adversarial subspace. That is, the input perturbed in certain directions (i.e., dimensions) that lead to misclassifications for the classifier can be correctly classified by the proposed correction modules. The extended experiments demonstrate our FACM model outperforms the existing methods against adversarial attacks, especially optimization-based white-box attacks and query-based black-box attacks.

Improving Fast Adversarial Training With Prior-Guided Knowledge.

Fast adversarial training (FAT) is an efficient method to improve robustness in white-box attack scenarios. However, the original FAT suffers from catastrophic overfitting, which dramatically and suddenly reduces robustness after a few training epochs. Although various FAT variants have been proposed to prevent overfitting, they require high training time. In this paper, we investigate the relationship between adversarial example quality and catastrophic overfitting by comparing the training processes of standard adversarial training and FAT. We find that catastrophic overfitting occurs when the attack success rate of adversarial examples becomes worse. Based on this observation, we propose a positive prior-guided adversarial initialization to prevent overfitting by improving adversarial example quality without extra training time. This initialization is generated by using high-quality adversarial perturbations from the historical training process. We provide theoretical analysis for the proposed initialization and propose a prior-guided regularization method that boosts the smoothness of the loss function. Additionally, we design a prior-guided ensemble FAT method that averages the different model weights of historical models using different decay rates. Our proposed method, called FGSM-PGK, assembles the prior-guided knowledge, i.e., the prior-guided initialization and model weights, acquired during the historical training process. The proposed method can effectively improve the model's adversarial robustness in white-box attack scenarios. Evaluations of four datasets demonstrate the superiority of the proposed method.

Security in Transformer Visual Trackers: A Case Study on the Adversarial Robustness of Two Models.

Visual object tracking is an important technology in camera-based sensor networks, which has a wide range of practicability in auto-drive systems. A transformer is a deep learning model that adopts the mechanism of self-attention, and it differentially weights the significance of each part of the input data. It has been widely applied in the field of visual tracking. Unfortunately, the security of the transformer model is unclear. It causes such transformer-based applications to be exposed to security threats. In this work, the security of the transformer model was investigated with an important component of autonomous driving, i.e., visual tracking. Such deep-learning-based visual tracking is vulnerable to adversarial attacks, and thus, adversarial attacks were implemented as the security threats to conduct the investigation. First, adversarial examples were generated on top of video sequences to degrade the tracking performance, and the frame-by-frame temporal motion was taken into consideration when generating perturbations over the depicted tracking results. Then, the influence of perturbations on performance was sequentially investigated and analyzed. Finally, numerous experiments on OTB100, VOT2018, and GOT-10k data sets demonstrated that the executed adversarial examples were effective on the performance drops of the transformer-based visual tracking. White-box attacks showed the highest effectiveness, where the attack success rates exceeded 90% against transformer-based trackers.

DBI-Attack: Dynamic Bi-Level Integrated Attack for Intensive Multi-Scale UAV Object Detection

Benefiting from the robust feature representation capability of convolutional neural networks (CNNs), the object detection technology of intelligent high-altitude UAV remote sensing has been developed rapidly. In this field, the adversarial examples (AEs) pose serious security risks and vulnerabilities to deep learning-based systems. Due to the limitation of object size, image degradation, and scene brightness, adding adversarial disturbances to small and dense objects is extremely challenging. To study the threat of AE for UAV object detection, a dynamic bi-level integrated attack (DBI-Attack) is proposed for intensive multi-scale UAV object detection. Firstly, we use the dynamic iterative attack (DIA) method to generate perturbation on the classification level by improving the momentum iterative fast gradient sign method (MIM). Secondly, the bi-level adversarial attack method (BAAM) is constructed to add global perturbation on the decision level for completing the white-box attack. Finally, the integrated black-box attack method (IBAM) is combined to realize the black-box mislabeling and fabrication attacks. We experiment on the real drone traffic vehicle detection datasets to better evaluate the attack effectiveness. The experimental results show that the proposed method can achieve mislabeling and fabrication attacks on the UAV object detectors in black-box conditions. Furthermore, the adversarial training is applied to improve the model robustness. This work aims to call more attention to the adversarial and defensive aspects of UAV target detection models.

Comprehensive comparisons of gradient-based multi-label adversarial attacks

Adversarial examples which mislead deep neural networks by adding well-crafted perturbations have become a major threat to classification models. Gradient-based white-box attack algorithms have been widely used to generate adversarial examples. However, most of them are designed for multi-class models, and only a few gradient-based adversarial attack algorithms specifically designed for multi-label classification models. Due to the correlation between multiple labels, the performance of these gradient-based algorithms in generating adversarial examples for multi-label classification is worthy of analyzing and evaluating comprehensively. In this paper, we first transplant five typical gradient-based adversarial attack algorithms in the multi-class environment to the multi-label environment. Secondly, we comprehensively compared the performance of these five attack algorithms and the other four existing multi-label adversarial attack algorithms by experiments on six different attack types, and evaluated the transferability of adversarial examples generated by all algorithms under two attack types. Experimental results show that, among different attack types, the majority of multi-step attack algorithms have higher attack success rates compared to one-step attack algorithms. Additionally, these gradient-based algorithms face greater difficulty in augmenting labels than in hiding them. For transfer experimental results, the adversarial examples generated by all attack algorithms exhibit weaker transferability when attacking other different models.

A knowledge distillation strategy for enhancing the adversarial robustness of lightweight automatic modulation classification models

AbstractAutomatic modulation classification models based on deep learning models are at risk of being interfered by adversarial attacks. In an adversarial attack, the attacker causes the classification model to misclassify the received signal by adding carefully crafted adversarial interference to the transmitted signal. Based on the requirements of efficient computing and edge deployment, a lightweight automatic modulation classification model is proposed. Considering that the lightweight automatic modulation classification model is more susceptible to interference from adversarial attacks and that adversarial training of the lightweight auto‐modulation classification model fails to achieve the desired results, an adversarial attack defense system for the lightweight automatic modulation classification model is further proposed, which can enhance the robustness when subjected to adversarial attacks. The defense method aims to transfer the adversarial robustness from a trained large automatic modulation classification model to a lightweight model through the technique of adversarial robust distillation. The proposed method exhibits better adversarial robustness than current defense techniques in feature fusion based automatic modulation classification models in white box attack scenarios.

Sliced Wasserstein adversarial training for improving adversarial robustness

Recently, deep-learning-based models have achieved impressive performance on tasks that were previously considered to be extremely challenging. However, recent works have shown that various deep learning models are susceptible to adversarial data samples. In this paper, we propose the sliced Wasserstein adversarial training method to encourage the logit distributions of clean and adversarial data to be similar to each other. We capture the dissimilarity between two distributions using the Wasserstein metric and then align distributions using an end-to-end training process. We present the theoretical background of the motivation for our study by providing generalization error bounds for adversarial data samples. We performed experiments on three standard datasets and the results demonstrate that our method is more robust against white box attacks compared to previous methods.

C2FMI: Corse-to-Fine Black-Box Model Inversion Attack

Privacy-preserving machine learning requires that models do not reveal any private information about their training data. However, model inversion attacks (MIAs), which aim to recover the features of training data, pose a huge threat to the security of AI models. Most existing MIAs assume that the target model is white-box, but most models deployed in reality are black-box, and these models can only be accessed like an oracle. There are a few studies for black-box scenarios, but their performance is limited. In this paper, we firstly formulate the MIA problem completely in Bayesian perspective. Second, we propose a novel two-stage MIA approach, the Coarse-to-Fine Model Inversion Attack (C2FMI), which efficiently addresses the MIA problem in the black-box scenario. In stage I of C2FMI, we design a reverse network that constrains the recovered images (also named attacked images) to fall near the manifold space of the training data. In stage II, we design a black-box oriented strategy which further facilitates the attacked images to approach the training data. Empirically, C2FMI achieves a performance that even surpasses existing white-box attack methods. Furthermore, we design the stability analysis method for analyzing the stability of C2FMI along with existing MIAs. Finally, we explore the potential countermeasures which could defend against our attacks.

A contrastive learning approach for enhanced robustness for strengthening federated intelligence in internet of visual things

As the Internet of Things (IoT) continues to grow, edge intelligence has developed as a favorable paradigm to enable effective and instantaneous processing of data at the network's edge. The deployment of an edge intelligence approach in untrusted environments exposes them to adversarial attacks, posing a substantial challenge to the reliability and confidentiality of the learned models especially in federated settings. Among these attacks, the white-box attacks are the most dangerous and challenging to defend in IoT systems. This study proposes a novel approach to robustify the federated edge intelligence technique by integrating the power of contrastive learning to enhance the model's ability to distinguish between clean and adversarial examples, thereby improving its robustness. Our robustification approach is based on two lines of defense. First, an elegant federated contrastive pretraining is introduced to robustify the model by adversarially training it using a label-free representation attack. Second, we introduce a novel Consistency Regularized Triplet (CRT) loss function that encourages the regularization of the federated edge intelligence model to learn discriminative representations while minimizing the influence of adversarial perturbations. The empirical evaluation of the Food-101 and CIFAR-100 datasets revealed that the proposed method outperformed the state-of-the-art robust methods on both standard accuracy and robustness accuracy.

Impacting Robustness in Deep Learning-Based NIDS through Poisoning Attacks

The rapid expansion and pervasive reach of the internet in recent years have raised concerns about evolving and adaptable online threats, particularly with the extensive integration of Machine Learning (ML) systems into our daily routines. These systems are increasingly becoming targets of malicious attacks that seek to distort their functionality through the concept of poisoning. Such attacks aim to warp the intended operations of these services, deviating them from their true purpose. Poisoning renders systems susceptible to unauthorized access, enabling illicit users to masquerade as legitimate ones, compromising the integrity of smart technology-based systems like Network Intrusion Detection Systems (NIDSs). Therefore, it is necessary to continue working on studying the resilience of deep learning network systems while there are poisoning attacks, specifically interfering with the integrity of data conveyed over networks. This paper explores the resilience of deep learning (DL)—based NIDSs against untethered white-box attacks. More specifically, it introduces a designed poisoning attack technique geared especially for deep learning by adding various amounts of altered instances into training datasets at diverse rates and then investigating the attack’s influence on model performance. We observe that increasing injection rates (from 1% to 50%) and random amplified distribution have slightly affected the overall performance of the system, which is represented by accuracy (0.93) at the end of the experiments. However, the rest of the results related to the other measures, such as PPV (0.082), FPR (0.29), and MSE (0.67), indicate that the data manipulation poisoning attacks impact the deep learning model. These findings shed light on the vulnerability of DL-based NIDS under poisoning attacks, emphasizing the significance of securing such systems against these sophisticated threats, for which defense techniques should be considered. Our analysis, supported by experimental results, shows that the generated poisoned data have significantly impacted the model performance and are hard to be detected.

Adversarial infrared blocks: A multi-view black-box attack to thermal infrared detectors in physical world

Thermal infrared detectors have a vast array of potential applications in pedestrian detection and autonomous driving, and their safety performance is of great concern. Recent works use bulb plate, “QR” suit, and infrared patches as physical perturbations to perform white-box attacks on thermal infrared detectors, which are effective but not practical for real-world scenarios. Some researchers have tried to utilize hot and cold blocks as physical perturbations for black-box attacks on thermal infrared detectors. However, this attempts has not yielded robust and multi-view physical attacks, indicating limitations in the approach. To overcome the limitations of existing approaches, we introduce a novel black-box physical attack method, called adversarial infrared blocks (AdvIB). By optimizing the physical parameters of the infrared blocks and deploying them to pedestrians from multiple views, including the front, side, and back, AdvIB can execute robust and multi-view attacks on thermal infrared detectors. Our physical tests show that the proposed method achieves a success rate of over 80% under most distance and view conditions, validating its effectiveness. For stealthiness, our method involves attaching the adversarial infrared block to the inside of clothing, enhancing its stealthiness. Additionally, we perform comprehensive experiments and compare the experimental results with baseline to verify the robustness of our method. In summary, AdvIB allows for potent multi-view black-box attacks, profoundly influencing ethical considerations in today’s society. Potential consequences, including disasters from technology misuse and attackers’ legal liability, highlight crucial ethical and security issues associated with AdvIB. Considering these concerns, we urge heightened attention to the proposed AdvIB. Our code can be accessed from the following link: https://github.com/ChengYinHu/AdvIB.git.

Facial soft-biometrics obfuscation through adversarial attacks

Sharing facial pictures through online services, especially on social networks, has become a common habit for thousands of users. This practice hides a possible threat to privacy: the owners of such services, as well as malicious users, could automatically extract information from faces using modern and effective neural networks. In this paper, we propose the harmless use of adversarial attacks, i.e. variations of images that are almost imperceptible to the human eye and that are typically generated with the malicious purpose to mislead Convolutional Neural Networks (CNNs). Such attacks have been instead adopted to (i) obfuscate soft biometrics (gender, age, ethnicity) but (ii) without degrading the quality of the face images posted online. We achieve the above mentioned two conflicting goals by modifying the implementations of four of the most popular adversarial attacks, namely FGSM, PGD, DeepFool and C&W, in order to constrain the average amount of noise they generate on the image and the maximum perturbation they add on the single pixel. We demonstrate, in an experimental framework including three popular CNNs, namely VGG16, SENet and MobileNetV3, that the considered obfuscation method, which requires at most four seconds for each image, is effective not only when we have a complete knowledge of the neural network that extracts the soft biometrics (white box attacks), but also when the adversarial attacks are generated in a more realistic black box scenario. Finally, we prove that an opponent can implement defense techniques to partially reduce the effect of the obfuscation, but substantially paying in terms of accuracy over clean images; this result, confirmed by the experiments carried out with three popular defense methods, namely adversarial training, denoising autoencoder and Kullback-Leibler autoencoder, shows that it is not convenient for the opponent to defend himself and that the proposed approach is robust to defenses.

Defend against adversarial attacks in malware detection through attack space management

In recent years, the application of machine learning techniques based on byte sequences in malware detection has become a prominent research area. However, relevant studies have shown that machine learning methods are susceptible to adversarial examples, and the use of byte sequences provides attackers with a convenient avenue for manipulation. Current research efforts primarily focus on data augmentation techniques to enhance detection capabilities. But these approaches require significant computational resources and lack robustness. In this paper, we propose a novel defense mechanism against adversarial attacks in the context of malware detection. Our approach effectively thwarts adversarial attacks by scanning the functionality-preserving attack space. Unlike existing methods, our approach eliminates the need for repetitive retraining, significantly reducing computational demands. Theoretically, it can also withstand unknown adversarial perturbations. Experimental validation demonstrates that our method not only maintains the prediction accuracy of MalConv but also enhances it. Furthermore, our best method successfully defended against almost all existing black-box and white-box attacks, reducing the number of escaping files from multiple to zero.

DCVAE-adv: A Universal Adversarial Example Generation Method for White and Black Box Attacks

DCVAE-adv: A Universal Adversarial Example Generation Method for White and Black Box Attacks

OFEI: A Semi-Black-Box Android Adversarial Sample Attack Framework Against DLaaS

With the growing popularity of Android devices, Android malware is seriously threatening the safety of users. Although such threats can be detected by deep learning as a service (DLaaS), deep neural networks as the weakest part of DLaaS are often deceived by the adversarial samples elaborated by attackers. In this paper, we propose a new semi-black-box attack framework called one-feature-each-iteration (OFEI) to craft Android adversarial samples. This framework modifies as few features as possible and requires less classifier information to fool the classifier. We conduct a controlled experiment to evaluate our OFEI framework by comparing it with the benchmark methods JSMF, GenAttack and pointwise attack. The experimental results show that our OFEI has a higher misclassification rate of 98.25%. Furthermore, OFEI can extend the traditional white-box attack methods in the image field, such as fast gradient sign method (FGSM) and DeepFool, to craft adversarial samples for Android. Finally, to enhance the security of DLaaS, we use two uncertainties of the Bayesian neural network to construct the combined uncertainty, which is used to detect adversarial samples and achieves a high detection rate of 99.28%.

DifAttack: Query-Efficient Black-Box Adversarial Attack via Disentangled Feature Space

This work investigates efficient score-based black-box adversarial attacks with high Attack Success Rate (ASR) and good generalizability. We design a novel attack method based on a Disentangled Feature space, called DifAttack, which differs significantly from the existing ones operating over the entire feature space. Specifically, DifAttack firstly disentangles an image's latent feature into an adversarial feature and a visual feature, where the former dominates the adversarial capability of an image, while the latter largely determines its visual appearance. We train an autoencoder for the disentanglement by using pairs of clean images and their Adversarial Examples (AEs) generated from available surrogate models via white-box attack methods. Eventually, DifAttack iteratively optimizes the adversarial feature according to the query feedback from the victim model until a successful AE is generated, while keeping the visual feature unaltered. In addition, due to the avoidance of using surrogate models' gradient information when optimizing AEs for black-box models, our proposed DifAttack inherently possesses better attack capability in the open-set scenario, where the training dataset of the victim model is unknown. Extensive experimental results demonstrate that our method achieves significant improvements in ASR and query efficiency simultaneously, especially in the targeted attack and open-set scenarios. The code is available The code is available at https://github.com/csjunjun/DifAttack.git.

Attacking Transformers with Feature Diversity Adversarial Perturbation

Understanding the mechanisms behind Vision Transformer (ViT), particularly its vulnerability to adversarial perturbations, is crucial for addressing challenges in its real-world applications. Existing ViT adversarial attackers rely on labels to calculate the gradient for perturbation, and exhibit low transferability to other structures and tasks. In this paper, we present a label-free white-box attack approach for ViT-based models that exhibits strong transferability to various black-box models, including most ViT variants, CNNs, and MLPs, even for models developed for other modalities. Our inspiration comes from the feature collapse phenomenon in ViTs, where the critical attention mechanism overly depends on the low-frequency component of features, causing the features in middle-to-end layers to become increasingly similar and eventually collapse. We propose the feature diversity attacker to naturally accelerate this process and achieve remarkable performance and transferability.