White-box Attacks Research Articles

Attacking Transformers with Feature Diversity Adversarial Perturbation

Understanding the mechanisms behind Vision Transformer (ViT), particularly its vulnerability to adversarial perturbations, is crucial for addressing challenges in its real-world applications. Existing ViT adversarial attackers rely on labels to calculate the gradient for perturbation, and exhibit low transferability to other structures and tasks. In this paper, we present a label-free white-box attack approach for ViT-based models that exhibits strong transferability to various black-box models, including most ViT variants, CNNs, and MLPs, even for models developed for other modalities. Our inspiration comes from the feature collapse phenomenon in ViTs, where the critical attention mechanism overly depends on the low-frequency component of features, causing the features in middle-to-end layers to become increasingly similar and eventually collapse. We propose the feature diversity attacker to naturally accelerate this process and achieve remarkable performance and transferability.

TTTS: Tree Test Time Simulation for Enhancing Decision Tree Robustness against Adversarial Examples

Decision trees are widely used for addressing learning tasks involving tabular data. Yet, they are susceptible to adversarial attacks. In this paper, we present Tree Test Time Simulation (TTTS), a novel inference-time methodology that incorporates Monte Carlo simulations into decision trees to enhance their robustness. TTTS introduces a probabilistic modification to the decision path, without altering the underlying tree structure. Our comprehensive empirical analysis of 50 datasets yields promising results. Without the presence of any attacks, TTTS has successfully improved model performance from an AUC of 0.714 to 0.773. Under the challenging conditions of white-box attacks, TTTS demonstrated its robustness by boosting performance from an AUC of 0.337 to 0.680. Even when subjected to black-box attacks, TTTS maintains high accuracy and enhances the model's performance from an AUC of 0.628 to 0.719. Compared to defenses such as Feature Squeezing, TTTS proves to be much more effective. We also found that TTTS exhibits similar robustness in decision forest settings across different attacks.

Improving adversarial transferability through frequency enhanced momentum

The emergence of adversarial examples seriously affects the practical security deployment of convolutional neural networks. The existing attack algorithms perform brilliantly under white-box scenarios, but they show weak transferability when faced with unknown black-box models. Recent studies have revealed that models have different interests in different frequency components of images, and the low-frequency characteristics play a non-negligible role in the decision-making of models. In this article, we present the frequency enhanced momentum iterative attack, called FE-MI-FGSM. Specifically, we use multiple convolution kernels for Gaussian filtering of the image before each gradient update to push the processed image closer to the common decision boundaries of multiple models. Then, the average gradient of the white-box model to these processed images is obtained and used as the perturbation direction to generate adversarial examples with a high success rate of white-box attack and high transferability. The empirical results show that compared with the current mainstream gradient-based methods, our method performs better on both normally trained and adversarially trained models. Besides, our method can combine with gradient-based methods which integrate convergence algorithms or input transformations for the sake of reaching satisfactory improvement of the transferability.

Enhancing Generalization in Few-Shot Learning for Detecting Unknown Adversarial Examples

Deep neural networks, particularly convolutional neural networks, are vulnerable to adversarial examples, undermining their reliability in visual recognition tasks. Adversarial example detection is a crucial defense mechanism against such attacks but often relies on empirical observations and specialized metrics, posing challenges in terms of data efficiency, generalization to unknown attacks, and scalability to high-resolution datasets like ImageNet. To address these issues, we propose a prototypical network-based method using a deep residual network as the backbone architecture. This approach is capable of extracting discriminative features of adversarial and normal examples from various known adversarial examples by constructing few-shot adversarial detection tasks. Then the optimal mapping matrix is computed using the Sinkhorn algorithm from optimal transport theory, and the class centers are iteratively updated, enabling the detection of unknown adversarial examples across scenarios. Experimental results show that the proposed approach outperforms existing methods in the cross-adversary benchmark and achieves enhanced generalization on a subset of ImageNet in detecting both new adversarial attacks and adaptive white-box attacks. The proposed approach offers a promising solution for improving the safety of deep neural networks in practical applications.

Adversarial Attacks with Defense Mechanisms on Convolutional Neural Networks and Recurrent Neural Networks for Malware Classification

In the field of behavioral detection, deep learning has been extensively utilized. For example, deep learning models have been utilized to detect and classify malware. Deep learning, however, has vulnerabilities that can be exploited with crafted inputs, resulting in malicious files being misclassified. Cyber-Physical Systems (CPS) may be compromised by malicious files, which can have catastrophic consequences. This paper presents a method for classifying Windows portable executables (PEs) using Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs). To generate malware executable adversarial examples of PE, we conduct two white-box attacks, Jacobian-based Saliency Map Attack (JSMA) and Carlini and Wagner attack (C&W). An adversarial payload was injected into the DOS header, and a section was added to the file to preserve the PE functionality. The attacks successfully evaded the CNN model with a 91% evasion rate, whereas the RNN model evaded attacks at an 84.6% rate. Two defense mechanisms based on distillation and training techniques are examined in this study for overcoming adversarial example challenges. Distillation and training against JSMA resulted in the highest reductions in the evasion rates of 48.1% and 41.49%, respectively. Distillation and training against C&W resulted in the highest decrease in evasion rates, at 48.1% and 49.9%, respectively.

Adversarial Robustness with Partial Isometry.

Despite their remarkable performance, deep learning models still lack robustness guarantees, particularly in the presence of adversarial examples. This significant vulnerability raises concerns about their trustworthiness and hinders their deployment in critical domains that require certified levels of robustness. In this paper, we introduce an information geometric framework to establish precise robustness criteria for l2 white-box attacks in a multi-class classification setting. We endow the output space with the Fisher information metric and derive criteria on the input-output Jacobian to ensure robustness. We show that model robustness can be achieved by constraining the model to be partially isometric around the training points. We evaluate our approach using MNIST and CIFAR-10 datasets against adversarial attacks, revealing its substantial improvements over defensive distillation and Jacobian regularization for medium-sized perturbations and its superior robustness performance to adversarial training for large perturbations, all while maintaining the desired accuracy.

Multi-scale architectures matter: Examining the adversarial robustness of flow-based lossless compression

Exact likelihood estimation on entropy coding makes flow-based models appealing for lossless image compression. However, the high fidelity storage cost is affected by the lossless compression ratio. The trade-off between efficiency and robustness of flow-based deep lossless compression models has not been fully explored. This paper characterizes the trade-off theoretically and empirically, revealing that flow-based models are susceptible to adversarial examples resulting in a significant change in compression ratio. The fragile robustness of flow-based models is due to their intrinsic multi-scale architectures lacking the Lipschitz property. Based on this insight, a stronger white-box attack, Auto-Weighted Projected Gradient Descent (AW-PGD), is developed to generate more universal adversarial examples. Additionally, a novel flow-based lossless compression model, Robust Integer Discrete Flow (R-IDF), is proposed to achieve comparable robustness to adversarial training without sacrificing compression efficiency. Experiments demonstrate that the PGD algorithm falls into local extreme values when attacking the compression model, but the proposed attack and defense methods effectively improve the invulnerability of the flow-based compression model.

Robust object detection for autonomous driving based on semi-supervised learning

Deep learning based on labeled data has brought massive success in computer vision, speech recognition, and natural language processing. Nevertheless, labeled data is just a drop in the ocean compared with unlabeled data. How can people utilize the unlabeled data effectively? Research has focused on unsupervised and semi-supervised learning to solve such a problem. Some theoretical and empirical studies have proved that unlabeled data can help boost the generalization ability and robustness under adversarial attacks. However, current theoretical research on the relationship between robustness and unlabeled data limits its scope to toy datasets. Meanwhile, the visual models in autonomous driving need a significant improvement in robustness to guarantee security and safety. This paper proposes a semi-supervised learning framework for object detection in autonomous vehicles, improving the robustness with unlabeled data. Firstly, we build a baseline with the transfer learning of an unsupervised contrastive learning method—Momentum Contrast (MoCo). Secondly, we propose a semi-supervised co-training method to label the unlabeled data for retraining, which improves generalization on the autonomous driving dataset. Thirdly, we apply the unsupervised Bounding Box data augmentation (BBAug) method based on a search algorithm, which uses reinforcement learning to improve the robustness of object detection for autonomous driving. We present an empirical study on the KITTI dataset with diverse adversarial attack methods. Our proposed method realizes the state-of-the-art generalization and robustness under white-box attacks (DPatch and Contextual Patch) and black-box attacks (Gaussian noise, Rain, Fog, and so on). Our proposed method and empirical study show that using more unlabeled data benefits the robustness of perception systems in autonomous driving.

SecureSense: Defending Adversarial Attack for Secure Device-Free Human Activity Recognition

Deep neural networks have empowered accurate device-free human activity recognition, which has wide applications. Deep models can extract robust features from various sensors and generalize well even in challenging situations such as data-insufficient cases. However, these systems could be vulnerable to input perturbations, i.e. adversarial attacks. We empirically demonstrate that both black-box Gaussian attacks and modern adversarial white-box attacks can render their accuracies to plummet. In this paper, we firstly point out that such phenomenon can bring severe safety hazards to device-free sensing systems, and then propose a novel learning framework, SecureSense, to defend common attacks. SecureSense aims to achieve consistent predictions regardless of whether there exists an attack on its input or not, alleviating the negative effect of distribution perturbation caused by adversarial attacks. Extensive experiments demonstrate that our proposed method can significantly enhance the model robustness of existing deep models, overcoming possible attacks. The results validate that our method works well on wireless human activity recognition and person identification systems. To the best of our knowledge, this is the first work to investigate adversarial attacks and further develop a novel defense framework for wireless human activity recognition in mobile computing research.

Gradient Correction for White-Box Adversarial Attacks.

Deep neural networks (DNNs) play key roles in various artificial intelligence applications such as image classification and object recognition. However, a growing number of studies have shown that there exist adversarial examples in DNNs, which are almost imperceptibly different from the original samples but can greatly change the output of DNNs. Recently, many white-box attack algorithms have been proposed, and most of the algorithms concentrate on how to make the best use of gradients per iteration to improve adversarial performance. In this article, we focus on the properties of the widely used activation function, rectified linear unit (ReLU), and find that there exist two phenomena (i.e., wrong blocking and over transmission) misguiding the calculation of gradients for ReLU during backpropagation. Both issues enlarge the difference between the predicted changes of the loss function from gradients and corresponding actual changes and misguide the optimized direction, which results in larger perturbations. Therefore, we propose a universal gradient correction adversarial example generation method, called ADV-ReLU, to enhance the performance of gradient-based white-box attack algorithms such as fast gradient signed method (FGSM), iterative FGSM (I-FGSM), momentum I-FGSM (MI-FGSM), and variance tuning MI-FGSM (VMI-FGSM). Through backpropagation, our approach calculates the gradient of the loss function with respect to the network input, maps the values to scores, and selects a part of them to update the misguided gradients. Comprehensive experimental results on ImageNet and CIFAR10 demonstrate that our ADV-ReLU can be easily integrated into many state-of-the-art gradient-based white-box attack algorithms, as well as transferred to black-box attacks, to further decrease perturbations measured in the l2 -norm.

LAFIT: Efficient and Reliable Evaluation of Adversarial Defenses With Latent Features.

Deep convolutional neural networks (CNNs) can be easily tricked to give incorrect outputs by adding tiny perturbations to the input that are imperceptible to humans. This makes them susceptible to adversarial attacks, and poses significant security risks to deep learning systems, and presents a great challenge in making CNNs robust against such attacks. An influx of defense strategies have thus been proposed to improve the robustness of CNNs. Current attack methods, however, may fail to accurately or efficiently evaluate the robustness of defending models. In this paper, we thus propose a unified lp white-box attack strategy, LAFIT, to harness the defender's latent features in its gradient descent steps, and further employ a new loss function to normalize logits to overcome floating-point-based gradient masking. We show that not only is it more efficient, but it is also a stronger adversary than the current state-of-the-art when examined across a wide range of defense mechanisms. This suggests that adversarial attacks/defenses could be contingent on the effective use of the defender's hidden components, and robustness evaluation should no longer view models holistically.

Practical Feature Inference Attack in Vertical Federated Learning During Prediction in Artificial Internet of Things

The emergence of edge computing guarantees the combination of the Internet of Things (IoT) and artificial intelligence (AI). The vertical federated learning (VFL) framework, usually deployed by split learning, can analyze and integrate information on different features collected by different terminals in the IoT. The complete model is divided into a top model and multiple bottom models in a specific middle layer. Each passive party as a terminal with certain features owns a bottom model, and an active party as an edge server with labels holds the top model. Feature inference attack aims to infer the party’s features from the model predictions during prediction in VFL. Existing attacks considered the adversary an active party under the white-box or black-box model. However, an attacker usually is a passive party in practice because terminals are more vulnerable than edge servers. Therefore, this paper discusses a practical feature inference attack in VFL during prediction in IoT under this setting. We design an adversary builds an inference model to minimize the distance between the predictions from the inferred features and target features. Because the information on the top model and other bottom models is unknown, the adversary cannot directly train the inference model. Therefore, we utilize the zeroth-order gradient estimation method to calculate the parameters’ gradients to train the inference model. Experimental results demonstrate that the performance of our attack is comparable to that of the white-box attacks while retaining apparent advantages over the existing black-box attacks.

AD 2 VNCS: A dversarial D efense and D evice V ariation-tolerance in Memristive Crossbar-based N euromorphic C omputing S ystems

In recent years, memristive crossbar-based neuromorphic computing systems (NCS) have obtained extremely high performance in neural network acceleration. However, adversarial attacks and conductance variations of memristors bring reliability challenges to NCS design. First, adversarial attacks can fool the neural network and pose a serious threat to security critical applications. However, device variations lead to degradation of the network accuracy. In this article, we propose DFS (Deep neural network Feature importance Sampling) and BFS (Bayesian neural network Feature importance Sampling) training strategies, which consist of Bayesian Neural Network (BNN) prior setting, clustering-based loss function, and feature importance sampling techniques, to simultaneously combat device variation, white-box attack, and black-box attack challenges. Experimental results clearly demonstrate that the proposed training framework can improve the NCS reliability.

AdaNI: Adaptive Noise Injection to improve adversarial robustness

Deep Neural Networks (DNNs) have been proven vulnerable to adversarial perturbations, which narrow their applications in safe-critical scenarios such as video surveillance and autonomous driving. To counter this threat, a very recent line of adversarial defense methods is proposed to increase the uncertainty of DNNs via injecting random noises in both the training and testing process. Note the existing defense methods usually inject noises uniformly to DNNs. We argue that the magnitude of noises is highly correlated with the response of corresponding features and the randomness on important feature spots can further weaken adversarial attacks. As such, we propose a new method, namely AdaNI, which can increase feature randomness via Adaptive Noise Injection to improve the adversarial robustness. Compared to existing methods, our method creates non-unified random noises guided by features and then injects them into DNNs adaptively. Extensive experiments are conducted on several datasets (e.g., CIFAR10, CIFAR100, Mini-ImageNet) with comparisons to state-of-the-art defense methods, which corroborates the efficacy of our method against a variety of powerful white-box attacks (e.g., FGSM, PGD, C&W, Auto Attack) and black-box attacks (e.g., Transferable, ZOO, Square Attack). Moreover, our method is adapted to improve the robustness of DeepFake detection to demonstrate its applicability.

Adv-BDPM: Adversarial attack based on Boundary Diffusion Probability Model

Deep neural networks have become increasingly significant in our daily lives due to their remarkable performance. The issue of adversarial examples, which are responsible for the vulnerability problem of deep neural networks, has attracted the attention of researchers in the study of robustness of these networks. To address the issues caused by the restricted diversity and precision of adversarial perturbations in neural networks, we introduce a novel technique called Adversarial Boundary Diffusion Probability Modeling (Adv-BDPM). This approach combines boundary analysis and diffusion probability modeling. First, we combined the denoising diffusion probability model with the boundary loss to design the boundary diffusion probability model, which can generate corresponding boundary perturbations for a specific neural network. Then, through the iterative process of boundary perturbations and its corresponding orthogonal perturbations, we proposed a decision boundary search algorithm to generate adversarial samples. The comparison experiments with black-box attacks in ImageNet demonstrate that Adv-BDPM has better attack success rate and perturbation precision. The comparison experiments with white-box attacks in CIFAR-10 and CIFAR-100 demonstrate that Adv-BDPM has better attack success rate, attack diversity for the same sample, and can effectively defend against adversarial training with shorter running time.

Adversarial image generation by spatial transformation in perceptual colorspaces

Deep neural networks are known to be vulnerable to adversarial perturbations. The amount of these perturbations are generally quantified using Lp metrics, such as L0, L2 and L∞. However, even when the measured perturbations are small, they tend to be noticeable by human observers since Lp distance metrics are not representative of human perception. On the other hand, humans are less sensitive to changes in colorspace. In addition, pixel shifts in a constrained neighborhood are hard to notice. Motivated by these observations, we propose a method that creates adversarial examples by applying spatial transformations, which creates adversarial examples by changing the pixel locations independently to chrominance channels of perceptual colorspaces such as YCbCr and CIELAB, instead of making an additive perturbation or manipulating pixel values directly. In a targeted white-box attack setting, the proposed method is able to obtain competitive fooling rates with very high confidence. The experimental evaluations show that the proposed method has favorable results in terms of approximate perceptual distance between benign and adversarially generated images. The source code is publicly available at https://github.com/ayberkydn/stadv-torch.

On the Privacy of Counting Bloom Filters Under a Black-Box Attacker

Counting Bloom Filters (CBFs) are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">approximate</i> membership checking data structures, and it is normally believed that at most an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">approximate</i> reconstruction of the underlying set can be derived when interacting with a CBF. This paper decisively refutes this assumption. In a recent paper, we considered the privacy of CBFs when the attacker has access to the implementation details and thus, it sees the filter as a white-box. In that setting, we showed that the attacker may be able to extract the elements stored in the filter when the number of false positives over the entire universe is not significantly larger than the number of elements stored in the filter. In this work, we consider a black-box attacker that can only perform user interactions on the CBF to insert, remove and query elements with no knowledge of the filter implementation details. We show that even in this case, an attacker may be able to extract information from the filter at the cost of using more complex and time-consuming attack algorithms. The proposed algorithms have been implemented and compared with the white-box attack, showing that in most cases, almost the same information can be extracted from the filter.

CANARY: An Adversarial Robustness Evaluation Platform for Deep Learning Models on Image Classification

The vulnerability of deep-learning-based image classification models to erroneous conclusions in the presence of small perturbations crafted by attackers has prompted attention to the question of the models’ robustness level. However, the question of how to comprehensively and fairly measure the adversarial robustness of models with different structures and defenses as well as the performance of different attack methods has never been accurately answered. In this work, we present the design, implementation, and evaluation of Canary, a platform that aims to answer this question. Canary uses a common scoring framework that includes 4 dimensions with 26 (sub)metrics for evaluation. First, Canary generates and selects valid adversarial examples and collects metrics data through a series of tests. Then it uses a two-way evaluation strategy to guide the data organization and finally integrates all the data to give the scores for model robustness and attack effectiveness. In this process, we use Item Response Theory (IRT) for the first time to ensure that all the metrics can be fairly calculated into a score that can visually measure the capability. In order to fully demonstrate the effectiveness of Canary, we conducted large-scale testing of 15 representative models trained on the ImageNet dataset using 12 white-box attacks and 12 black-box attacks and came up with a series of in-depth and interesting findings. This further illustrates the capabilities and strengths of Canary as a benchmarking platform. Our paper provides an open-source framework for model robustness evaluation, allowing researchers to perform comprehensive and rapid evaluations of models or attack/defense algorithms, thus inspiring further improvements and greatly benefiting future work.

LWED: Lightweight white-box encryption communication system for drones over CARX algorithm

The application field of drones is being broadened with the development of communication and computer technology, but the communication link and data security of drones are facing great threats due to the limited hardware and software resource environment. Attackers can access not only the intermediate data of drone nodes through various means, but even tamper or manipulate their communication environment, when the drone and the control terminal are not trusted and can be regarded as a white-box attack environment. To solve the security problem in drone communication, a lightweight block cipher CARX for drones is proposed and white-boxed for its implementation to cope with the increasingly severe white-box attacks. And over this cryptographic algorithm, a simulated lightweight white-box encryption communication system for drones (LWED) is designed, which can complete the encryption and decryption operations and secure transmission of data and images, etc. The average encryption rate of the 64-bit version of the algorithm is 7.29×10-2Mbps, and the average encryption rate of white-box CARX (WCARX) is 1.18×10-2Mbps, and the average decryption time of the flight status parameters data of the encrypted communication control system is 0.14s. Compared with similar algorithms, the CARX algorithm can guarantee the encryption and decryption efficiency while providing high security and resistance to a variety of attacks, and can fully secure the data during the communication process after being built into the security system of drone.

Towards evaluating robustness of violence detection in videos using cross-domain transferability

Recent studies have demonstrated the applicability of classification models based on deep learning for detecting human action. For this reason, automatic violence detection from videos has become a crying need to prevent the spreading of vulnerable content on digital platforms. In spite of the remarkable success of these neural networks, they are prone to fail against adversarial attacks and thus, highlighted the need for evaluating the robustness of these state-of-the-art violence detection classifiers. Here, we propose a transferable logit attack for binary misclassification of video data which can evade the system by our spatially perturbed synthesized adversarial samples. We utilize the adversarial falsification-based threat model to validate a non-sparse white-box attack setting which will generate cross-domain adversarial video samples by perturbing only spatial features without affecting the temporal features. We carry out extensive experiments on the validation set of two popular violence detection datasets: Hockey Fight Dataset, and Movie Dataset and verify that our proposed attack method has high attack success rate for these datasets against the state-of-the-art violence detection classifier. This work aims to make future violence detection models more resistant to adversarial examples.