Adversarial Perturbations Research Articles

How Resilient Are Kolmogorov–Arnold Networks in Classification Tasks? A Robustness Investigation

Kolmogorov–Arnold Networks (KANs) are a novel class of neural network architectures based on the Kolmogorov–Arnold representation theorem, which has demonstrated potential advantages in accuracy and interpretability over Multilayer Perceptron (MLP) models. This paper comprehensively evaluates the robustness of various KAN architectures—including KAN, KAN-Mixer, KANConv_KAN, and KANConv_MLP—against adversarial attacks, which constitute a critical aspect that has been underexplored in current research. We compare these models with MLP-based architectures such as MLP, MLP-Mixer, and ConvNet_MLP across three traffic sign classification datasets: GTSRB, BTSD, and CTSD. The models were subjected to various adversarial attacks (FGSM, PGD, CW, and BIM) with varying perturbation levels and were trained under different strategies, including standard training, adversarial training, and Randomized Smoothing. Our experimental results demonstrate that KAN-based models, particularly the KAN-Mixer, exhibit superior robustness to adversarial attacks compared to their MLP counterparts. Specifically, the KAN-Mixer consistently achieved lower Success Attack Rates (SARs) and Degrees of Change (DoCs) across most attack types and datasets while maintaining high accuracy on clean data. For instance, under FGSM attacks with ϵ=0.01, the KAN-Mixer outperformed the MLP-Mixer by maintaining higher accuracy and lower SARs. Adversarial training and Randomized Smoothing further enhanced the robustness of KAN-based models, with t-SNE visualizations revealing more stable latent space representations under adversarial perturbations. These findings underscore the potential of KAN architectures to improve neural network security and reliability in adversarial settings.

Increasing the Robustness of Image Quality Assessment Models Through Adversarial Training

The adversarial robustness of image quality assessment (IQA) models to adversarial attacks is emerging as a critical issue. Adversarial training has been widely used to improve the robustness of neural networks to adversarial attacks, but little in-depth research has examined adversarial training as a way to improve IQA model robustness. This study introduces an enhanced adversarial training approach tailored to IQA models; it adjusts the perceptual quality scores of adversarial images during training to enhance the correlation between an IQA model’s quality and the subjective quality scores. We also propose a new method for comparing IQA model robustness by measuring the Integral Robustness Score; this method evaluates the IQA model resistance to a set of adversarial perturbations with different magnitudes. We used our adversarial training approach to increase the robustness of five IQA models. Additionally, we tested the robustness of adversarially trained IQA models to 16 adversarial attacks and conducted an empirical probabilistic estimation of this feature.

Review-Based Explainable Recommendations: A Transparency Perspective

With the prevalence of deep learning based embedding approaches, recommender systems have become a proven and indispensable tool in various information filtering applications. However, many of them remain difficult to diagnose what aspects of the deep models’ input drive the final ranking decision, thus, they cannot often be understood by human stakeholders. In this paper, we investigate the dilemma between recommendation and explainability, and show that by utilizing the contextual features (e.g., item reviews from users), we can design a series of explainable recommender systems without sacrificing their performance. In particular, we propose three types of explainable recommendation strategies with gradual changes in model transparency: whitebox, graybox, and blackbox. Each strategy explains its ranking decisions via different mechanisms: attention weights, adversarial perturbations, and counterfactual perturbations. We apply these explainable models on five real-world data sets under the contextualized setting where the users provided explicit reviews of the items. The empirical results show that our models can achieve highly competitive ranking performance, and meanwhile, generate accurate and effective explanations in terms of numerous quantitative metrics and qualitative visualizations.

CMDN: Pre-Trained Visual Representations Boost Adversarial Robustness for UAV Tracking

Visual object tracking is widely adopted to unmanned aerial vehicle (UAV)-related applications, which demand reliable tracking precision and real-time performance. However, UAV trackers are highly susceptible to adversarial attacks, while research on developing effective adversarial defense methods for UAV tracking remains limited. To tackle these challenges, we propose CMDN, a novel pre-processing defense network that effectively purifies adversarial perturbations by reconstructing video frames. This network learns robust visual representations from video frames, guided by meaningful features from both the search region and the template. Comprehensive experiments on three benchmarks demonstrate that CMDN is capable of enhancing a UAV tracker’s adversarial robustness in both adaptive and non-adaptive attack scenarios. In addition, CMDN maintains stable defense effectiveness when transferred to heterogeneous trackers. Real-world tests on the UAV platform also validate its reliable defense effectiveness and real-time performance, with CMDN achieving 27 FPS on NVIDIA Jetson Orin 16 GB (25 W mode).

Leveraging Universal Adversarial Perturbation and Frequency Band Filters Against Face Recognition

Universal adversarial perturbation (UAP) exhibits universality as it is independent of specific images. Although previous investigations have shown that the classification of natural images is susceptible to universal adversarial attacks, the impact of UAP on face recognition has not been fully investigated. Thus, in this paper we assess the vulnerability of face recognition for UAP. We propose FaUAP-FBF, which exploits the frequency domain by learning high, middle, and low band filters as an additional dimension of refining facial UAP. The facial UAP and filters are alternately and repeatedly learned from a training set. Furthermore, we convert non-target attacks to target attacks by customizing a target example, which is an out-of-distribution sample for a training set. Accordingly, non-target and target attacks form a uniform target attack. Finally, the variance of cosine similarity is incorporated into the adversarial loss, thereby enhancing the attacking capability. Extensive experiments on LFW and CASIA-WebFace datasets show that FaUAP-FBF has a higher fooling rate and better objective stealthiness metrics across the evaluated network structures compared to existing universal adversarial attacks, which confirms the effectiveness of the proposed FaUAP-FBF. Our results also imply that UAP poses a real threat for face recognition systems and should be taken seriously when face recognition systems are being designed.

Efficient Ensemble Adversarial Attack for a Deep Neural Network (DNN)-Based Unmanned Aerial Vehicle (UAV) Vision System

In recent years, unmanned aerial vehicles (UAVs) vision systems based on deep neural networks (DNNs) have made remarkable advancements, demonstrating impressive performance. However, due to the inherent characteristics of DNNs, these systems have become increasingly vulnerable to adversarial attacks. Traditional black-box attack methods typically require a large number of queries to generate adversarial samples successfully. In this paper, we propose a novel adversarial attack technique designed to achieve efficient black-box attacks with a minimal number of queries. We define a perturbation generator that first decomposes the image into four frequency bands using wavelet decomposition and then searches for adversarial perturbations across these bands by minimizing a weighted loss function on a set of fixed surrogate models. For the target victim model, the perturbation images generated by the perturbation generator are used to query and update the weights in the loss function, as well as the weights for different frequency bands. Experimental results show that, compared to state-of-the-art methods on various image classifiers trained on ImageNet (such as VGG-19, DenseNet-121, and ResNext-50), our method achieves a success rate over 98% for targeted attacks and nearly a 100% success rate for non-targeted attacks with only 1–2 queries per image.

Countering adversarial perturbations in graphs using error correcting codes

Countering adversarial perturbations in graphs using error correcting codes

Vibrational Spectroscopy Can Be Vulnerable to Adversarial Attacks.

Nondestructive detection methods based on vibrational spectroscopy have been widely used in many critical applications in a variety of fields such as the chemical industry, pharmacy, national defense, security, and so on. As these methods/applications rely on machine learning models for data analysis, studying the threats associated with adversarial examples in vibrational spectroscopy and defenses against them is of great importance. In this paper, we propose a novel adversarial method to attack vibrational spectroscopy, named SynPat, where synthetic peaks produced by a physical model are placed at key locations to form adversarial perturbations. Our new attack generates perturbations that successfully deceive machine learning models for Raman and infrared spectrum analysis while they blend much better into the spectra and hence are unnoticeable to human operators, unlike the existing state-of-the-art adversarial attacking methods, e.g., images and audio. We verified the superiority of the proposed SynPat by an imperceptibility test conducted by human experts and of defense experiments by an AI detector. To the best of our knowledge, this is a first thorough study on the robustness of vibrational spectroscopic techniques against adversarial samples and defense mechanisms. Our extensive experiments show that machine learning models for vibrational spectroscopy, including conventional and deep models for Raman or infrared classification and regression, are all vulnerable to adversarial perturbations and thus may pose serious security threats to our society.

Adversarial robust image processing in medical digital twin

Recent advancements in state-of-the-art technologies, including Artificial Intelligence (AI), Internet of Things (IoT), and cloud computing, have led to the emergence of an innovative technology known as digital twins (DTs). A digital twin is a virtual replica of the physical entity, with data connections in between. This technology has proven highly effective in several industries by improving decision-making and operational efficiency. In critical areas like healthcare, digital twins are increasingly being used to address the limitations of conventional approaches by creating virtual simulations of hospitals, medical equipment, patients, or even individual organs. These medical digital twins (MDT) revolutionize the healthcare industry by offering advanced solutions to enhance treatment outcomes and overall patient care. However, these systems are challenging because of the security and critical issues involved. Therefore, despite their achievements, the numerous security threats make it crucial to address the security challenges of digital twin technology. Given the lack of research on attacks targeting MDT functionalities, we concentrated on a specific cyber threat called adversarial attacks. Adversarial attacks exploit the model’s performance by introducing small, carefully crafted perturbations to manipulate the input data. To assess the vulnerability of medical digital twins to such attacks, we carried out a proof-of-concept study. Using image processing techniques and an artificial neural network model, we created a digital twin to diagnose breast cancer through thermography images. Then, we employed this digital twin to initiate an adversarial attack. For this purpose, we inserted adversarial perturbation as input to the trained model. Our results demonstrated the vulnerability of the digital twin model to adversarial attacks. To tackle this problem, we implemented an innovative modification to the digital twin’s architecture to enhance its robustness against various attacks. We proposed a novel defense method that fuses wavelet denoising and adversarial training, substantially strengthening the model’s resilience to adversarial attacks. Furthermore, the proposed digital twin is evaluated using a dataset of diabetic foot ulcers. To the best of our knowledge, it is the first defense method that makes the medical digital twin significantly robust against adversarial attacks.

WiCAM2.0: Imperceptible and Targeted Attack on Deep Learning based WiFi Sensing

With the widespread adoption of deep learning models in wireless sensing, substantial efforts have been made to develop sophisticated models that improve the accuracy and performance of sensing applications. However, the exploration of potential vulnerabilities in deep models has been limited, with existing studies primarily focusing on evaluating wireless adversarial performance in communication or sensing alone. Moreover, there is a lack of a comprehensive definition for attack imperceptibility. In this paper, we come up with a definition of the wireless attack imperceptibility for both communication and sensing. Our objective is to create an adversarial perturbation capable of degrading WiFi sensing performance while preserving WiFi communication integrity. To achieve this, we propose WiCAM2.0 to reveal the temporal and spatial attention of a DNN, capturing the crucial portions of its input. Then we design a mask to confine adversarial perturbations in the attended parts only, minimizing the impact on WiFi communication. WiCAM2.0 is a general adversarial framework that integrates adversarial methods like FGSM and PGD to generate perturbations, capable of initiating both non-targeted and targeted attacks. We carry out experiments on three popular WiFi sensing applications, including human activity recognition, gesture recognition, and user identification. Extensive experiments are conducted on both public datasets and self-collected datasets.

Deep video steganography using temporal-attention-based frame selection and spatial sparse adversarial attack

With the development of deep learning-based steganalysis, video steganography is facing with great challenges. To address the insufficient security against steganalysis of existing deep video steganography, given that the video has both spatial and temporal dimensions, this paper proposes a deep video steganography method using temporal frame selection and spatial sparse adversarial attack. In temporal dimension, a stego frame selection module based on temporal attention is designed to calculate the weight of each frame and selects frames with high weights for message and sparse perturbation embedding. In spatial dimension, sparse adversarial perturbations are performed in the selected frames to improve the ability of resisting steganalysis. Moreover, to control the adversarial perturbations’ sparsity flexibly, an intra-frame dynamic sparsity threshold mechanism is designed by using percentile. Experimental results demonstrate that the proposed method effectively enhances the visual quality and security against steganalysis of video steganography and has controllable sparsity of adversarial perturbations.

A Robust Mean-Field Actor-Critic Reinforcement Learning Against Adversarial Perturbations on Agent States.

Multiagent deep reinforcement learning (DRL) makes optimal decisions dependent on system states observed by agents, but any uncertainty on the observations may mislead agents to take wrong actions. The mean-field actor-critic (MFAC) reinforcement learning is well-known in the multiagent field since it can effectively handle a scalability problem. However, it is sensitive to state perturbations that can significantly degrade the team rewards. This work proposes a Robust MFAC (RoMFAC) reinforcement learning that has two innovations: 1) a new objective function of training actors, composed of a policy gradient function that is related to the expected cumulative discount reward on sampled clean states and an action loss function that represents the difference between actions taken on clean and adversarial states and 2) a repetitive regularization of the action loss, ensuring the trained actors to obtain excellent performance. Furthermore, this work proposes a game model named a state-adversarial stochastic game (SASG). Despite the Nash equilibrium of SASG may not exist, adversarial perturbations to states in the RoMFAC are proven to be defensible based on SASG. Experimental results show that RoMFAC is robust against adversarial perturbations while maintaining its competitive performance in environments without perturbations.

ATVis: Understanding and diagnosing adversarial training processes through visual analytics

Adversarial training has emerged as a major strategy against adversarial perturbations in deep neural networks, which mitigates the issue of exploiting model vulnerabilities to generate incorrect predictions. Despite enhancing robustness, adversarial training often results in a trade-off with standard accuracy on normal data, a phenomenon that remains a contentious issue. In addition, the opaque nature of deep neural network models renders it more difficult to inspect and diagnose how adversarial training processes evolve. This paper introduces ATVis, a visual analytics framework for examining and diagnosing adversarial training processes. Through multi-level visualization design, ATVis enables the examination of model robustness from various granularity, facilitating a detailed understanding of the dynamics in the training epochs. The framework reveals the complex relationship between adversarial robustness and standard accuracy, which further offers insights into the mechanisms that drive the trade-offs observed in adversarial training. The effectiveness of the framework is demonstrated through case studies.

New Adversarial Image Detection Based on Sentiment Analysis.

Deep neural networks (DNNs) are vulnerable to adversarial examples, while adversarial attack models, e.g., DeepFool, are on the rise and outrunning adversarial example detection techniques. This article presents a new adversarial example detector that outperforms state-of-the-art detectors in identifying the latest adversarial attacks on image datasets. Specifically, we propose to use sentiment analysis for adversarial example detection, qualified by the progressively manifesting impact of an adversarial perturbation on the hidden-layer feature maps of a DNN under attack. Accordingly, we design a modularized embedding layer with the minimum learnable parameters to embed the hidden-layer feature maps into word vectors and assemble sentences ready for sentiment analysis. Extensive experiments demonstrate that the new detector consistently surpasses the state-of-the-art detection algorithms in detecting the latest attacks launched against ResNet and Inception neutral networks on the CIFAR-10, CIFAR-100, and SVHN datasets. The detector only has about 2 million parameters and takes less than 4.6 ms to detect an adversarial example generated by the latest attack models using a Tesla K80 GPU card.

Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks.

Despite providing high-performance solutions for computer vision tasks, the deep neural network (DNN) model has been proved to be extremely vulnerable to adversarial attacks. Current defense mainly focuses on the known attacks, but the adversarial robustness to the unknown attacks is seriously overlooked. Besides, commonly used adaptive learning and fine-tuning technique is unsuitable for adversarial defense since it is essentially a zero-shot problem when deployed. Thus, to tackle this challenge, we propose an attack-agnostic defense method named Meta Invariance Defense (MID). Specifically, various combinations of adversarial attacks are randomly sampled from a manually constructed Attacker Pool to constitute different defense tasks against unknown attacks, in which a student encoder is supervised by multi-consistency distillation to learn the attack-invariant features via a meta principle. The proposed MID has two merits: 1) Full distillation from pixel-, feature- and prediction-level between benign and adversarial samples facilitates the discovery of attack-invariance. 2) The model simultaneously achieves robustness to the imperceptible adversarial perturbations in high-level image classification and attack-suppression in low-level robust image regeneration. Theoretical and empirical studies on numerous benchmarks such as ImageNet verify the generalizable robustness and superiority of MID under various attacks.

Analytical Bounds on the Local Lipschitz Constants of ReLU Networks.

In this article, we determine analytical upper bounds on the local Lipschitz constants of feedforward neural networks with rectified linear unit (ReLU) activation functions. We do so by deriving Lipschitz constants and bounds for ReLU, affine-ReLU, and max pooling functions, and combining the results to determine a network-wide bound. Our method uses several insights to obtain tight bounds, such as keeping track of the zero elements of each layer, and analyzing the composition of affine and ReLU functions. Furthermore, we employ a careful computational approach which allows us to apply our method to large networks such as AlexNet and VGG-16. We present several examples using different networks, which show how our local Lipschitz bounds are tighter than the global Lipschitz bounds. We also show how our method can be applied to provide adversarial bounds for classification networks. These results show that our method produces the largest known bounds on minimum adversarial perturbations for large networks such as AlexNet and VGG-16.

On the Effectiveness of Adversarial Training Against Backdoor Attacks.

Although adversarial training (AT) is regarded as a potential defense against backdoor attacks, AT and its variants have only yielded unsatisfactory results or have even inversely strengthened backdoor attacks. The large discrepancy between expectations and reality motivates us to thoroughly evaluate the effectiveness of AT against backdoor attacks across various settings for AT and backdoor attacks. We find that the type and budget of perturbations used in AT are important, and AT with common perturbations is only effective for certain backdoor trigger patterns. Based on these empirical findings, we present some practical suggestions for backdoor defense, including relaxed adversarial perturbation and composite AT. This work not only boosts our confidence in AT's ability to defend against backdoor attacks but also provides some important insights for future research.

Enhancing Security and Performance in Vehicular Adhoc Networks: A Machine Learning Approach to Combat Adversarial Attacks

Integrating Machine Learning (ML) techniques into Vehicular Adhoc Networks (VANETs) provides promising features in autonomous driving and ITS applications. In this paper, DSRC data is used to evaluate the effectiveness of different ML models, including Naive Bayes, Random Forest, KNN, and Gradient Boosting, in normal and adversarial scenarios. Since the dataset is relatively imbalanced, the Synthetic Minority Over-sampling Technique (SMOTE) is employed for sampling, and defensive distillation for improving model resilience to adversarial perturbations. From the results, it is clear that models such as Gradient Boosting and Random Forest show high accuracy in both cases, thus showing the potential of using Machine Learning to improve VANET security and reliability when new threats appear. Through this research, the significance of the application of ML in the protection of vehicular communication in order to enhance both traffic safety and flow has been articulated.

Rethinking the validity of perturbation in single-step adversarial training

The neural network model has the drawback of making incorrect predictions under the influence of slight adversarial perturbations. Single-step adversarial training (AT) is an effective tool to bring adversarial robustness for the model to resist such attack. From the perspective of perturbation setting in training, we identify a conflict between the pursuit of greater robustness and the need to prevent catastrophic overfitting within the AT framework. To get out of this dilemma, we delve into the impact of perturbations on human visual perception. Our analysis reveals that examples containing more misleading features should be assigned a smaller perturbation magnitude to preserve subtle yet significant features. Conversely, examples encompassing more relevant features should be assigned a larger perturbation magnitude, enabling the model to adapt to stronger attacks effectively. Motivated by these insights, we propose a concise refinement to the AT framework to unleash its full potential for single-step AT. Instead of employing a fixed perturbation magnitude, we introduce a “band” of magnitudes, allowing each example to select an appropriate magnitude based on its visual characteristic. Through extensive experiments conducted on three datasets, we demonstrate the efficacy of our proposed strategy. Our approach not only improves the model’s robustness and prevents catastrophic overfitting but also effectively mitigates robust overfitting—an issue that has remained unresolved in the context of single-step AT, marking a significant advancement in the field.

Robust Remote Sensing Scene Interpretation Based on Unsupervised Domain Adaptation

Deep learning models excel in interpreting the exponentially growing amounts of remote sensing data; however, they are susceptible to deception and spoofing by adversarial samples, posing catastrophic threats. The existing methods to combat adversarial samples have limited performance in robustness and efficiency, particularly in complex remote sensing scenarios. To tackle these challenges, an unsupervised domain adaptation algorithm is proposed for the accurate identification of clean images and adversarial samples by exploring a robust generative adversarial classification network that can harmonize the features between clean images and adversarial samples to minimize distribution discrepancies. Furthermore, linear polynomial loss as a replacement for cross-entropy loss is integrated to guide robust representation learning. Additionally, we leverage the fast gradient sign method (FGSM) and projected gradient descent (PGD) algorithms to generate adversarial samples with varying perturbation amplitudes to assess model robustness. A series of experiments was performed on the RSSCN7 dataset and SIRI-WHU dataset. Our experimental results illustrate that the proposed algorithm performs exceptionally well in classifying clean images while demonstrating robustness against adversarial perturbations.