Adversarial Samples Research Articles

Adversarial Camera Patch: An Effective and Robust Physical-World Attack on Object Detectors

Physical adversarial attacks present a novel and growing challenge in cybersecurity, especially for systems reliant on physical inputs for Deep Neural Networks (DNNs), such as those found in Internet of Things (IoT) devices. They are vulnerable to physical adversarial attacks where real-world objects or environments are manipulated to mislead DNNs, thereby threatening the operational integrity and security of IoT devices. The camera-based attacks are one of the most practical adversarial attacks, which are easy to implement and more robust than all the other attack methods, and pose a big threat to the security of IoT. This paper proposes Adversarial Camera Patch (ADCP), a novel approach that employs a single-camera patch to launch robust physical adversarial attacks against object detectors. ADCP optimizes the physical parameters of the camera patch using Particle Swarm Optimization (PSO) to identify the most adversarial configuration. The optimized camera patch is then attached to the lens to generate stealthy and robust adversarial samples physically. The effectiveness of the proposed approach is validated through ablation experiments in a digital environment, with experimental results demonstrating its effectiveness even under worst-case scenarios (minimal width, maximum transparency). Notably, ADCP exhibits higher robustness in both digital and physical domains compared to the baseline. Given the simplicity, robustness, and stealthiness of ADCP, we advocate for attention towards the ADCP framework as it offers a means to achieve streamlined, robust, and stealthy physical attacks. Our adversarial attacks pose new challenges and requirements for cybersecurity.

A two-stage frequency-domain generation algorithm based on differential evolution for black-box adversarial samples

Adversarial sample generation problem is a hot issue in the security field of deep learning. Evolutionary algorithm has been widely used to solve this problem in recent years because of its good global search ability. However, existing methods still suffer from the “curse of dimensionality” when attacking high-resolution images. In this paper, a two-stage frequency domain generation algorithm of black-box adversarial samples based on differential evolution is proposed. In the first stage, a representative image-guided differential evolution method is proposed to quickly generate a universal adversarial perturbation with a high attack success rate in the frequency-domain. In the second stage, a space reduction strategy based on frequency-domain pixel blocks is designed to reduce the search space and alleviate the problem of “curse of dimensionality”. In addition, a new space–frequency interaction sensitivity measure is introduced to evaluate the similarity between the adversarial samples and the original images. The adversarial perturbations obtained by the measure are more in line with the subjective perception of the human eye. Finally, compared with several typical black-box adversarial sample generation algorithms, experimental results show that the proposed algorithm can achieve higher attack success rate with less prediction times.

Adversarial Attacks on Medical Segmentation Model via Transformation of Feature Statistics

Deep learning-based segmentation models have made a profound impact on medical procedures, with U-Net based computed tomography (CT) segmentation models exhibiting remarkable performance. Yet, even with these advances, these models are found to be vulnerable to adversarial attacks, a problem that equally affects automatic CT segmentation models. Conventional adversarial attacks typically rely on adding noise or perturbations, leading to a compromise between the success rate of the attack and its perceptibility. In this study, we challenge this paradigm and introduce a novel generation of adversarial attacks aimed at deceiving both the target segmentation model and medical practitioners. Our approach aims to deceive a target model by altering the texture statistics of an organ while retaining its shape. We employ a real-time style transfer method, known as the texture reformer, which uses adaptive instance normalization (AdaIN) to change the statistics of an image’s feature.To induce transformation, we modify the AdaIN, which typically aligns the source and target image statistics. Through rigorous experiments, we demonstrate the effectiveness of our approach. Our adversarial samples successfully pass as realistic in blind tests conducted with physicians, surpassing the effectiveness of contemporary techniques. This innovative methodology not only offers a robust tool for benchmarking and validating automated CT segmentation systems but also serves as a potent mechanism for data augmentation, thereby enhancing model generalization. This dual capability significantly bolsters advancements in the field of deep learning-based medical and healthcare segmentation models.

A distortion model guided adversarial surrogate for recaptured document detection

Due to the ever-growing need for e-business, document presentation attack detection (DPAD) is an important forensic task. With deep learning, the performances of DPAD methods have been improved significantly. However, the cross-domain performance under different types of document images is not yet satisfactory. In this work, we propose to study the document recapturing distortion model (DM), which is employed in guiding the training of an end-to-end surrogate distortion model. This surrogate model is incorporated into the DPAD scheme in an adversarial training fashion, yielding the proposed Adversarial Surrogate-based DPAD (AS-DPAD) scheme. The surrogate model actively generates adversarial samples with recapturing distortions that confuse the DPAD model. Meanwhile, the DPAD backbone learns a latent space that considers the distances between the generated and real recaptured samples to achieve a better generalization performance. A challenging cross-domain protocol is conducted in our experiment. The results confirm that our DM improves the efficacy of the trained surrogate distortion model and also validate that the proposed adversarial training strategy leads to a significant performance gain in the evaluation of document images with different contents.

AdverSPAM: Adversarial SPam Account Manipulation in Online Social Networks

In recent years, the widespread adoption of Machine Learning (ML) at the core of complex IT systems has driven researchers to investigate the security and reliability of ML techniques. A very specific kind of threats concerns the adversary mechanisms through which an attacker could induce a classification algorithm to provide the desired output. Such strategies, known as Adversarial Machine Learning (AML), have a twofold purpose: to calculate a perturbation to be applied to the classifier’s input such that the outcome is subverted, while maintaining the underlying intent of the original data. Although any manipulation that accomplishes these goals is theoretically acceptable, in real scenarios perturbations must correspond to a set of permissible manipulations of the input, which is rarely considered in the literature. In this article, we present AdverSPAM , an AML technique designed to fool the spam account detection system of an Online Social Network (OSN). The proposed black-box evasion attack is formulated as an optimization problem that computes the adversarial sample while maintaining two important properties of the feature space, namely statistical correlation and semantic dependency . Although being demonstrated in an OSN security scenario, such an approach might be applied in other context where the aim is to perturb data described by mutually related features. Experiments conducted on a public dataset show the effectiveness of AdverSPAM compared to five state-of-the-art competitors, even in the presence of adversarial defense mechanisms.

Improving Adversarial Robustness of Ensemble Classifiers by Diversified Feature Selection and Stochastic Aggregation

Learning-based classifiers are found to be vulnerable to attacks by adversarial samples. Some works suggested that ensemble classifiers tend to be more robust than single classifiers against evasion attacks. However, recent studies have shown that this is not necessarily the case under more realistic settings of black-box attacks. In this paper, we propose a novel ensemble approach to improve the robustness of classifiers against evasion attacks by using diversified feature selection and a stochastic aggregation strategy. Our proposed scheme includes three stages. Firstly, the adversarial feature selection algorithm is used to select a feature each time that can trade-offbetween classification accuracy and robustness, and add it to the feature vector bank. Secondly, each feature vector in the bank is used to train a base classifier and is added to the base classifier bank. Finally, m classifiers from the classifier bank are randomly selected for decision-making. In this way, it can cause each classifier in the base classifier bank to have good performance in terms of classification accuracy and robustness, and it also makes it difficult to estimate the gradients of the ensemble accurately. Thus, the robustness of classifiers can be improved without reducing the classification accuracy. Experiments performed using both Linear and Kernel SVMs on genuine datasets for spam filtering, malware detection, and handwritten digit recognition demonstrate that our proposed approach significantly improves the classifiers’ robustness against evasion attacks.

Enhancing CT Segmentation Security against Adversarial Attack: Most Activated Filter Approach

This study introduces a deep-learning-based framework for detecting adversarial attacks in CT image segmentation within medical imaging. The proposed methodology includes analyzing features from various layers, particularly focusing on the first layer, and utilizing a convolutional layer-based model with specialized training. The framework is engineered to differentiate between tampered adversarial samples and authentic or noise-altered images, focusing on attack methods predominantly utilized in the medical sector. A significant aspect of the approach is employing a random forest algorithm as a binary classifier to detect attacks. This method has shown efficacy in identifying genuine samples and reducing false positives due to Gaussian noise. The contributions of this work include robust attack detection, layer-specific feature analysis, comprehensive evaluations, physician-friendly visualizations, and distinguishing between adversarial attacks and noise. This research enhances the security and reliability of CT image analysis in diagnostics.

Adversarial Patch Attack Method for Vehicle Target Detection

Due to the susceptibility of intelligent recognition systems to adversarial sample attacks, research on adversarial sample attack methods for target detection is of great significance for improving the safety of vehicle autonomous driving. By studying the transfer-based self-ensemble attack adversarial sample generation algorithm, and based on the trained Faster R-CNN and YOLOX white box target detectors, a cross-model and cross-instance vehicle target detection attack method was designed. Through training, a universal adversarial patch was obtained, which achieved recognition attacks on two types of target detectors and three types of vehicles, resulting in incorrect recognition results being output. Through testing, the average decrease in mAP was about 41.6%, The research results have certain reference significance for the study of autonomous driving safety.

Rethinking Cost-Sensitive Classification in Deep Learning via Adversarial Data Augmentation

Cost-sensitive classification is critical in applications where misclassification errors widely vary in cost. However, overparameterization poses fundamental challenges to the cost-sensitive modeling of deep neural networks (DNNs). The ability of a DNN to fully interpolate a training data set can render a DNN, evaluated purely on the training set, ineffective in distinguishing a cost-sensitive solution from its overall accuracy maximization counterpart. This necessitates rethinking cost-sensitive classification in DNNs. To address this challenge, this paper proposes a cost-sensitive adversarial data augmentation (CSADA) framework to make overparameterized models cost sensitive. The overarching idea is to generate targeted adversarial examples that push the decision boundary in cost-aware directions. These targeted adversarial samples are generated by maximizing the probability of critical misclassifications and used to train a model with more conservative decisions on costly pairs. Experiments on well-known data sets and a pharmacy medication image (PMI) data set, made publicly available, show that our method can effectively minimize the overall cost and reduce critical errors while achieving comparable overall accuracy. Funding: Research reported in this publication was supported by the National Library of medicine of the National Institutes of Health in the United States under award number R01LM013624. Data Ethics & Reproducibility Note: This paper abides by data ethics requirements. Data used are publicly available online at https://deepblue.lib.umich.edu/data/concern/data_sets/6d56zw997 . Codes to replicate the results of this paper are available at https://doi.org/10.24433/CO.2139841.v1 .

Research on Parkinson's disease recognition algorithm based on sample enhancement

Parkinson's disease patients have early vocal cord damage, and their voiceprint characteristics differ significantly from those of healthy individuals, which can be used to identify Parkinson's disease. However, the samples of the voiceprint dataset of Parkinson's disease patients are insufficient, so this paper proposes a double self-attention deep convolutional generative adversarial network model for sample enhancement to generate high-resolution spectrograms, based on which deep learning is used to recognize Parkinson's disease. This model improves the texture clarity of samples by increasing network depth and combining gradient penalty and spectral normalization techniques, and a family of pure convolutional neural networks (ConvNeXt) classification network based on Transfer learning is constructed to extract voiceprint features and classify them, which improves the accuracy of Parkinson's disease recognition. The validation experiments of the effectiveness of this paper's algorithm are carried out on the Parkinson's disease speech dataset. Compared with the pre-sample enhancement, the clarity of the samples generated by the proposed model in this paper as well as the Fréchet inception distance (FID) are improved, and the network model in this paper is able to achieve an accuracy of 98.8%. The results of this paper show that the Parkinson's disease recognition algorithm based on double self-attention deep convolutional generative adversarial network sample enhancement can accurately distinguish between healthy individuals and Parkinson's disease patients, which helps to solve the problem of insufficient samples for early recognition of voiceprint data in Parkinson's disease. In summary, the method effectively improves the classification accuracy of small-sample Parkinson's disease speech dataset and provides an effective solution idea for early Parkinson's disease speech diagnosis.

Improvements to adversarial training for text classification tasks

Although deep learning models show powerful performance, they are still easily deceived by adversarial samples. Some methods for generating adversarial samples have the drawback of high time loss, which is problematic for adversarial training, and the existing adversarial training methods are difficult to adapt to the dynamic nature of the model, so it is still challenging to study an efficient adversarial training method. In this paper, we propose an adversarial training method, the core of which is the improved adversarial sample generation method AGFAT for adversarial training and the improved dynamic adversarial training method AGFAT-DAT. AGFAT uses a word frequency-based approach to identify significant words, filter replacement candidates, and use an efficient semantic constraint module as a means to reduce the time of adversarial sample generation; AGFAT-DAT is a dynamic adversarial training approach that uses a cyclic attack on the model after adversarial training and generates adversarial samples for adversarial training again. It is demonstrated that the proposed method can significantly reduce the generation time of adversarial samples, and the adversarial-trained model can also effectively defend against other types of word-level adversarial attacks.

Transferable adversarial sample purification by expanding the purification space of diffusion models

Transferable adversarial sample purification by expanding the purification space of diffusion models

Robust Charging Network Planning for Metropolitan Taxi Fleets

We study the robust charging station location problem for a large-scale commercial taxi fleet. Vehicles within the fleet coordinate on charging operations but not on customer acquisition. We decide on a set of charging stations to open to ensure operational feasibility. To make this decision, we propose a novel solution method situated between the location routing problems with intraroute facilities and flow refueling location problems. Additionally, we introduce a problem variant that makes a station sizing decision. Using our exact approach, charging stations for a robust operation of citywide taxi fleets can be planned. We develop a deterministic core problem employing a cutting plane method for the strategic problem and a branch-and-price decomposition for the operational problem. We embed this problem into a robust solution framework based on adversarial sampling, which allows for planner-selectable risk tolerance. We solve instances derived from real-world data of the metropolitan area of Munich containing 1,000 vehicles and 60 potential charging station locations. Our investigation of the sensitivity of technological developments shows that increasing battery capacities shows a more favorable impact on vehicle feasibility of up to 10 percentage points compared with increasing charging speeds. Allowing for depot charging dominates both of these options. Finally, we show that allowing just 1% of operational infeasibility risk lowers infrastructure costs by 20%. Funding: This work was partially funded by the Deutsche Forschungsgemeinschaft [Project 277991500]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/trsc.2022.0207 .

Adversarial sample attacks and defenses based on LSTM-ED in industrial control systems

The challenge faced by industrial control systems is that they are vulnerable to adversarial sample attacks. In the ICS field, the challenge with adversarial sample attacks is that the adversarial samples generated by the attack do not conform to protocol specifications. The challenge of adversarial sample defense is that it is difficult to design a defense model without information about the adversarial samples. To tackle these challenges, we propose an adversarial sample attack and defense method based on Long Short-Term Memory Networks based Encoder-Decoder (LSTM-ED). The objectives are to address challenges of adversarial samples not conforming to protocol specifications and physical meaning, inefficient generation of adversarial samples, and the difficulty of designing a defense model without information about adversarial samples. Our adversarial sample attack efficiently generates samples conforming to protocol specifications and physical meaning by adding perturbation values to sensors and actuators, while complying with feature constraints. Subsequently, we introduce an LSTM-ED Feature Weight defense method (LSTM-FWED) designed without explicit adversarial sample information. In LSTM-FWED, we normalize reconstruction errors across different features to prevent anomaly scores from being influenced by poorly predicted features, thereby ensuring robust defense results. We validate the effectiveness of our approach on a real-world critical infrastructure testbed. The proposed adversarial sample attack reduces the precision of the LSTM-ED model by an average of 66.26%, with a maximum adversarial sample generation time of 18 seconds, significantly improving attack efficiency. Furthermore, in comprehensive experiments, LSTM-FWED demonstrates an average AUC improvement of 21.83% compared to state-of-the-art anomaly detection baseline methods.

Research on Privacy Protection in Federated Learning Combining Distillation Defense and Blockchain

Traditional federated learning addresses the data security issues arising from the need to centralize client datasets on a central server for model training. However, this approach still poses privacy protection risks. For instance, central servers cannot verify privacy leaks resulting from poisoning attacks by malicious clients. Additionally, adversarial sample attacks can infer specific samples from the original data by testing the local models on client devices. This paper proposes a federated learning privacy protection method combining distillation defense technology with blockchain architecture. The method utilizes distillation defense technology to reduce the sensitivity of client devices participating in federated learning to perturbations and enhance their ability to resist adversarial sample attacks locally. This not only reduces communication overhead and improves learning efficiency but also enhances the model’s generalization ability. Furthermore, the method leverages the “decentralized” nature of blockchain architecture as a trusted record-keeping mechanism to audit information interactions among clients and shared model parameters. This addresses privacy leakage issues resulting from poisoning attacks by some clients during the model construction process. Simulation experiment results demonstrate that the proposed method, compared with traditional federated learning, ensures model convergence, detects malicious clients, and improves the participation level of highly reputable clients. Moreover, by reducing the sensitivity of local clients to perturbations, it enhances their ability to effectively resist adversarial sample attacks.

DeepInsight: Topology Changes Assisting Detection of Adversarial Samples on Graphs

With the rapid development of artificial intelligence, a number of machine learning algorithms, such as graph neural networks (GNNs), have been proposed to facilitate network analysis or graph data mining. Although effective, recent studies show that these advanced methods may suffer from adversarial attacks, i.e., they may lose effectiveness when only a small fraction of links are unexpectedly changed. This article investigates three well-known adversarial attack methods, i.e., Nettack, Meta Attack, and GradArgmax. It is found that different attack methods have their specific attack preferences on changing the target network structures. Such attack patterns are further verified by experimental results on some real-world networks, revealing that, generally, the top-4 most important network attributes on detecting adversarial samples suffice to explain the preference of an attack method. Based on these findings, the network attributes are utilized to design machine learning models for adversarial sample detection and attack method recognition with outstanding performance.

An ensemble learning method with GAN-based sampling and consistency check for anomaly detection of imbalanced data streams with concept drift.

A challenge to many real-world data streams is imbalance with concept drift, which is one of the most critical tasks in anomaly detection. Learning nonstationary data streams for anomaly detection has been well studied in recent years. However, most of the researches assume that the class of data streams is relatively balanced. Only a few approaches tackle the joint issue of imbalance and concept drift. To overcome this joint issue, we propose an ensemble learning method with generative adversarial network-based sampling and consistency check (EGSCC) in this paper. First, we design a comprehensive anomaly detection framework that includes an oversampling module by generative adversarial network, an ensemble classifier, and a consistency check module. Next, we introduce double encoders into GAN to better capture the distribution characteristics of imbalanced data for oversampling. Then, we apply the stacking ensemble learning to deal with concept drift. Four base classifiers of SVM, KNN, DT and RF are used in the first layer, and LR is used as meta classifier in second layer. Last but not least, we take consistency check of the incremental instance and check set to determine whether it is anormal by statistical learning, instead of threshold-based method. And the validation set is dynamic updated according to the consistency check result. Finally, three artificial data sets obtained from Massive Online Analysis platform and two real data sets are used to verify the performance of the proposed method from four aspects: detection performance, parameter sensitivity, algorithm cost and anti-noise ability. Experimental results show that the proposed method has significant advantages in anomaly detection of imbalanced data streams with concept drift.

Cybersecurity Threat Detection using Machine Learning and Network Analysis

Cybercriminals continually develop innovative strategies to confound and frustrate their victims, necessitating constant vigilance to protect the availability, confidentiality, and integrity of digital systems. Machine learning (ML) has emerged as a powerful technique for intelligent cyber analysis, enabling proactive defenses by studying recurring patterns of successful attacks. However, two significant drawbacks hinder the widespread adoption of ML in security analysis: high computing overheads and the need for specialized frameworks. This study aims to quantify the extent to which a hub can enhance ecosystem safety. Typical cyberattacks were executed on an Internet of Things (IoT) network within a smart house to validate the hub's efficacy. Furthermore, the resistance of the intrusion detection system (IDS) to adversarial machine learning (AML) attacks was investigated, where models are targeted with adversarial samples exploiting weaknesses in the pre-trained detector.

Cybersecurity Threat Detection using Machine Learning and Network Analysis

Cybercriminals continually develop innovative strategies to confound and frustrate their victims, necessitating constant vigilance to protect the availability, confidentiality, and integrity of digital systems. Machine learning (ML) has emerged as a powerful technique for intelligent cyber analysis, enabling proactive defenses by studying recurring patterns of successful attacks. However, two significant drawbacks hinder the widespread adoption of ML in security analysis: high computing overheads and the need for specialized frameworks. This study aims to quantify the extent to which a hub can enhance ecosystem safety. Typical cyberattacks were executed on an Internet of Things (IoT) network within a smart house to validate the hub's efficacy. Furthermore, the resistance of the intrusion detection system (IDS) to adversarial machine learning (AML) attacks was investigated, where models are targeted with adversarial samples exploiting weaknesses in the pre-trained detector.

An effective deep learning adversarial defense method based on spatial structural constraints in embedding space

Deep neural networks are highly vulnerable to adversarial samples. Most existing adversarial defense methods do not consider the distribution of adversarial samples. We argue that very few adversarial samples in the natural sample set prevent the deep neural networks from learning a complete and effective representation of the adversarial samples. This causes the spatial structures between the natural and the adversarial samples to be vastly different from that of the input space, thus making the models vulnerable to adversarial attacks. Based on this viewpoint, this paper proposes an effective deep-learning adversarial defense method, which incorporates information about the spatial structures of the natural and the adversarial samples in the embedding space during the training process. This proposed approach improves the deep learning model’s generalization to new adversarial samples and achieves the purpose of defending against adversarial attacks. Four deep neural networks with different scales are used and experimentally verified on four typical publicly available image data. The experimental results show that our method effectively improves the defense ability of deep learning models against adversarial attacks.