Attack Algorithm Research Articles

Adversarial attack and training for deep neural network based power quality disturbance classification

Power quality disturbance (PQD) can significantly affect the normal operation of the power system. Deep neural network (DNN) can classify PQD with extremely high accuracy. However, adversarial attacks can drastically degrade the performance of DNN, which raises security issues for DNN-based PQD classification. At present, there are few researches on adversarial attacks and their defenses in PQD classification. In this study, we first adopt convolutional neural network-long short-term memory (CNN-LSTM) to classify PQD signals. Then we propose an adversarial attack algorithm for PQD classification, i.e., momentum iterative-fast gradient sign method (MI-FGSM). MI-FGSM generates adversarial perturbations along the gradient direction of the loss function by incorporating momentum during iterations. Finally, we propose a defense algorithm to defend against adversarial attacks, i.e., iterative adversarial training (IAT). IAT can improve the robustness of the classification model by making it learn to minimize the worst loss caused by multi-class adversarial perturbation strengths. The experimental results demonstrate that compared with the fast gradient sign method, the MI-FGSM produces smaller perturbations and can further reduce the classification accuracy of CNN-LSTM. In addition, compared with the most advanced adversarial training defense algorithm in PQD classification, IAT can effectively enhance the robustness of the CNN-LSTM against adversarial attacks.

Enhancing the Transferability of Adversarial Examples Based on Nesterov Momentum for Recommendation Systems

The attacker's malicious behavior of injecting well-designed adversarial examples (i.e. fake users) into recommender systems will severely affect the security of systems. It's difficult to fully obtain details of victim recommendation models (i.e. black-box model) in practical recommendation scenarios, using the transferability of adversarial examples to achieve black-box attacks is still an effective way. At present, adversarial examples generated by existing gradient-based methods are prone to drop into local minima, making it impossible to achieve the expected attack effect and reducing the transferability. In this paper, we propose an attack algorithm that enhances the transferability of adversarial examples based on the Nesterov Momentum for Recommendation Systems (ETANRS). With white-box recommendation surrogate models, we utilize Nesterov momentum to generate better adversarial examples, then inject them into black-box victim models to attack. We utilize the accumulated gradients and pre-determine the update direction of the gradients to keep the optimal value from being lost, thus enhancing the transferability of the adversarial examples. Experimental results demonstrate that our method is better than state-of-the-art gradient-based attack algorithms, which affect recommendation performance.

Exploring the Physical-World Adversarial Robustness of Vehicle Detection

Adversarial attacks can compromise the robustness of real-world detection models. However, evaluating these models under real-world conditions poses challenges due to resource-intensive experiments. Virtual simulations offer an alternative, but the absence of standardized benchmarks hampers progress. Addressing this, we propose an innovative instant-level data generation pipeline using the CARLA simulator. Through this pipeline, we establish the Discrete and Continuous Instant-level (DCI) dataset, enabling comprehensive experiments involving three detection models and three physical adversarial attacks. Our findings highlight diverse model performances under adversarial conditions. YOLO v6 demonstrates remarkable resilience, exhibiting just a marginal 6.59% average drop in average precision (AP). In contrast, the ASA attack yields a substantial 14.51% average AP reduction, twice the effect of other algorithms. We also note that static scenes yield higher recognition AP values, and outcomes remain relatively consistent across varying weather conditions. Intriguingly, our study suggests that advancements in adversarial attack algorithms may be approaching its “limitation”. In summary, our work underscores the significance of adversarial attacks in real-world contexts and introduces the DCI dataset as a versatile benchmark. Our findings provide valuable insights for enhancing the robustness of detection models and offer guidance for future research endeavors in the realm of adversarial attacks.

Revocable Signature Scheme with Implicit and Explicit Certificates.

This paper addresses the certificate revocation problem and proposes the first revocable pairing-based signature scheme with implicit and explicit certificates (IE-RCBS-kCAA). We should no longer discuss whether to revoke certificates but how to do it effectively, ensuring both the scalability of the revocation operation and the non-repudiation of the signature in the short or long term. Under the computational difficulty assumptions of the modified collusion attack algorithm with k traitors (k-mCAA) and discrete logarithm (DL) problems, we demonstrate that our scheme is secure against existential unforgeability under chosen message attacks (EUF-IERCBS-kCAA-CMA) in a random oracle model. The proposed solution is scaled and allows the use of many trusted status authorities that issue explicit short-term certificates confirming the validity of explicit long-term certificates. Furthermore, we demonstrate that our signature scheme has a short-term non-repudiation property for the shell validity model.

Resilient Distributed Classification Learning Against Label Flipping Attack: An ADMM-Based Approach

Distributed classification learning (DCL) is a promising solution to establish IoT-based smart applications, especially due to its strong ability in dealing with large-scale and high-concurrency data. However, the performance of DCL may be seriously affected by the label flipping attack (LFA). Regarding the LFA-resilient learning problem, most existing works are built in more centralized settings. The work addressing the secure DCL issue makes an assumption that the label flipping rates are symmetric and available for scheme design. In this paper, we remove this assumption and propose an LFA-resilient DCL scheme, named FENDER, without knowing the asymmetric flipping rates. The challenge is to guarantee both attack resilience and algorithm convergence. We carefully integrate a resilient loss and the ADMM scheme, making FENDER resilient to LFA. Further, we systematically analyze the performance of FENDER according to a metric reflecting the models obtained by all the servers at different iterations. In addition, we discuss and compare FENDER with some existing methods from the aspects of algorithm establishment and performance guarantee. Finally, extensive experiments with multiple real-world datasets are performed to validate the developed theory and evaluate the performance of the trained models.

On the Privacy of Counting Bloom Filters Under a Black-Box Attacker

Counting Bloom Filters (CBFs) are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">approximate</i> membership checking data structures, and it is normally believed that at most an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">approximate</i> reconstruction of the underlying set can be derived when interacting with a CBF. This paper decisively refutes this assumption. In a recent paper, we considered the privacy of CBFs when the attacker has access to the implementation details and thus, it sees the filter as a white-box. In that setting, we showed that the attacker may be able to extract the elements stored in the filter when the number of false positives over the entire universe is not significantly larger than the number of elements stored in the filter. In this work, we consider a black-box attacker that can only perform user interactions on the CBF to insert, remove and query elements with no knowledge of the filter implementation details. We show that even in this case, an attacker may be able to extract information from the filter at the cost of using more complex and time-consuming attack algorithms. The proposed algorithms have been implemented and compared with the white-box attack, showing that in most cases, almost the same information can be extracted from the filter.

Accessibility of covariance information creates vulnerability in Federated Learning frameworks.

Federated Learning (FL) is gaining traction in various fields as it enables integrative data analysis without sharing sensitive data, such as in healthcare. However, the risk of data leakage caused by malicious attacks must be considered. In this study, we introduce a novel attack algorithm that relies on being able to compute sample means, sample covariances, and construct known linearly independent vectors on the data owner side. We show that these basic functionalities, which are available in several established FL frameworks, are sufficient to reconstruct privacy-protected data. Additionally, the attack algorithm is robust to defense strategies that involve adding random noise. We demonstrate the limitations of existing frameworks and propose potential defense strategies analyzing the implications of using differential privacy. The novel insights presented in this study will aid in the improvement of FL frameworks. The code examples are provided at GitHub (https://github.com/manuhuth/Data-Leakage-From-Covariances.git). The CNSIM1 dataset, which we used in the manuscript, is available within the DSData R package (https://github.com/datashield/DSData/tree/main/data).

Adversarial Attack and Defense in Breast Cancer Deep Learning Systems.

Deep-learning-assisted medical diagnosis has brought revolutionary innovations to medicine. Breast cancer is a great threat to women's health, and deep-learning-assisted diagnosis of breast cancer pathology images can save manpower and improve diagnostic accuracy. However, researchers have found that deep learning systems based on natural images are vulnerable to attacks that can lead to errors in recognition and classification, raising security concerns about deep systems based on medical images. We used the adversarial attack algorithm FGSM to reveal that breast cancer deep learning systems are vulnerable to attacks and thus misclassify breast cancer pathology images. To address this problem, we built a deep learning system for breast cancer pathology image recognition with better defense performance. Accurate diagnosis of medical images is related to the health status of patients. Therefore, it is very important and meaningful to improve the security and reliability of medical deep learning systems before they are actually deployed.

A variable adversarial attack method based on filtering

Previous works have pointed out that deep learning models are vulnerable to adversarial examples crafted by adding negligible confusing noise to clean images. However, adversarial examples are hard to transfer to attack other defense models or do not conform to human-imperceptible under the black-box setting. To bridge this gap, a variable adversarial attack method based on filtering called VFI-FGSM is proposed. Variable Step Size Method (VSSM) is designed to adaptively control perturbation to avoid overfitting, the Filtering based Iterative Fast Gradient Sign Method (FI-FGSM) is explored to enhance robustness of attack, an auxiliary loss of high-level representation is added to get a more precise gradient direction and improve the attack success rate, and a new tactic named the contribution for reducing perturbation size(CRPS) are presented to measure the performance of attack algorithms in the paper. The results show that scheme proposed increases the attack success rate by an average of 14.36% in ensemble attack method and 7.9% on combined trained model for defense models in black-box setting. Fortunately, the noise is perceptually small in both L1 and L2 metrics.

The Study and Implementation on Adversarial Attack Algorithm Based on Deep Convolutional Generative Adversarial Network

The field of machine learning has been growing rapidly in recent years, with significant advancements in computer vision. However, this progress has also led to increased concerns over the safety and security of machine learning systems. Despite previous research efforts in this area, traditional adversarial algorithms continue to fall short in defending against attacks in most circumstances. As a response to this challenge, this study seeks to develop an adversarial algorithm based on Deep Convolutional Generative Adversarial Network (DCGAN), which aims to reconstruct the data distribution of input data and generate new data to improve model robustness. To evaluate the efficacy of the proposed method, a Deep Neural Network (DNN) classifier is employed to perform classification on the generated dataset. The experimental results suggest the potential feasibility of the proposed hypothesis, but further improvement is required to strengthen the defence mechanism. Overall, this study contributes to the ongoing efforts to enhance machine learning safety and security in practical applications.

Adaptive local adversarial attacks on 3D point clouds

Modern artificial intelligence systems rely heavily on deep learning techniques. However, deep neural networks are easily disturbed by adversarial objects. Adversarial examples are beneficial to improve the robustness of the 3D neural network model and enhance the stability of the artificial intelligence system. At present, most 3D adversarial attack methods perturb the entire point cloud to generate adversarial examples, which results in high perturbation costs and low operability in the physical world. In this paper, we propose an adaptive local adversarial attack method (AL-Adv) on 3D point clouds to generate adversarial point clouds. First, we analyze the vulnerability of the 3D network model and extract the salient regions of the input point cloud, namely the vulnerable regions. Second, we propose an adaptive gradient attack algorithm that targets salient regions. The proposed attack algorithm adaptively assigns different disturbances in different directions of the three-dimensional coordinates of the point cloud. Experimental results show that our proposed AL-Adv method achieves a higher attack success rate than the global attack method. Specifically, the adversarial examples generated by AL-Adv demonstrate good imperceptibility and small generation costs.

Reward poisoning attacks in deep reinforcement learning based on exploration strategies

In recent years, deep reinforcement learning (DRL) has been widely applied in various fields, and its vulnerability to adversarial samples, which is inherited from deep learning, has also received considerable attention for training robust DRL agents. However, most of existing attack approaches aim to perturb the observations of DRL agents, and poisoning attacks against DRL through perturbing the reward from the environment have been studied less. In this paper, we propose a novel reward poisoning adversarial attack method that employs the exploration strategy of reinforcement learning to help effectively attack a target. The adversary first builds a deep neural network to attain the exploration value of each state–action pair, i.e., the E-value, which is utilized to determine the appropriate time to attack the target and the sign of adversarial perturbation to generate adversary samples to the reward. The experimental results demonstrate that the reward poisoning attack algorithm based on E-values proposed in this paper enables an adversary to improve the efficiency of attacks with fewer adversarial samples.

A2SC: Adversarial Attacks on Subspace Clustering

Many studies demonstrate that supervised learning techniques are vulnerable to adversarial examples. However, adversarial threats in unsupervised learning have not drawn sufficient scholarly attention. In this article, we formally address the unexplored adversarial attacks in the equally important unsupervised clustering field and propose the concept of the adversarial set and adversarial set attack for clustering. To illustrate the basic idea, we design a novel adversarial space-mapping attack algorithm to confuse subspace clustering, one of the mainstream branches of unsupervised clustering. It maps a sample into one wrong class by moving it towards the closest point on the linear subspace of the target class, that is, along the normal of the closest point. This simple single-step algorithm has the power to craft the adversarial set where the image samples can be wrongly clustered, even into the targeted labels. Empirical results on different image datasets verify the effectiveness and superiority of our algorithm. We further show that deep supervised learning algorithms (such as VGG and ResNet) are also vulnerable to our crafted adversarial set, which illustrates the good cross-task transferability of the adversarial set.

Adversarial ink: componentwise backward error attacks on deep learning

Abstract Deep neural networks are capable of state-of-the-art performance in many classification tasks. However, they are known to be vulnerable to adversarial attacks—small perturbations to the input that lead to a change in classification. We address this issue from the perspective of backward error and condition number, concepts that have proved useful in numerical analysis. To do this, we build on the work of Beuzeville, T., Boudier, P., Buttari, A., Gratton, S., Mary, T. and Pralet S. (2021) Adversarial attacks via backward error analysis. hal-03296180, version 3. In particular, we develop a new class of attack algorithms that use componentwise relative perturbations. Such attacks are highly relevant in the case of handwritten documents or printed texts where, for example, the classification of signatures, postcodes, dates or numerical quantities may be altered by changing only the ink consistency and not the background. This makes the perturbed images look natural to the naked eye. Such ‘adversarial ink’ attacks therefore reveal a weakness that can have a serious impact on safety and security. We illustrate the new attacks on real data and contrast them with existing algorithms. We also study the use of a componentwise condition number to quantify vulnerability.

Similarity Distribution Based Membership Inference Attack on Person Re-identification

While person Re-identification (Re-ID) has progressed rapidly due to its wide real-world applications, it also causes severe risks of leaking personal information from training data. Thus, this paper focuses on quantifying this risk by membership inference (MI) attack. Most of the existing MI attack algorithms focus on classification models, while Re-ID follows a totally different training and inference paradigm. Re-ID is a fine-grained recognition task with complex feature embedding, and model outputs commonly used by existing MI like logits and losses are not accessible during inference. Since Re-ID focuses on modelling the relative relationship between image pairs instead of individual semantics, we conduct a formal and empirical analysis which validates that the distribution shift of the inter-sample similarity between training and test set is a critical criterion for Re-ID membership inference. As a result, we propose a novel membership inference attack method based on the inter-sample similarity distribution. Specifically, a set of anchor images are sampled to represent the similarity distribution conditioned on a target image, and a neural network with a novel anchor selection module is proposed to predict the membership of the target image. Our experiments validate the effectiveness of the proposed approach on both the Re-ID task and conventional classification task.

Measuring Re-identification Risk

Compact user representations (such as embeddings) form the backbone of personalization services. In this work, we present a new theoretical framework to measure re-identification risk in such user representations. Our framework, based on hypothesis testing, formally bounds the probability that an attacker may be able to obtain the identity of a user from their representation. As an application, we show how our framework is general enough to model important real-world applications such as the Chrome's Topics API for interest-based advertising. We complement our theoretical bounds by showing provably good attack algorithms for re-identification that we use to estimate the re-identification risk in the Topics API. We believe this work provides a rigorous and interpretable notion of re-identification risk and a framework to measure it that can be used to inform real-world applications.

Certified Robustness to Text Adversarial Attacks by Randomized [MASK

Abstract Very recently, few certified defense methods have been developed to provably guarantee the robustness of a text classifier to adversarial synonym substitutions. However, all the existing certified defense methods assume that the defenders have been informed of how the adversaries generate synonyms, which is not a realistic scenario. In this study, we propose a certifiably robust defense method by randomly masking a certain proportion of the words in an input text, in which the above unrealistic assumption is no longer necessary. The proposed method can defend against not only word substitution-based attacks, but also character-level perturbations. We can certify the classifications of over 50% of texts to be robust to any perturbation of five words on AGNEWS, and two words on SST2 dataset. The experimental results show that our randomized smoothing method significantly outperforms recently proposed defense methods across multiple datasets under different attack algorithms.

Robust Adversarial Attacks on Deep Learning-Based RF Fingerprint Identification

Deep learning (DL)-based radio frequency fingerprint identification (RFFI), despite its state-of-the-art capability in improving the security performance of communication networks, is still vulnerable to carefully crafted and imperceptible adversarial attack. However, conventional radio frequency (RF) adversarial attacks ignore the impact of the practical fading channels between the attacker and the sensor. In this letter, a generation adversarial perturbations (GAP) problem is formulated in a DL-based RFFI network. Both the case of ideally perfect channel state information (CSI) and the practical imperfect CSI are considered. The closed-form expression of the adversarial example based on the perfect CSI is provided. In order to address the intractable non-convex robust optimization problem under the imperfect CSI, a spoofing attack algorithm is proposed by exploiting the S-procedure. Simulation results demonstrate that our proposed scheme achieves a superior spoofing ratio compared with other benchmark schemes.

Physical Black-Box Adversarial Attacks Through Transformations

Deep learning has shown impressive performance in numerous applications. However, recent studies have found that deep learning models are vulnerable to adversarial attacks, where the attacker adds imperceptible perturbations into benign samples to induce misclassifications. Adversarial attacks in the digital domain focus on constructing imperceptible perturbations. However, they are always less effective in the physical world because the perturbations may be destroyed when captured by the camera. Most physical adversarial attacks require adding invisible adversarial features (e.g., a sticker or a laser) to the target object, which may be noticed by human eyes. In this work, we propose to employ image transformation to generate more natural adversarial samples in the physical world. Concretely, we propose two attack algorithms to satisfy different attack goals: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Efficient-AATR</i> employs a greedy strategy to generate adversarial samples with fewer queries; <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Effective-AATR</i> employs an adaptive particle swarm optimization algorithm to search for the most effective adversarial samples within the given the number of queries. Extensive experiments demonstrate the superiority of our attacks compared with state-of-the-art adversarial attacks under mainstream defenses.

Rethinking Label Flipping Attack: From Sample Masking to Sample Thresholding.

Nowadays, machine learning (ML) and deep learning (DL) methods have become fundamental building blocks for a wide range of AI applications. The popularity of these methods also makes them widely exposed to malicious attacks, which may cause severe security concerns. To understand the security properties of the ML/DL methods, researchers have recently started to turn their focus to adversarial attack algorithms that could successfully corrupt the model or clean data owned by the victim with imperceptible perturbations. In this paper, we study the Label Flipping Attack (LFA) problem, where the attacker expects to corrupt an ML/DL model's performance by flipping a small fraction of the labels in the training data. Prior art along this direction adopts combinatorial optimization problems, leading to limited scalability toward deep learning models. To this end, we propose a novel minimax problem which provides an efficient reformulation of the sample selection process in LFA. In the new optimization problem, the sample selection operation could be implemented with a single thresholding parameter. This leads to a novel training algorithm called Sample Thresholding. Since the objective function is differentiable and the model complexity does not depend on the sample size, we can apply Sample Thresholding to attack deep learning models. Moreover, since the victim's behavior is not predictable in a poisonous attack setting, we have to employ surrogate models to simulate the true model employed by the victim model. Seeing the problem, we provide a theoretical analysis of such a surrogate paradigm. Specifically, we show that the performance gap between the true model employed by the victim and the surrogate model is small under mild conditions. On top of this paradigm, we extend Sample Thresholding to the crowdsourced ranking task, where labels collected from the annotators are vulnerable to adversarial attacks. Finally, experimental analyses on three real-world datasets speak to the efficacy of our method.