Untargeted Attacks Research Articles

A Survey of Text Watermarking in the Era of Large Language Models

Text watermarking algorithms are crucial for protecting the copyright of textual content. Historically, their capabilities and application scenarios were limited. However, recent advancements in large language models (LLMs) have revolutionized these techniques. LLMs not only enhance text watermarking algorithms with their advanced abilities but also create a need for employing these algorithms to protect their own copyrights or prevent potential misuse. This work conducts a comprehensive survey of the current state of text watermarking technology, covering four main aspects: (1) an overview and comparison of different text watermarking techniques; (2) evaluation methods for text watermarking algorithms, including their detectability, impact on text or LLM quality, and robustness under target or untargeted attacks; (3) potential application scenarios for text watermarking technology; and (4) current challenges and future directions for text watermarking. This survey aims to provide researchers with a thorough understanding of text watermarking technology in the era of LLMs, thereby promoting its further advancement.

Multi-agent Attacks for Black-box Social Recommendations

The rise of online social networks has facilitated the evolution of social recommender systems, which incorporate social relations to enhance users’ decision-making process. With the great success of Graph Neural Networks (GNNs) in learning node representations, GNN-based social recommendations have been widely studied to model user-item interactions and user-user social relations simultaneously. Despite their great successes, recent studies have shown that these advanced recommender systems are highly vulnerable to adversarial attacks, in which attackers can inject well-designed fake user profiles to disrupt recommendation performances. While most existing studies mainly focus on targeted attacks to promote target items on vanilla recommender systems, untargeted attacks to degrade the overall prediction performance are less explored on social recommendations under a black-box scenario. To perform untargeted attacks on social recommender systems, attackers can construct malicious social relationships for fake users to enhance the attack performance. However, the coordination of social relations and item profiles is challenging for attacking black-box social recommendations. To address this limitation, we first conduct several preliminary studies to demonstrate the effectiveness of cross-community connections and cold-start items in degrading recommendations performance. Specifically, we propose a novel framework MultiAttack based on multi-agent reinforcement learning to coordinate the generation of cold-start item profiles and cross-community social relations for conducting untargeted attacks on black-box social recommendations. Comprehensive experiments on various real-world datasets demonstrate the effectiveness of our proposed attacking framework under the black-box setting.

On Feasibility of Intent Obfuscating Attacks

Intent obfuscation is a common tactic in adversarial situations, enabling the attacker to both manipulate the target system and avoid culpability. Surprisingly, it has rarely been implemented in adversarial attacks on machine learning systems. We are the first to propose using intent obfuscation to generate adversarial examples for object detectors: by perturbing another non-overlapping object to disrupt the target object, the attacker hides their intended target. We conduct a randomized experiment on 5 prominent detectors---YOLOv3, SSD, RetinaNet, Faster R-CNN, and Cascade R-CNN---using both targeted and untargeted attacks and achieve success on all models and attacks. We analyze the success factors characterizing intent obfuscating attacks, including target object confidence and perturb object sizes. We then demonstrate that the attacker can exploit these success factors to increase success rates for all models and attacks. Finally, we discuss main takeaways and legal repercussions. If you are reading the AAAI/ACM version, please download the technical appendix on arXiv at https://arxiv.org/abs/2408.02674

OSSIntegrity: Collaborative open-source code integrity verification

Open-source software (OSS) libraries have become popular among developers due to their ability to reduce development time and costs. However, OSS can also be exploited and used as a means of conducting OSS supply chain attacks. In OSS attacks, malicious code is injected into libraries used by the target. Previous studies have proposed various methods for preventing and detecting such attacks, however most of them focused on untargeted attacks. In contrast, this paper focuses on targeted OSS supply chain attacks which are performed by skilled and persistent attackers with strong technical aptitude. Targeted OSS attacks are crafted towards a specific target (i.e., developer). Since these attacks do not target general OSS repositories, they tend to go under the radar for a long period of time, allowing an attacker to gain access to sensitive data or systems. In this paper, we propose (SC)2V — secure crowdsource-based code verification, a novel distributed and scalable framework for verifying OSS libraries. (SC)2V is aimed at preventing targeted supply chain attacks and is integrated in the build phase of software production, serving as an additional code verification step before packaging the application and deploying it. (SC)2V involves both users (developers seeking to verify an OSS library) and verifiers that contribute to the collaborative verification effort. (SC)2V considers a library as verified and safe when a consensus is reached among the verifiers. We evaluated the proposed method using eight different attack scenarios (including cold start and edge cases), on around 900 popular OSS libraries and their dependencies, each of which included an average of 10 files and was verified by at least five participants; a total of 127,000 files were evaluated, and the results indicate that it took our framework an average of just 26 s to issue an alert against the attacks.

SUETA: Speaker-specific utterance ensemble based transfer attack on speaker identification system

With the widespread application of speaker identification (SI) systems in security-related tasks, the robustness of SI systems against adversarial examples has garnered increasing attention. Existing works have demonstrated the vulnerability of SI systems to transfer-based black-box attacks, wherein attackers generate adversarial examples with a surrogate model and then transfer them to attack the target system. However, the attack success rate (ASR) of transfer-based black-box attacks is limited by the transferability of adversarial examples. As far as we know, few works have investigated enhancing the transferability of adversarial examples for speech utterances to attack SI systems. Furthermore, existing works only utilize a single utterance in the attack process, but in practical situations, an attacker can usually collect multiple utterances of a speaker. In this paper, we propose a novel transfer-based black-box attack method, called speaker-specific utterance ensemble based transfer attack (SUETA), to attack SI systems. To the best of our knowledge, SUETA is the first transfer-based black-box attack method that utilizes multiple utterances instead of a single one. Furthermore, we also propose an improved variant of SUETA, called SUETA+, by sharing gradients of utterances at the speaker-embedding level. Empirical results show that both SUETA and SUETA+ improve the ASR compared to the baselines under the classical cross-model situation. SUETA+ further shows additional improvement over SUETA, especially in the case of the untargeted attack. SUETA+ also outperforms the baselines in cross-dataset and cross-preprocessor situations, although the ASR for all transfer-based attacks decreases compared to that in the cross-model situation. Moreover, SUETA+ can significantly improve the ASR against commercial APIs (iFlytek and TalentedSoft) and the voice assistant (Tmall Genie) for the untargeted attack compared to the baselines.

Cube-Evo: A Query-Efficient Black-Box Attack on Video Classification System

The current progressive research in the domain of black-box adversarial attack enhances the reliability of deep neural network (DNN)-based video systems. Recent works mainly carry out black-box adversarial attacks on video systems by query-based parameter dimension reduction. However, the additional temporal dimension of video data leads to massive query consumption and low attack success rate. In this article, we embark on our efforts to design an effective adversarial attack on popular video classification systems. We deeply root the observations that the DNN-based systems are sensitive to adversarial perturbations with high frequency and reconstructed shape. Specifically, we propose a systematic attack pipeline Cube-Evo, aiming to reduce the search space dimension and obtain the effective adversarial perturbation via the optimal parameter group updating. We evaluate the proposed attack pipeline on two popular datasets: UCF101 and JESTER. Our attack pipeline reduces query consumption and achieves a high success rate on various DNN-based video classification systems. Compared with the state-of-the-art method Geo-Trap-Att, our pipeline averagely reduces 1.6× query consumption in untargeted attacks and 2.9× in targeted attacks. Besides, Cube-Evo improves 13% attack success rate on average, achieving new state-of-the-art results over diverse video classification systems.

Transferable Adversarial Attacks for Object Detection Using Object-Aware Significant Feature Distortion

Transferable black-box adversarial attacks against classifiers by disturbing the intermediate-layer features have been extensively studied in recent years. However, these methods have not yet achieved satisfactory performances when directly applied to object detectors. This is largely because the features of detectors are fundamentally different from that of the classifiers. In this study, we propose a simple but effective method to improve the transferability of adversarial examples for object detectors by leveraging the properties of spatial consistency and limited equivariance of object detectors’ features. Specifically, we combine a novel loss function and deliberately designed data augmentation to distort the backbone features of object detectors by suppressing significant features corresponding to objects and amplifying the surrounding vicinal features corresponding to object boundaries. As such the target object and background area on the generated adversarial samples are more likely to be confused by other detectors. Extensive experimental results show that our proposed method achieves state-of-the-art black-box transferability for untargeted attacks on various models, including one/two-stage, CNN/Transformer-based, and anchor-free/anchor-based detectors.

Adversarial patch-based false positive creation attacks against aerial imagery object detectors

Although adversarial attacks have revealed weaknesses in Deep Neural Networks (DNNs)-based aerial detectors, they present a new paradigm for concealing vulnerable assets from autonomous detection systems onboard satellites. Among them, adversarial patches were widely applied due to their physical realizability. Nonetheless, most existing adversarial patch-based attack methods are developed to hide objects from detectors to achieve a vanishing attack. This paper proposes a novel Adversarial Patch False Positive Creation Attack (APFP-CA) framework that enables aerial detectors to recognize non-existent objects, thereby realizing a creation attack. Concretely, the APFP-CA models the creation attack as a two-stage optimization problem. The first stage uses a well-designed efficient loss function to increase the objectness score of patches placed anywhere in the scene to achieve untargeted attacks. The second stage involves elaborating two category-dependent loss functions for fulfilling targeted attacks. Experiments conducted on diverse datasets and detectors demonstrate the effectiveness and universality of our method. Transfer attacks across different datasets and models validate the generalizability of the optimized adversarial patches. Finally, we perform several proportionally scaled experiments physically to illustrate that the optimized adversarial patches can successfully deceive aerial detectors in the physical world. The proposed approach offers a novel contribution to adversarial attacks against aerial imagery objectors and holds potential for practical applications in high-security systems.

Time to Think the Security of WiFi-Based Behavior Recognition Systems

Behavior recognition plays an essential role in numerous behavior-driven applications ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</i> . <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g</i> ., virtual reality and smart home) and even in the security-critical applications ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</i> . <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g</i> ., security surveillance and elder healthcare). Recently, WiFi-based behavior recognition (WBR) technique stands out among many behavior recognition techniques due to its advantages of being non-intrusive, device-free, and ubiquitous. However, existing WBR research mainly focuses on improving the recognition precision, while rarely studying the security aspects. In this paper, we reveal that WBR systems are vulnerable to manipulating physical signals. For instance, our observation shows that WiFi signals can be changed by jamming signals. By exploiting the vulnerability, we propose two approaches to generate physically online adversarial samples to perform untargeted attack and targeted attack, respectively. The effectiveness of these attacks are extensively evaluated over four real-world WBR systems. The experiment results show that our attack approaches can achieve 80% and 60% success rates for untargeted attack and targeted attack in physical world, respectively. We also show that our attack approaches can be generalized to other WiFi-based sensing applications, such as user authentication.

A robust graph attention network with dynamic adjusted graph

Graph Attention Networks(GATs) are useful deep learning models to deal with graph data. However, recent works show that the classical GAT is vulnerable to adversarial attacks. In general, the information aggregated by GAT can be divided into two categories: one type of information has a positive effect and comes from neighbor nodes with the same label, while the other type has a negative effect and comes from neighbor nodes with different labels. Adversarial attacks can modify the graph’s features by adding or deleting edges, increasing the influence of nodes with the same label, and decreasing the influence of nodes with different labels, which degrades GAT dramatically. Therefore, how to enhance the robustness of GAT is a critical problem. Robust GAT (RoGAT) is proposed in this paper to improve the robustness of GAT based on the revision of the attention mechanism. RoGAT adjusts the effect of the positive and negative neighbors based on an extra dynamic attention score, which is computed by the Laplacian regularization. It generates an extra dynamic attention score progressively and improves the robustness. Firstly, RoGAT revises the edge‘s weight based on the smoothness assumption which is quite common for ordinary graphs. Secondly, RoGAT further revises the features to suppress features’ noise. Then, an extra attention score is generated by the dynamic edge’s weight and can be used to reduce the impact of adversarial attacks. Different experiments against targeted and untargeted attacks on citation data demonstrate that RoGAT outperforms most of the recent defensive methods.

DE-JSMA: a sparse adversarial attack algorithm for SAR-ATR models

The vulnerability of DNN makes the SAR-ATR system that uses an intelligent algorithm for recognition also somewhat vulnerable. In order to verify the vulnerability, this paper proposes DE-JSMA, a novel sparse adversarial attack algorithm based on a salient map's adversarial attack algorithm and differential evolution algorithm, with the synthetic aperture radar (SAR) image feature sparsity considered. After accurately screening out the salient features that have a great impact on the model inference results, the DE-JSMA algorithm optimizes the appropriate feature values for the salient features. In order to verify its effectiveness more comprehensively, a new metric that combines the attack success rate with the average confidence interval of adversarial examples is proposed. The experimental results show that DE-JSMA extends JSMA, which can be used only for targeted attack scenario, to untargeted attack scenario without increasing too much time consumption but ensuring a high attack success rate, thus achieving sparse adversarial attack with higher reliability and better sparsity in both attack scenarios. The pixel perturbations of only 0.31% and 0.85% can achieve the untargeted and targeted attack success rates up to 100% and 78.79% respectively.

FGDA-GS: Fast guided decision attack based on gradient signs for skeletal action recognition

The black-box adversarial attacks for skeletal action recognition currently construct high-quality adversarial examples by sacrificing queries and perturbation imperceptibility. To overcome the above drawbacks, we propose a novel fast guided decision attack based on gradient signs for skeletal action recognition (FGDA-GS) scheme. Firstly, to reduce the queries, we guide the adversary to attack along the gradient direction obtained by sampling in a custom Gaussian distribution and design a gradient direction estimation method based on successful history information queries, which can quickly determine the updated direction of the effective perturbation without a large query. Secondly, to ensure the imperceptibility of the adversarial examples, we design a manifold projection function matching the first-order derivatives based on joint position constraints, joint angle constraints, and bone-length constraints to generate complete and natural skeletal adversarial examples that are invisible to the human eye. Finally, comprehensive experimental results demonstrate that FGDA-GS can generate strong-natured, highly effective adversarial examples with significantly reduced query. Specifically, FGDA-GS achieves a 100% attack success rate on different datasets and models with reduced queries by hundreds to thousands and reduced time by more than half. Particularly in the untargeted attack setting, compared to BASAR, FGDA-GS outperforms in terms of average joint position deviation, average joint acceleration deviation, average joint angle acceleration deviation, and average bone-length deviation percentage for the ST-GCN model on the NTU dataset, with percentage reductions of 91.13%, 91.81%, 80.60%, and 81.08%, respectively. We also conducted experiments on defense methods to validate our proposed scheme's effectiveness further.

Security in defect detection: A new one-pixel attack for fooling DNNs

The Industrial 5.0 Model integrates enabling technologies such as deep learning, digital twins, and the meta-universe with new development concepts. However, model and data security may pose challenges for developing zero-defect production and other industrial manufacturing industries. To address this issue, we generate adversarial examples using a one-pixel attack in adversarial machine learning, which can fool the defect detection classification model. The traditional one-pixel attack based on the Differential Evolution (DE) algorithm has limited global search ability. Therefore, we use a novel algorithm called Teaching andLearning-basedMoth-FlameOptimization (TLMFO), which enhances the global search performance and improves the attack effectiveness. We evaluate TLMFO on benchmark functions and attacks on Cifar10 and ImageNet datasets, and compare it with MFO and DE. The results show that TLMFO outperforms both MFO and DE in terms of accuracy and speed of convergence. Moreover, TLMFO achieves notably better attack effectiveness than DE under targeted and untargeted attacks on the Cifar10 dataset and under-targeted attacks on the ImageNet dataset. Our research confirms that safety prevention is a link worth considering in developing Industry 5.0.

Adversarial robustness in deep neural networks based on variable attributes of the stochastic ensemble model.

Deep neural networks (DNNs) have been shown to be susceptible to critical vulnerabilities when attacked by adversarial samples. This has prompted the development of attack and defense strategies similar to those used in cyberspace security. The dependence of such strategies on attack and defense mechanisms makes the associated algorithms on both sides appear as closely processes, with the defense method being particularly passive in these processes. Inspired by the dynamic defense approach proposed in cyberspace to address endless arm races, this article defines ensemble quantity, network structure, and smoothing parameters as variable ensemble attributes and proposes a stochastic ensemble strategy based on heterogeneous and redundant sub-models. The proposed method introduces the diversity and randomness characteristic of deep neural networks to alter the fixed correspondence gradient between input and output. The unpredictability and diversity of the gradients make it more difficult for attackers to directly implement white-box attacks, helping to address the extreme transferability and vulnerability of ensemble models under white-box attacks. Experimental comparison of ASR-vs.-distortion curves with different attack scenarios under CIFAR10 preliminarily demonstrates the effectiveness of the proposed method that even the highest-capacity attacker cannot easily outperform the attack success rate associated with the ensemble smoothed model, especially for untargeted attacks.

Neural Adversarial Attacks with Random Noises

In this paper, we present an approach which relies on the use of random noises to generate adversarial examples of deep neural network classifiers. We argue that existing deterministic attacks, which perform by sequentially applying maximal perturbations on selected components of the input, fail at reaching accurate adversarial examples on real-world large scale datasets. By exploiting a simple Taylor expansion of the expected output probability under the noise perturbation, we introduce noise-based sparse (or L0) targeted and untargeted attacks. Our proposed method, called Voting Folded Gaussian Attack (VFGA), achieves significantly better L0 scores than state-of-the-art L0 attacks (such as SparseFool and Sparse-RS) while being faster on both CIFAR-10 and ImageNet. Moreover, we show that VFGA is also applicable as an L∞ attack and outperforms the state-of-the-art projected gradient attack (PGD) method.

Untargeted Attack against Federated Recommendation Systems via Poisonous Item Embeddings and the Defense

Federated recommendation (FedRec) can train personalized recommenders without collecting user data, but the decentralized nature makes it susceptible to poisoning attacks. Most previous studies focus on the targeted attack to promote certain items, while the untargeted attack that aims to degrade the overall performance of the FedRec system remains less explored. In fact, untargeted attacks can disrupt the user experience and bring severe financial loss to the service provider. However, existing untargeted attack methods are either inapplicable or ineffective against FedRec systems. In this paper, we delve into the untargeted attack and its defense for FedRec systems. (i) We propose ClusterAttack, a novel untargeted attack method. It uploads poisonous gradients that converge the item embeddings into several dense clusters, which make the recommender generate similar scores for these items in the same cluster and perturb the ranking order. (ii) We propose a uniformity-based defense mechanism (UNION) to protect FedRec systems from such attacks. We design a contrastive learning task that regularizes the item embeddings toward a uniform distribution. Then the server filters out these malicious gradients by estimating the uniformity of updated item embeddings. Experiments on two public datasets show that ClusterAttack can effectively degrade the performance of FedRec systems while circumventing many defense methods, and UNION can improve the resistance of the system against various untargeted attacks, including our ClusterAttack.

Improving Adversarial Robustness with Self-Paced Hard-Class Pair Reweighting

Deep Neural Networks are vulnerable to adversarial attacks. Among many defense strategies, adversarial training with untargeted attacks is one of the most effective methods. Theoretically, adversarial perturbation in untargeted attacks can be added along arbitrary directions and the predicted labels of untargeted attacks should be unpredictable. However, we find that the naturally imbalanced inter-class semantic similarity makes those hard-class pairs become virtual targets of each other. This study investigates the impact of such closely-coupled classes on adversarial attacks and develops a self-paced reweighting strategy in adversarial training accordingly. Specifically, we propose to upweight hard-class pair losses in model optimization, which prompts learning discriminative features from hard classes. We further incorporate a term to quantify hard-class pair consistency in adversarial training, which greatly boosts model robustness. Extensive experiments show that the proposed adversarial training method achieves superior robustness performance over state-of-the-art defenses against a wide range of adversarial attacks. The code of the proposed SPAT is published at https://github.com/puerrrr/Self-Paced-Adversarial-Training.

Over-the-Air Adversarial Attacks on Deep Learning Wi-Fi Fingerprinting

Empowered by deep neural networks (DNNs), Wi-Fi fingerprinting has recently achieved astonishing localization performance to facilitate many security-critical applications in wireless networks, but it is inevitably exposed to adversarial attacks, where subtle perturbations can mislead DNNs to wrong predictions. Such vulnerability provides new security breaches to malicious devices for hampering wireless network security, such as malfunctioning geofencing or asset management. The prior adversarial attack on localization DNNs uses additive perturbations on channel state information (CSI) measurements, which is impractical in Wi-Fi transmissions. To transcend this limitation, this paper presents FooLoc, which fools Wi-Fi CSI fingerprinting DNNs over the realistic wireless channel between the attacker and the victim access point (AP). We observe that though uplink CSIs are unknown to the attacker, the accessible downlink CSIs could be their reasonable substitutes at the same spot. We thoroughly investigate the multiplicative and repetitive properties of over-the-air perturbations and devise an efficient optimization problem to generate imperceptible yet robust adversarial perturbations. We implement FooLoc using commercial Wi-Fi APs and Wireless Open-Access Research Platform (WARP) v3 boards in offline and online experiments, respectively. The experimental results show that FooLoc achieves overall attack success rates of about 70% in targeted attacks and of above 90% in untargeted attacks with small perturbation-to-signal ratios of about -18 dB.

ShuffleDetect: Detecting Adversarial Images against Convolutional Neural Networks

Recently, convolutional neural networks (CNNs) have become the main drivers in many image recognition applications. However, they are vulnerable to adversarial attacks, which can lead to disastrous consequences. This paper introduces ShuffleDetect as a new and efficient unsupervised method for the detection of adversarial images against trained convolutional neural networks. Its main feature is to split an input image into non-overlapping patches, then swap the patches according to permutations, and count the number of permutations for which the CNN classifies the unshuffled input image and the shuffled image into different categories. The image is declared adversarial if and only if the proportion of such permutations exceeds a certain threshold value. A series of 8 targeted or untargeted attacks was applied on 10 diverse and state-of-the-art ImageNet-trained CNNs, leading to 9500 relevant clean and adversarial images. We assessed the performance of ShuffleDetect intrinsically and compared it with another detector. Experiments show that ShuffleDetect is an easy-to-implement, very fast, and near memory-free detector that achieves high detection rates and low false positive rates.

Priority Evasion Attack: An Adversarial Example That Considers the Priority of Attack on Each Classifier

Deep neural networks (DNNs) provide excellent services in machine learning tasks such as image recognition, speech recognition, pattern recognition, and intrusion detection. However, an adversarial example created by adding a little noise to the original data can result in misclassification by the DNN and the human eye cannot tell the difference from the original data. For example, if an attacker creates a modified right-turn traffic sign that is incorrectly categorized by a DNN, an autonomous vehicle with the DNN will incorrectly classify the modified right-turn traffic sign as a U-Turn sign, while a human will correctly classify that changed sign as right turn sign. Such an adversarial example is a serious threat to a DNN. Recently, an adversarial example with multiple targets was introduced that causes misclassification by multiple models within each target class using a single modified image. However, it has the weakness that as the number of target models increases, the overall attack success rate decreases. Therefore, if there are multiple models that the attacker wishes to attack, the attacker must control the attack success rate for each model by considering the attack priority for each model. In this paper, we propose a priority adversarial example that considers the attack priority for each model in cases targeting multiple models. The proposed method controls the attack success rate for each model by adjusting the weight of the attack function in the generation process while maintaining minimal distortion. We used MNIST and CIFAR10 as data sets and Tensorflow as machine learning library. Experimental results show that the proposed method can control the attack success rate for each model by considering each model's attack priority while maintaining minimal distortion (average 3.95 and 2.45 with MNIST for targeted and untargeted attacks, respectively, and average 51.95 and 44.45 with CIFAR10 for targeted and untargeted attacks, respectively).