Black-box Scenarios Research Articles

SSAT: Active Authorization Control and User’s Fingerprint Tracking Framework for DNN IP Protection

As training a high-performance deep neural network (DNN) model requires a large amount of data, powerful computing resources and expert knowledge, protecting well-trained DNN models from intellectual property (IP) infringement has raised serious concerns in recent years. Most existing methods using DNN watermarks to verify the ownership of the models after IP infringement occurs, which is reactive in the sense that they cannot prevent unauthorized users from using the model in the first place. Different from these methods, in this article, we propose an active authorization control and user’s fingerprint tracking method for the IP protection of DNN models by utilizing sample-specific backdoor attack. The proposed method inversely and multiplely exploits sample-specific trigger as the key to implement authorization control for DNN model, in which the generated triggers are imperceptible and sample-specific for clean images. Specifically, a U-Net model is used to generate backdoor instances. Then, the target model is trained on the clean images and backdoor instances, which are inversely labeled as wrong classes and correct classes, respectively. Only authorized users can use the target model normally by pre-processing the clean images through the U-Net model. Moreover, the images processed by the U-Net model will contain unique fingerprint that can be extracted to verify and track the corresponding user’s identity. This article is the first work that utilizes the sample-specific backdoor attack to implement active authorization control and user’s fingerprint management for DNN model under black-box scenarios. Extensive experimental results on ImageNet dataset and YouTube Aligned Face dataset demonstrate that the proposed method is effective in protecting the DNN model from unauthorized usage. Specifically, the protected model has a low inference accuracy (1.00%) for unauthorized users, while maintaining a normal inference accuracy (97.67%) for authorized users. Besides, the proposed method can achieve 100% fingerprint tracking success rates on both the ImageNet and YouTube Aligned Face datasets. Moreover, it is demonstrated that the proposed method is robust against fine-tuning attack, pruning attack, pruning attack with retraining, reverse-engineering attack, adaptive attack, and JPEG compression attack. The code is available at https://github.com/nuaaaisec/SSAT .

Ranking the Transferability of Adversarial Examples

Adversarial transferability in blackbox scenarios presents a unique challenge: while attackers can employ surrogate models to craft adversarial examples, they lack assurance on whether these examples will successfully compromise the target model. Until now, the prevalent method to ascertain success has been trial and error—testing crafted samples directly on the victim model. This approach, however, risks detection with every attempt, forcing attackers to either perfect their first try or face exposure. Our article introduces a ranking strategy that refines the transfer attack process, enabling the attacker to estimate the likelihood of success without repeated trials on the victim’s system. By leveraging a set of diverse surrogate models, our method can predict transferability of adversarial examples. This strategy can be used to either select the best sample to use in an attack or the best perturbation to apply to a specific sample. Using our strategy, we were able to raise the transferability of adversarial examples from a mere 20%—akin to random selection—up to near upper-bound levels, with some scenarios even witnessing a 100% success rate. This substantial improvement not only sheds light on the shared susceptibilities across diverse architectures but also demonstrates that attackers can forego the detectable trial-and-error tactics raising increasing the threat of surrogate-based attacks.

A review of black-box adversarial attacks and defenses in machine learning-based malware detection

In recent years, significant advancements have been made in cyber security, particularly through the application of machine learning (ML) methods. ML-based techniques have enhanced system security by effectively distinguishing between malicious and benign objects across various domains, including spam email detection, social media content filtering, intrusion detection systems, and malware detection. This paper focuses on the specific area of ML-based malware detection and its advantages over traditional methods, such as improved accuracy and the ability to generalize to unknown threats. Despite these advancements, ML-based malware detection systems are vulnerable to adversarial attacks, especially in black-box scenarios where the internal workings of the detection model are not accessible. This paper provides a comprehensive review of adversarial attacks on black-box malware detection systems and examines the current defense mechanisms against these attacks. Understanding the challenges and strategies in this field, this review aims to offer insights into enhancing the robustness and security of ML-based malware detection systems.

Enhancing the Transferability of Adversarial Patch via Alternating Minimization

Adversarial patches, a type of adversarial example, pose serious security threats to deep neural networks (DNNs) by inducing erroneous outputs. Existing gradient stabilization methods aim to stabilize the optimization direction of adversarial examples through accumulating gradient momentum to enhance attack transferability on black-box models. However, they are not fully effective for adversarial patches. The accumulated momentum during optimization often misaligns with the optimization direction, reducing their efficacy in black-box scenarios. We introduce an optimization method called Alternating Minimization for Adversarial Patch (AMAP). This method decomposes the original AP into multiple sub-patches and utilizes their update direction to stabilize the optimization of the original AP. Additionally, we propose an adaptive step size optimization method that accelerates convergence and boosts the attack performance. In face recognition tasks, AMAP outperforms baseline methods by a remarkable 5.21% and exceeds the second-best method by 1.4%. Furthermore, AMAP demonstrates practical feasibility in the physical domain, highlighting its potential for robust computer security testing applications.

Enhancing Transferability with Intra-Class Transformations and Inter-Class Nonlinear Fusion on SAR Images

Recent research has revealed that the deep neural network (DNN)-based synthetic-aperture radar (SAR) automatic target recognition (ATR) techniques are vulnerable to adversarial examples, which poses significant security risks for their deployment in real-world systems. At the same time, the adversarial examples often exhibit transferability across DNN models, whereby when they are generated on the surrogate model they can also attack other target models. As the significant property in black-box scenarios, transferability has been enhanced by various methods, among which input transformations have demonstrated excellent effectiveness. However, we find that existing transformations suffer from limited enhancement of transferability due to the unique imaging mechanism and scattering characteristics of SAR images. To overcome this issue, we propose a novel method called intra-class transformations and inter-class nonlinear fusion attack (ITINFA). It enhances transferability from two perspectives: intra-class single image transformations and inter-class multiple images fusion. The intra-class transformations module utilizes a series of diverse transformations that align with the intrinsic characteristics of SAR images to obtain a more stable gradient update direction and prevent the adversarial examples from overfitting the surrogate model. The inter-class fusion strategy incorporates the information from other categories in a nonlinear manner, effectively enhances the feature fusion effect, and guides the misclassification of adversarial examples. Extensive experiments on the MSTAR dataset and SEN1-2 dataset demonstrate that ITINFA exhibits significantly better transferability than the existing transfer-based methods, with the average transfer attack success rate increases exceeding 8% for single models and over 4% for ensemble models.

Adaptive Whitening and Feature Gradient Smoothing-Based Anti-Sample Attack Method for Modulated Signals in Frequency-Hopping Communication

Adaptive Whitening and Feature Gradient Smoothing-Based Anti-Sample Attack Method for Modulated Signals in Frequency-Hopping Communication

C2FMI: Corse-to-Fine Black-Box Model Inversion Attack

Privacy-preserving machine learning requires that models do not reveal any private information about their training data. However, model inversion attacks (MIAs), which aim to recover the features of training data, pose a huge threat to the security of AI models. Most existing MIAs assume that the target model is white-box, but most models deployed in reality are black-box, and these models can only be accessed like an oracle. There are a few studies for black-box scenarios, but their performance is limited. In this paper, we firstly formulate the MIA problem completely in Bayesian perspective. Second, we propose a novel two-stage MIA approach, the Coarse-to-Fine Model Inversion Attack (C2FMI), which efficiently addresses the MIA problem in the black-box scenario. In stage I of C2FMI, we design a reverse network that constrains the recovered images (also named attacked images) to fall near the manifold space of the training data. In stage II, we design a black-box oriented strategy which further facilitates the attacked images to approach the training data. Empirically, C2FMI achieves a performance that even surpasses existing white-box attack methods. Furthermore, we design the stability analysis method for analyzing the stability of C2FMI along with existing MIAs. Finally, we explore the potential countermeasures which could defend against our attacks.

CFOA: Exploring transferable adversarial examples by content feature optimization with attention guidance

Deep neural networks are known to be highly susceptible to adversarial examples, created by adding small and carefully-designed perturbations on original images. In black-box scenarios, attacking neural networks is challenging since the information about models is not available. Generating transferable adversarial examples on surrogate models is a viable way. However, current mainstream transfer-based attacks seldomly consider the relationship between transferability and content features of the image. To effectively enhance the attack success rate at the feature level, we newly propose an attack method called Content Feature Optimization Attack (CFOA). This novel CFOA approach operates on white-box models, aiming to generate transferable adversarial examples by targeting the content feature representations of images in the feature space. The method involves the extraction and parameterization of content features, followed by the iterative generation of feature-level perturbations guided by attention loss and adversarial loss. Ultimately, the adversarial examples generated by CFOA exhibit strong transferability on other black-box models. Experimental results show that our method produces adversarial examples with higher success rates in transferability compared to state-of-the-art methods, and importantly, they can effectively evade certain defense mechanisms, presenting a challenge for adversarial defense.

Dynamic loss yielding more transferable targeted adversarial examples

Adversarial examples are known to have the property of transferability; as a result, deep neural networks can be compromised by transfer-based attacks in black-box scenarios. Recent studies have shown that targeted adversarial examples are far more difficult to transfer than untargeted adversarial examples. We find that the widely used pattern for targeted attacks (i.e., minimizing the loss with respect to the target label) is insufficient for the transfer scenario. To address this issue, we propose a dynamic loss to yield more transferable targeted adversarial examples. In each iteration of attack optimization, in addition to minimizing the loss with respect to the target label, we dynamically select the top k labels (excluding the target label) with high classification probability to penalize (i.e., maximize) their corresponding loss. As a result, the probability of adversarial examples being classified as target labels will be significantly higher than that of being classified as other labels. Extensive experiments on ImageNet and the Google Cloud Vision API show that the dynamic loss significantly outperforms the traditional loss. For example, in the ensemble-based transfer scenario, the dynamic Cross-Entropy (CE) loss can improve the transferability of targeted adversarial examples by 2∼6 times relative to the traditional CE loss. Code is available at https://github.com/mingcheung/Targeted-Transfer-with-Dynamic-Loss.

Enhancing reinforcement learning based adversarial malware generation to evade static detection

The anti-detection capabilities of adversarial malware examples have drawn the attention of antivirus vendors and researchers. In black-box scenarios where internal information of the target model cannot be accessed, with the ability of the reinforcement learning (RL) agents to adjust strategies based on the feedback from the environment, existing RL-based methods enable evasion of Windows PE malware detectors. However, obtaining evasion rewards as positive feedback in the black-box setting proves challenging, resulting in low training efficiency. To address this issue, we introduce the intrinsic curiosity reward into the framework to motivate the agent to explore unknown state spaces and learn effective evasion strategies. Additionally, we employ the generative adversarial network (GAN) to obtain varying synthetic data, which replaces random or benign bytes as adversarial payloads for the agent's action content, improving attack capabilities and reducing the risk of hard-coded adversarial perturbations being anchored. We compare the attack performance with other RL-based baseline methods, and experimental results show that our framework is more flexible and effective, achieving a 63%-85% attack success rate against EMBER, FireEye and MalConv. Even if defensive measures are taken, the proposed method still has a certain attack capability, and the success rate remains between 48%-67%.

Link-Backdoor: Backdoor Attack on Link Prediction via Node Injection

Link prediction, inferring the undiscovered or potential links of the graph, is widely applied in the real world. By facilitating labeled links of the graph as the training data, numerous deep learning-based link prediction methods have been studied, which have dominant prediction accuracy compared with nondeep methods. However, the threats of maliciously crafted training graphs will leave a specific backdoor in the deep model; thus, when some specific examples are fed into the model, it will make a wrong prediction, defined as a backdoor attack. It is an important aspect that has been overlooked in the current literature. In this article, we prompt the concept of a backdoor attack on link prediction and propose Link-Backdoor to reveal the training vulnerability of the existing link prediction methods. Specifically, the Link-Backdoor combines the fake nodes with the nodes of the target link to form a trigger. Moreover, it optimizes the trigger by the gradient information from the target model. Consequently, the link prediction model trained on the backdoored dataset will predict the link with a trigger to the target state. Extensive experiments on five benchmark datasets and five well-performing link prediction models demonstrate that the Link-Backdoor achieves the state-of-the-art attack success rate under both the white box (i.e., available of the target model parameter) and the black box (i.e., unavailable of the target model parameter) scenarios. In addition, we testify the attack under the defensive circumstance, and the results indicate that the Link-Backdoor still can construct a successful attack on the well-performing link prediction methods. The code and data are available at https://github.com/Seaocn/Link-Backdoor.

LaViP: Language-Grounded Visual Prompting

We introduce a language-grounded visual prompting method to adapt the visual encoder of vision-language models for downstream tasks. By capitalizing on language integration, we devise a parameter-efficient strategy to adjust the input of the visual encoder, eliminating the need to modify or add to the model's parameters. Due to this design choice, our algorithm can operate even in black-box scenarios, showcasing adaptability in situations where access to the model's parameters is constrained. We will empirically demonstrate that, compared to prior art, grounding visual prompts with language enhances both the accuracy and speed of adaptation. Moreover, our algorithm excels in base-to-novel class generalization, overcoming limitations of visual prompting and exhibiting the capacity to generalize beyond seen classes. We thoroughly assess and evaluate our method across a variety of image recognition datasets, such as EuroSAT, UCF101, DTD, and CLEVR, spanning different learning situations, including few-shot adaptation, base-to-novel class generalization, and transfer learning.

Enhancing adversarial attacks with resize-invariant and logical ensemble

In black-box scenarios, most transfer-based attacks usually improve the transferability of adversarial examples by optimizing the gradient calculation of the input image. Unfortunately, since the gradient information is only calculated and optimized for each pixel point in the image individually, the generated adversarial examples tend to overfit the local model and have poor transferability to the target model. To tackle the issue, we propose a resize-invariant method (RIM) and a logical ensemble transformation method (LETM) to enhance the transferability of adversarial examples. Specifically, RIM is inspired by the resize-invariant property of Deep Neural Networks (DNNs). The range of resizable pixel is first divided into multiple intervals, and then the input image is randomly resized and padded within each interval. Finally, LETM performs logical ensemble of multiple images after RIM transformation to calculate the final gradient update direction. The proposed method adequately considers the information of each pixel in the image and the surrounding pixels. The probability of duplication of image transformations is minimized and the overfitting effect of adversarial examples is effectively mitigated. Numerous experiments on the ImageNet dataset show that our approach outperforms other advanced methods and is capable of generating more transferable adversarial examples.

Generative Adversarial Attacks Against Deep-Learning-Based Camera Model Identification

Recently, deep learning techniques have gained popularity in multimedia forensics research designed to accomplish tasks such as camera model identification. However, despite the success of deep learning techniques, research has shown that they are vulnerable to adversarial perturbations. These adversarial perturbations can cause deep learning classifiers to misclassify images even though the perturbations are imperceptible to human eyes. To understand the vulnerabilities of deep-learning-based forensic algorithms, we propose a novel anti-forensic framework inspired by generative adversarial networks that is capable of falsifying an image’s source camera model. To accomplish this, we design a generator to anti-forensically falsify camera model traces in an image without introducing visually perceptible changes or artifacts. We propose two techniques to adversarially train this generator depending on the knowledge available to the attacker. In a white-box scenario when complete knowledge of an investigator’s camera model identification network is available to an attacker, we directly incorporate the network into our generator’s adversarial training strategy. In a black-box scenario when no internal details of the camera model classifier are available to the attacker, we construct a substitute network to mimic its decisions, then utilize this substitute network to adversarially train our generator. We conduct a series of experiments to evaluate the performance of our attack against several well-known CNNbased camera model classifiers. Experimental results show that our attack can successfully fool these CNNs in both white-box and black-box scenarios. Furthermore, our attack maintains high image quality and can be generalized to attack images from arbitrary source camera models.

The γ/γ′ microstructure in CoNiAlCr-based superalloys using triple-objective optimization

Optimizing several properties simultaneously based on small data-driven machine learning in complex black-box scenarios can present difficulties and challenges. Here we employ a triple-objective optimization algorithm deduced from probability density functions of multivariate Gaussian distributions to optimize the γ′ volume fraction, size, and morphology in CoNiAlCr-based superalloys. The effectiveness of the algorithm is demonstrated by synthesizing alloys with desired γ/γ′ microstructure and optimizing γ′ microstructural parameters. In addition, the method leads to incorporating refractory elements to improve γ/γ′ microstructure in superalloys. After four iterations of experiments guided by the algorithm, we synthesize sixteen alloys of relatively high creep strength from ~120,000 candidates of which three possess high γ′ volume fraction (>54%), small γ′ size (<480 nm), and high cuboidal γ′ fraction (>77%).

Assessment of spatiotemporally coordinated cyberattacks on renewable energy forecasting in smart energy system

The employment of deep learning (DL) in renewable energy forecasting (REF) may bring novel cyberthreats. However, this fact has not drawn sufficient attention in the existing literature. To fill the gap, this article, among the first, investigates the latent cyberthreat incurred by the DL-based REF model. First, a spatiotemporally coordinated cyberattack strategy is proposed by considering attackable geo-distributed renewable energy resources (RESs) and periods. It is fulfilled by compromising meteorological data transmitted from external online weather forecast interfaces which are required for REF but also exposed to potential adversaries. In this manner, the malicious adversary is able to deteriorate the REF performance, thereby jeopardizing the operation of the smart energy system. Second, the cyberattack optimization models under the white-box and black-box scenarios are designed. In the white-box scenario, the adversary has full knowledge of these operator-own REF models to obtain false data that should be injected into the meteorological data for launching cyberattacks; in the black-box scenario, the adversary will train surrogate models to replace these operator-own models for attack guidance. Third, an iterative solving method is proposed to solve the proposed white-box and black-box cyberattack optimization models. As DL models (sealed, non-differentiable, and highly non-linear) are introduced to the optimization problem, conventional optimization solvers are unable to handle the cyberattack optimization problem. The proposed solving method utilizes the proximal gradient descent principle, and is feasible to explore the near-optimal solution to the problem. At last, the efficacy of the proposed cyberattack strategy under different RES penetration levels and attack strengths and its catastrophic impacts are comprehensively assessed on the IEEE 39-bus benchmark. The results reveal that the cyberattack would cause enormous economic losses and even collapse the smart energy system.

Global-Local Characteristic Excited Cross-Modal Attacks from Images to Videos

The transferability of adversarial examples is the key property in practical black-box scenarios. Currently, numerous methods improve the transferability across different models trained on the same modality of data. The investigation of generating video adversarial examples with imagebased substitute models to attack the target video models, i.e., cross-modal transferability of adversarial examples, is rarely explored. A few works on cross-modal transferability directly apply image attack methods for each frame and no factors especial for video data are considered, which limits the cross-modal transferability of adversarial examples. In this paper, we propose an effective cross-modal attack method which considers both the global and local characteristics of video data. Firstly, from the global perspective, we introduce inter-frame interaction into attack process to induce more diverse and stronger gradients rather than perturb each frame separately. Secondly, from the local perspective, we disrupt the inherently local correlation of frames within a video, which prevents black-box video model from capturing valuable temporal clues. Extensive experiments on the UCF-101 and Kinetics-400 validate the proposed method significantly improves cross-modal transferability and even surpasses strong baseline using video models as substitute model. Our source codes are available at https://github.com/lwmming/Cross-Modal-Attack.

Deep Fusion: Crafting Transferable Adversarial Examples and Improving Robustness of Industrial Artificial Intelligence of Things

Industry 5.0 is aimed at merging the cognitive computing capabilities of Deep neural networks (DNNs) with human resourcefulness in collaborative operations. DNNs have been widely used in Industrial Artificial Intelligence of Things (Industrial AIoT) systems. However, DNNs are vulnerable to adversarial attacks, which bring a considerable risk to Industrial Artificial Intelligence of Things systems. The adversary uses adversarial examples crafted on the local ensemble model to attack black-box target of Industrial Artificial Intelligence of Things systems, resulting in catastrophic consequences. It is essential to study ensemble adversarial attack and defense strategies in black-box scenarios. Nevertheless, current ensemble attacks' performance is limited by the diversity of local models and ensemble strategies, and defensive strategies are inefficient. To solve these problems, we propose two novel Deep Fusion methods from both an attacker's and a defender's perspective. For initiating attacks, we propose Deep Fusion Attack. The erosion models are applied to compensate for local models' insufficiency in diversity. We fuse erosion models in the output space and the feature space simultaneously and continuously accumulate historical gradients to retain adversarial information, thereby improving transferability. Extensive experimental results show that our approach achieves superior performance in black-box attacks than state-of-the-art ensemble attacks. The average success rate of our targeted black-box attack reaches a compelling 87.4%. For constructing defenses, we propose Deep Fusion Defense, using a fusion of multiple predictions with erosion models as a novel approach.

Adversarial Attack for Deep Steganography Based on Surrogate Training and Knowledge Diffusion

Deep steganography (DS), using neural networks to hide one image in another, has performed well in terms of invisibility, embedding capacity, etc. Current steganalysis methods for DS can only detect or remove secret images hidden in natural images and cannot analyze or modify secret content. Our technique is the first approach to not only effectively prevent covert communications using DS, but also analyze and modify the content of covert communications. We proposed a novel adversarial attack method for DS considering both white-box and black-box scenarios. For the white-box attack, several novel loss functions were applied to construct a gradient- and optimizer-based adversarial attack that could delete and modify secret images. As a more realistic case, a black-box method was proposed based on surrogate training and a knowledge distillation technique. All methods were tested on the Tiny ImageNet and MS COCO datasets. The experimental results showed that the proposed attack method could completely remove or even modify the secret image in the container image while maintaining the latter’s high quality. More importantly, the proposed adversarial attack method can also be regarded as a new DS approach.

Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration

Previous works have extensively studied the transferability of adversarial samples in untargeted black-box scenarios. However, it still remains challenging to craft targeted adversarial examples with higher transferability than non-targeted ones. Recent studies reveal that the traditional Cross-Entropy (CE) loss function is insufficient to learn transferable targeted adversarial examples due to the issue of vanishing gradient. In this work, we provide a comprehensive investigation of the CE loss function and find that the logit margin between the targeted and untargeted classes will quickly obtain saturation in CE, which largely limits the transferability. Therefore, in this paper, we devote to the goal of continually increasing the logit margin along the optimization to deal with the saturation issue and propose two simple and effective logit calibration methods, which are achieved by downscaling the logits with a temperature factor and an adaptive margin, respectively. Both of them can effectively encourage optimization to produce a larger logit margin and lead to higher transferability. Besides, we show that minimizing the cosine distance between the adversarial examples and the classifier weights of the target class can further improve the transferability, which is benefited from downscaling logits via L2-normalization. Experiments conducted on the ImageNet dataset validate the effectiveness of the proposed methods, which outperform the state-of-the-art methods in black-box targeted attacks. The source code is available at Link.