Black-box Adversarial Attacks Research Articles

Black-box adversarial attacks against image quality assessment models

The problem of No-Reference Image Quality Assessment (NR-IQA) is to predict the perceptual quality of an image in line with its subjective evaluation. However, the vulnerabilities of NR-IQA models to the adversarial attacks have not been thoroughly studied for model refinement. This paper aims to investigate the potential loopholes of NR-IQA models via black-box adversarial attacks. Specifically, we first formulate the attack problem as maximizing the deviation between the estimated quality scores of original and perturbed images, while restricting the perturbed image distortions for visual quality preservation. Under such formulation, we then design a Bi-directional loss function to mislead the estimated quality scores of adversarial examples towards an opposite direction with maximum deviation. On this basis, we finally develop an efficient and effective black-box attack method for NR-IQA models based on a random search paradigm. Comprehensive experiments on three benchmark datasets show that all evaluated NR-IQA models are significantly vulnerable to the proposed attack method. After being attacked, the average change rates in terms of two well-known IQA performance metrics achieved by victim models reach 97% and 101%, respectively. In addition, our attack method also outperforms a newly introduced black-box attack approach on IQA models. We also observe that the generated perturbations are not transferable, which points out a new research direction in NR-IQA community. The source code is available at https://github.com/GZHU-DVL/AttackIQA.

A review of black-box adversarial attacks and defenses in machine learning-based malware detection

In recent years, significant advancements have been made in cyber security, particularly through the application of machine learning (ML) methods. ML-based techniques have enhanced system security by effectively distinguishing between malicious and benign objects across various domains, including spam email detection, social media content filtering, intrusion detection systems, and malware detection. This paper focuses on the specific area of ML-based malware detection and its advantages over traditional methods, such as improved accuracy and the ability to generalize to unknown threats. Despite these advancements, ML-based malware detection systems are vulnerable to adversarial attacks, especially in black-box scenarios where the internal workings of the detection model are not accessible. This paper provides a comprehensive review of adversarial attacks on black-box malware detection systems and examines the current defense mechanisms against these attacks. Understanding the challenges and strategies in this field, this review aims to offer insights into enhancing the robustness and security of ML-based malware detection systems.

A review of black-box adversarial attacks on image classification

In recent years, deep learning-based image classification models have been extensively studied in academia and widely applied in industry. However, deep learning is inherently vulnerable to adversarial attacks, posing security threats to image classification models in security sensitive field, such as face recognition, medical image diagnosis and traffic sign recognition. Especially for black-box adversarial attacks, which can be carried out even without remote model information, the security issues facing deep learning are even more serious. Despite more and more attentions on this issue, existing reviews always analyze black-box adversarial attack only from one perspective, focus on only a certain application field. This paper systematically reviews and discusses existing progress, demonstrating black-box adversarial attacks from multiple perspectives and systematically classifying existing methods. Besides, we also sort out and categorize the application of current black-box adversarial attacks and identify several promising directions for future research.

Black-box adversarial patch attacks using differential evolution against aerial imagery object detectors

Adversarial patch attacks are commonly used to attack aerial imagery object detectors. However, most existing methods are designed for white-box settings, which are impractical in real world scenarios. In this work, we propose a novel framework for black-box adversarial patch attacks against aerial imagery object detectors using differential evolution (DE). Specifically, we first introduce a new dimensionality reduction strategy to address the high-dimensionality curse in DE optimization problems. Then, we design three universal fitness functions to help DE to select better individuals within a limited computational budget. Finally, we conduct extensive experiments on the Dataset for Object Detection in Aerial images (DOTA) against You Only Look Once (YOLOv3, YOLOv4), and Faster Region-based Convolutional Network (Faster R-CNN), utilizing attack success rate (ASR) and average precision (AP) to quantitatively evaluate the attack efficiency. Results on YOLOv3 for the plane category indicate that our proposed strategy achieves at least a 20.97% improvement in ASR compared to existing dimensionality reduction schemes. Moreover, experiments conducted on detectors with different architectures reveal inconsistent adversarial robustness across categories. To the best of our knowledge, this is the first work to explore the use of DE in black-box adversarial patch attacks against aerial imagery object detectors. Our code has been released at https://github.com/tang-agui/Bbox-Att-Detector.

Explainable multi-layer COSFIRE filters robust to corruptions and boundary attack with application to retina and palmprint biometrics

We propose a novel and versatile computational approach, based on hierarchical COSFIRE filters, that addresses the challenge of explainable retina and palmprint recognition for automatic person identification. Unlike traditional systems that treat these biometrics separately, our method offers a unified solution, leveraging COSFIRE filters’ trainable nature for enhanced selectivity and robustness, while exhibiting explainability and resilience to decision-based black-box adversarial attack and partial matching. COSFIRE filters are trainable, in that their selectivity can be determined with a one-shot learning step. In practice, we configure a COSFIRE filter that is selective for the mutual spatial arrangement of a set of automatically selected keypoints of each retina or palmprint reference image. A query image is then processed by all COSFIRE filters and it is classified with the reference image that was used to configure the COSFIRE filter that gives the strongest similarity score. Our approach, tested on the VARIA and RIDB retina datasets and the IITD palmprint dataset, achieved state-of-the-art results, including perfect classification for retina datasets and a 97.54% accuracy for the palmprint dataset. It proved robust in partial matching tests, achieving over 94% accuracy with 80% image visibility and over 97% with 90% visibility, demonstrating effectiveness with incomplete biometric data. Furthermore, while effectively resisting a decision-based black-box adversarial attack and impervious to imperceptible adversarial images, it is only susceptible to highly perceptible adversarial images with severe noise, which pose minimal concern as they can be easily detected through histogram analysis in preprocessing. In principle, the proposed learning-free hierarchical COSFIRE filters are applicable to any application that requires the identification of certain spatial arrangements of moderately complex features, such as bifurcations and crossovers. Moreover, the selectivity of COSFIRE filters is highly intuitive; and therefore, they provide an explainable solution.

Generation of black-box adversarial attacks using many independent objective-based algorithm for testing the robustness of deep neural networks

Deep neural networks (DNNs) have become increasingly ubiquitous in our daily lives, finding applications in areas such as image recognition, voice recognition, and natural language processing. However, a growing concern revolves around the vulnerability of DNNs to adversarial examples—malicious inputs that can compromise the safety and accuracy of their outcomes. Existing studies predominantly fall into two categories: white-box and black-box techniques. White-box techniques require detailed internal information about the model, often focusing on gradient-based methods. In contrast, black-box techniques, which emulate real-world scenarios more closely, only rely on input–output knowledge. This study focuses specifically on black-box strategies, with key contributions from Single-Objective Variant of Differential Evolution (Pixel-SOO) and Multi-Objective Variant of Differential Evolution (Pixel-MOO) algorithms. While these approaches show promise, they suffer from drawbacks like long execution times and being unable to generate instances for adversarial purposes. To address these challenges, we introduce a novel archive-based Many Independent Objective (MIO) algorithm for using the first time in this context. Our proposed algorithm identifies the most vulnerable image pixels through the MIO algorithm, enabling efficient label flip attacks with minimized attempts. Furthermore, we balance exploration and exploitation by incorporating an adaptive parameter mechanism. The effectiveness of the proposed algorithm is assessed by employing VGG (VGG16 and VGG19) and ResNet (ResNet50, ResNet101, and ResNet152) architectures, both of which are convolutional neural network (CNN) models. The success criterion for the algorithms is to minimize the number of pixel changes while achieving high flip rates with a minimal number of requests to the network. A comprehensive analysis of the experimental results reveals that our algorithm consistently outperforms Pixel-SOO and Pixel-MOO, exhibiting an average speedup of three times compared to Pixel-SOO and seven times compared to Pixel-MOO. In most runs, adversarial attacks are generated with fewer pixel changes than Pixel-MOO and Pixel-SOO. In addition, our findings are openly accessible on GitHub to ensure transparency and reproducibility and encourage future research efforts.

Transferable adversarial attack based on sensitive perturbation analysis in frequency domain

Recently, transferable adversarial attacks on deep neural networks (DNNs) have attracted significant attention. Although existing adversarial attacks have achieved high attack success rates in white-box scenarios, they perform poorly in black-box attack scenarios. To address the issue of low transferability in black-box adversarial attacks, we propose a frequency-sensitive perturbation-based black-box adversarial attack method to enhance transferability performance from the perspective of Fourier domain sensitivity. Initially, the Fourier sensitivity heatmaps of multiple substitute models are integrated to identify the common sensitive frequency region. Subsequently, the perturbations in the frequency domain are optimized through joint loss, thereby constraining the sensitive region and limiting the perturbations in the non-sensitive region. Finally, in each iteration, a spectral model enhancement is performed to simulate additional substitute models and reduce the spectral discrepancy between the substitute and target models. Extensive experimental results on benchmark datasets demonstrated the effectiveness of the proposed method. Compared to existing black-box adversarial attack methods, the proposed approach generates targeted and non-targeted adversarial samples with higher transfer success rates and good invisibility, thereby confirming its superiority.

Cube-Evo: A Query-Efficient Black-Box Attack on Video Classification System

The current progressive research in the domain of black-box adversarial attack enhances the reliability of deep neural network (DNN)-based video systems. Recent works mainly carry out black-box adversarial attacks on video systems by query-based parameter dimension reduction. However, the additional temporal dimension of video data leads to massive query consumption and low attack success rate. In this article, we embark on our efforts to design an effective adversarial attack on popular video classification systems. We deeply root the observations that the DNN-based systems are sensitive to adversarial perturbations with high frequency and reconstructed shape. Specifically, we propose a systematic attack pipeline Cube-Evo, aiming to reduce the search space dimension and obtain the effective adversarial perturbation via the optimal parameter group updating. We evaluate the proposed attack pipeline on two popular datasets: UCF101 and JESTER. Our attack pipeline reduces query consumption and achieves a high success rate on various DNN-based video classification systems. Compared with the state-of-the-art method Geo-Trap-Att, our pipeline averagely reduces 1.6× query consumption in untargeted attacks and 2.9× in targeted attacks. Besides, Cube-Evo improves 13% attack success rate on average, achieving new state-of-the-art results over diverse video classification systems.

Enhancing cross-domain transferability of black-box adversarial attacks on speaker recognition systems using linearized backpropagation

Enhancing cross-domain transferability of black-box adversarial attacks on speaker recognition systems using linearized backpropagation

Feature-aware transferable adversarial attacks against image classification

Compared to white-box adversarial attacks, black-box adversarial attacks are more applicable in practical scenarios and have received significant attention. However, most existing black-box attacks are optimized at the output layer, and the generated adversarial samples are difficult to transfer from the surrogate model to the target model. Existing adversarial attacks indiscriminately perturb features in intermediate layers, which are prone to fall into local optima of surrogate models and have limited transferability. In this paper, we propose a transferable targeted adversarial attack framework based on a feature-aware triplet, which achieves a better tradeoff between attack ability and transferability by perturbing salient features and constructing representative sample features. The importance-weighted optimization objective interferes with the salient object-aware features of images in a targeted manner and guides adversarial perturbations in the optimization process to pull the intermediate features of samples toward a target class while pushing them far from a source class to achieve targeted transferable attacks. Moreover, a construction method of a feature library with the weighted average of feature importance is built to obtain more expressive intermediate features of the target and source classes. The modified target features and source features are fed into the triplets to guide the optimization objectives to find more transferable adversarial samples. Extensive experiments on the ImageNet-compatible dataset verify the effectiveness of the proposed method, e.g., improving the untargeted success rate by 1.8% and the targeted success rate by 2.3% against normally trained models as compared to the existing methods.

Understanding the vulnerability of skeleton-based Human Activity Recognition via black-box attack

Human Activity Recognition (HAR) has been employed in a wide range of applications, e.g. self-driving cars, where safety and lives are at stake. Recently, the robustness of skeleton-based HAR methods have been questioned due to their vulnerability to adversarial attacks. However, the proposed attacks require the full-knowledge of the attacked classifier, which is overly restrictive. In this paper, we show such threats indeed exist, even when the attacker only has access to the input/output of the model. To this end, we propose the very first black-box adversarial attack approach in skeleton-based HAR called BASAR. BASAR explores the interplay between the classification boundary and the natural motion manifold. To our best knowledge, this is the first time data manifold is introduced in adversarial attacks on time series. Via BASAR, we find on-manifold adversarial samples are extremely deceitful and rather common in skeletal motions, in contrast to the common belief that adversarial samples only exist off-manifold. Through exhaustive evaluation, we show that BASAR can deliver successful attacks across classifiers, datasets, and attack modes. By attack, BASAR helps identify the potential causes of the model vulnerability and provides insights on possible improvements. Finally, to mitigate the newly identified threat, we propose a new adversarial training approach by leveraging the sophisticated distributions of on/off-manifold adversarial samples, called mixed manifold-based adversarial training (MMAT). MMAT can successfully help defend against adversarial attacks without compromising classification accuracy.

Black-box adversarial transferability: An empirical study in cybersecurity perspective

The rapid advancement of artificial intelligence within the realm of cybersecurity raises significant security concerns. The vulnerability of deep learning models in adversarial attacks is one of the major issues. In adversarial machine learning, malicious users try to fool the deep learning model by inserting adversarial perturbation inputs into the model during its training or testing phase. Subsequently, it reduces the model confidence score and results in incorrect classifications. The novel key contribution of the research is to empirically test the black-box adversarial transferability phenomena in cyber attack detection systems. It indicates that the adversarial perturbation input generated through the surrogate model has a similar impact on the target model in producing the incorrect classification. To empirically validate this phenomenon, surrogate and target models are used. The adversarial perturbation inputs are generated based on the surrogate-model for which the hacker has complete information. Based on these adversarial perturbation inputs, both surrogate and target models are evaluated during the inference phase. We have done extensive experimentation over the CICDDoS-2019 dataset, and the results are classified in terms of various performance metrics like accuracy, precision, recall and f1-score. The findings indicate that any deep learning model is highly susceptible to adversarial attacks, even if the attacker does not have access to the internal details of the target model. The results also indicate that white-box adversarial attacks have a severe impact compared to black-box adversarial attacks. There is a need to investigate and explore adversarial defence techniques to increase the robustness of the deep learning models against adversarial attacks.

Exploiting Frequency Characteristics for Boosting the Invisibility of Adversarial Attacks

Mainstream transferable adversarial attacks tend to introduce noticeable artifacts into the generated adversarial examples, which will impair the invisibility of adversarial perturbation and make these attacks less practical in real-world scenarios. To deal with this problem, in this paper, we propose a novel black-box adversarial attack method that can significantly improve the invisibility of adversarial examples. We analyze the sensitivity of a deep neural network in the frequency domain and take into account the characteristics of the human visual system in order to quantify the contribution of each frequency component in adversarial perturbation. Then, we collect a set of candidate frequency components that are insensitive to the human visual system by applying K-means clustering and we propose a joint loss function during the generation of adversarial examples, limiting the frequency distribution of perturbations during attacks. The experimental results show that the proposed method significantly outperforms existing transferable black-box adversarial attack methods in terms of invisibility, which verifies the superiority, applicability and potential of this work.

A black-box adversarial attack on demand side management

Demand side management (DSM) contributes to the industry’s transition to renewables by shifting electricity consumption in time while maintaining feasible operations. Machine learning is promising for DSM with reasonable computation times and electricity price forecasting (EPF), which is paramount to obtaining the necessary data. Increased usage of machine learning makes production processes susceptible to so-called adversarial attacks. This work proposes a black-box attack on DSM and EPF based on an adversarial surrogate model that intercepts and modifies the data flow of load forecasts and forces the DSM to result in financial losses. Notably, adversaries can design the data modifications without knowledge of the EPF model or the DSM optimization model. The results show how barely noticeable modifications of the input data lead to significant deterioration of the decisions by the optimizer. The results implicate a significant threat, as attackers can design and implement powerful attacks without infiltrating secure company networks.

BFS2Adv: Black-box adversarial attack towards hard-to-attack short texts

The advent of Machine Learning as a Service (MLaaS) and deep learning applications has increased the susceptibility of models to adversarial textual attacks, particularly in black-box settings. Prior work on black-box adversarial textual attacks generally follows a stable strategy that involves leveraging char-level, world-level, and sentence-level perturbations, as well as using queries to the target model to find adversarial examples in the search space. However, existing approaches prioritize query efficiency by reducing the search space, thereby overlooking hard-to-attack textual instances. To address this issue, we propose BFS2Adv, a brute force algorithm that generates adversarial examples for both easy-to-attack and hard-to-attack textual inputs. BFS2Adv, starting with an original text, employs word-level perturbations and synonym substitution to construct a comprehensive search space, with each node representing a potential adversarial example. The algorithm systematically explores this space through a breadth-first search, combined with queries to the target model, to effectively identify qualified adversarial examples. We implemented and evaluated a prototype of BFS2Adv against renowned models such as ALBERT and BERT, utilizing the SNLI and MR datasets. Our results demonstrate that BFS2Adv outperforms state-of-the-art algorithms and effectively improves the success rate of short-text adversarial attacks. Furthermore, we provide detailed insights into the robustness of BFS2Adv by analyzing those hard-to-attack examples.

DifAttack: Query-Efficient Black-Box Adversarial Attack via Disentangled Feature Space

This work investigates efficient score-based black-box adversarial attacks with high Attack Success Rate (ASR) and good generalizability. We design a novel attack method based on a Disentangled Feature space, called DifAttack, which differs significantly from the existing ones operating over the entire feature space. Specifically, DifAttack firstly disentangles an image's latent feature into an adversarial feature and a visual feature, where the former dominates the adversarial capability of an image, while the latter largely determines its visual appearance. We train an autoencoder for the disentanglement by using pairs of clean images and their Adversarial Examples (AEs) generated from available surrogate models via white-box attack methods. Eventually, DifAttack iteratively optimizes the adversarial feature according to the query feedback from the victim model until a successful AE is generated, while keeping the visual feature unaltered. In addition, due to the avoidance of using surrogate models' gradient information when optimizing AEs for black-box models, our proposed DifAttack inherently possesses better attack capability in the open-set scenario, where the training dataset of the victim model is unknown. Extensive experimental results demonstrate that our method achieves significant improvements in ASR and query efficiency simultaneously, especially in the targeted attack and open-set scenarios. The code is available The code is available at https://github.com/csjunjun/DifAttack.git.

LRS: Enhancing Adversarial Transferability through Lipschitz Regularized Surrogate

The transferability of adversarial examples is of central importance to transfer-based black-box adversarial attacks. Previous works for generating transferable adversarial examples focus on attacking given pretrained surrogate models while the connections between surrogate models and adversarial trasferability have been overlooked. In this paper, we propose Lipschitz Regularized Surrogate (LRS) for transfer-based black-box attacks, a novel approach that transforms surrogate models towards favorable adversarial transferability. Using such transformed surrogate models, any existing transfer-based black-box attack can run without any change, yet achieving much better performance. Specifically, we impose Lipschitz regularization on the loss landscape of surrogate models to enable a smoother and more controlled optimization process for generating more transferable adversarial examples. In addition, this paper also sheds light on the connection between the inner properties of surrogate models and adversarial transferability, where three factors are identified: smaller local Lipschitz constant, smoother loss landscape, and stronger adversarial robustness. We evaluate our proposed LRS approach by attacking state-of-the-art standard deep neural networks and defense models. The results demonstrate significant improvement on the attack success rates and transferability. Our code is available at https://github.com/TrustAIoT/LRS.

Transferable Adversarial Attacks for Object Detection Using Object-Aware Significant Feature Distortion

Transferable black-box adversarial attacks against classifiers by disturbing the intermediate-layer features have been extensively studied in recent years. However, these methods have not yet achieved satisfactory performances when directly applied to object detectors. This is largely because the features of detectors are fundamentally different from that of the classifiers. In this study, we propose a simple but effective method to improve the transferability of adversarial examples for object detectors by leveraging the properties of spatial consistency and limited equivariance of object detectors’ features. Specifically, we combine a novel loss function and deliberately designed data augmentation to distort the backbone features of object detectors by suppressing significant features corresponding to objects and amplifying the surrounding vicinal features corresponding to object boundaries. As such the target object and background area on the generated adversarial samples are more likely to be confused by other detectors. Extensive experimental results show that our proposed method achieves state-of-the-art black-box transferability for untargeted attacks on various models, including one/two-stage, CNN/Transformer-based, and anchor-free/anchor-based detectors.

Generalizable Black-Box Adversarial Attack With Meta Learning.

In the scenario of black-box adversarial attack, the target model's parameters are unknown, and the attacker aims to find a successful adversarial perturbation based on query feedback under a query budget. Due to the limited feedback information, existing query-based black-box attack methods often require many queries for attacking each benign example. To reduce query cost, we propose to utilize the feedback information across historical attacks, dubbed example-level adversarial transferability. Specifically, by treating the attack on each benign example as one task, we develop a meta-learning framework by training a meta generator to produce perturbations conditioned on benign examples. When attacking a new benign example, the meta generator can be quickly fine-tuned based on the feedback information of the new task as well as a few historical attacks to produce effective perturbations. Moreover, since the meta-train procedure consumes many queries to learn a generalizable generator, we utilize model-level adversarial transferability to train the meta generator on a white-box surrogate model, then transfer it to help the attack against the target model. The proposed framework with the two types of adversarial transferability can be naturally combined with any off-the-shelf query-based attack methods to boost their performance, which is verified by extensive experiments. The source code is available at https://github.com/SCLBD/MCG-Blackbox.

Enhancing cybersecurity in smart grids: Deep black box adversarial attacks and quantum voting ensemble models for blockchain privacy-preserving storage

Smart grids are getting important in today’s power management, so with that, smart grid technologies are increasingly important too. There have been a lot of concerns about smart grid technologies being hacked, and as a result, some deep black box adversarial attacks have been conducted and presented. We propose a new experimental methodology for benchmarking smart grid security with black box attacks. Additionally, concerning the type of smart grids, Smart Power Grids, deep black box adversarial attacks which can be crafted using virtually no knowledge about the target due to the inherent complexity of content available in cryptographic libraries like SecLib or Bouncy Castle how it affects security of cyber-physical power systems. We identify potential impacts of deep black box attacks on Smart Power Grids as implemented by the Department of Energy in 1996, we evaluate existing protection methods, and we find out the pitfalls thereof. With the aim of overcoming the aforementioned drawbacks, we initiate a study on deep black box adversarial attacks against Smart Power Grids showing that statistically significant effects against a national Smart Power Grid are achievable with absolute security. We also probe detection of cyber security attacks on Smart Power Grids. We illustrate landscape of smart grids with numerous cyber threats and demonstrate the limitations of traditional security practices. We show the importance of machine learning to detect attacks and the unlikelihood of identification of dependable and efficient detection schemes. We describe quantum voting ensemble models as one of the most powerful techniques in the detection of cyber security attacks. Finally, we propose an experimental setup and evaluation criteria to detect cyber security attacks in smart grids using quantum voting ensemble models. Then, we talk about private data storage in blockchain based smart grid infrastructure. We give an introduction of block chain and its essentiality in smart grids. We discuss privacy issues in block chain based smart grids. We acknowledge the strength of privacy safeguards, but on the same wavelength, we realize their weaknesses. Next, we propose a quantum resistant encryption technique that enhances the privacy of smart grids. We propose quantum voting ensemble models as one of the most promising techniques to address the issue of private data storage in block chains. As a result, we provide a comparison between the proposed models and traditional approaches to privacy protection in smart grids based on an experimental performance review. Then, we propose a unified strategy to improve smart grid cyber security by incorporating deep black box attacks with quantum voting ensemble models. Finally, we disclose several benefits of such integration and perform an experimental evaluation to investigate the effectiveness of the unified approach. The results of our study identify security gaps in smart grids and propose state-of-the-art mechanisms to address them. The challenges of smart grids system require the amalgamation of blockchain, quantum voting ensemble models and deep black box adversarial attacks. We achieve this objective proposing a unified strategy. The results of this study will equally be helpful for future research and smart grid cyber security implementations.