Property-preserving hashing for $$\ell _1$$-distance predicates: Applications to countering adversarial input attacks

Many studies demonstrate that supervised learning techniques are vulnerable to adversarial examples. However, adversarial threats in unsupervised learning have not drawn sufficient scholarly attention. In this article, we formally address the unexplored adversarial attacks in the equally important unsupervised clustering field and propose the concept of the adversarial set and adversarial set attack for clustering. To illustrate the basic idea, we design a novel adversarial space-mapping attack algorithm to confuse subspace clustering, one of the mainstream branches of unsupervised clustering. It maps a sample into one wrong class by moving it towards the closest point on the linear subspace of the target class, that is, along the normal of the closest point. This simple single-step algorithm has the power to craft the adversarial set where the image samples can be wrongly clustered, even into the targeted labels. Empirical results on different image datasets verify the effectiveness and superiority of our algorithm. We further show that deep supervised learning algorithms (such as VGG and ResNet) are also vulnerable to our crafted adversarial set, which illustrates the good cross-task transferability of the adversarial set.

Image denoising can remove natural noise that widely exists in images captured by multimedia devices due to low-quality imaging sensors, unstable image transmission processes, or low light conditions. Recent works also find that image denoising benefits the high-level vision tasks, <i>e.g</i>., image classification. In this work, we try to challenge this common sense and explore a totally new problem, <i>i.e</i>., whether the image denoising can be given the capability of fooling the state-of-the-art deep neural networks (DNNs) while enhancing the image quality. To this end, we initiate the very first attempt to study this problem from the perspective of adversarial attack and propose the <i>adversarial denoise attack</i>. More specifically, our main contributions are three-fold: <i>First</i>, we identify a new task that stealthily embeds attacks inside the image denoising module widely deployed in multimedia devices as an image post-processing operation to simultaneously enhance the visual image quality and fool DNNs. <i>Second</i>, we formulate this new task as a kernel prediction problem for image filtering and propose the <i>adversarial-denoising kernel prediction</i> that can produce adversarial-noiseless kernels for effective denoising and adversarial attacking simultaneously. <i>Third</i>, we implement an adaptive <i>perceptual region localization</i> to identify semantic-related vulnerability regions with which the attack can be more effective while not doing too much harm to the denoising. We name the proposed method as <i>Pasadena</i> (Perceptually Aware and Stealthy Adversarial DENoise Attack) and validate our method on the NeurIPS&#x2019;17 adversarial competition dataset, CVPR2021-AIC-VI: unrestricted adversarial attacks on ImageNet, and Tiny-ImageNet-C dataset. The comprehensive evaluation and analysis demonstrate that our method not only realizes denoising but also achieves a significantly higher success rate and transferability over state-of-the-art attacks.

Person re-identification (re-ID) has attracted much attention recently due to its great importance in video surveillance. In general, distance metrics used to identify two person images are expected to be robust under various appearance changes. However, our work observes the extreme vulnerability of existing distance metrics to adversarial examples, generated by simply adding human-imperceptible perturbations to person images. Hence, the security danger is dramatically increased when deploying commercial re-ID systems in video surveillance. Although adversarial examples have been extensively applied for classification analysis, it is rarely studied in metric analysis like person re-identification. The most likely reason is the natural gap between the training and testing of re-ID networks, that is, the predictions of a re-ID network cannot be directly used during testing without an effective metric. In this work, we bridge the gap by proposing Adversarial Metric Attack, a parallel methodology to adversarial classification attacks. Comprehensive experiments clearly reveal the adversarial effects in re-ID systems. Meanwhile, we also present an early attempt of training a metric-preserving network, thereby defending the metric against adversarial attacks. At last, by benchmarking various adversarial settings, we expect that our work can facilitate the development of adversarial attack and defense in metric-based applications.

Adversarial attacks against Windows PE malware detection: A survey of the state-of-the-art

Neural networks play a significant role in the field of image classification. When an input image is modified by adversarial attacks, the changes are imperceptible to the human eye, but it still leads to misclassification of the images. Researchers have demonstrated these attacks to make production self-driving cars misclassify Stop Road signs as 45 Miles Per Hour (MPH) road signs and a turtle being misclassified as AK47. Three primary types of defense approaches exist which can safeguard against such attacks i.e., Gradient Masking, Robust Optimization, and Adversarial Example Detection. Very few approaches use Generative Adversarial Networks (GAN) for Defense against Adversarial Attacks. In this paper, we create a new approach to defend against adversarial attacks, dubbed Chained Dual-Generative Adversarial Network (CD-GAN) that tackles the defense against adversarial attacks by minimizing the perturbations of the adversarial image using iterative oversampling and undersampling using GANs. CD-GAN is created using two GANs, i.e., CDGAN’s Sub-Resolution GAN and CDGAN’s Super-Resolution GAN. The first is CDGAN’s Sub-Resolution GAN which takes the original resolution input image and oversamples it to generate a lower resolution neutralized image. The second is CDGAN’s Super-Resolution GAN which takes the output of the CDGAN’s Sub-Resolution and undersamples, it to generate the higher resolution image which removes any remaining perturbations. Chained Dual GAN is formed by chaining these two GANs together. Both of these GANs are trained independently. CDGAN’s Sub-Resolution GAN is trained using higher resolution adversarial images as inputs and lower resolution neutralized images as output image examples. Hence, this GAN downscales the image while removing adversarial attack noise. CDGAN’s Super-Resolution GAN is trained using lower resolution adversarial images as inputs and higher resolution neutralized images as output images. Because of this, it acts as an Upscaling GAN while removing the adversarial attak noise. Furthermore, CD-GAN has a modular design such that it can be pre-fixed to any existing classifier without any retraining or extra effort, and can defend any classifier model against adversarial attack. In this way, it is a Generalized Defense against adversarial attacks, capable of defending any classifier model against any attacks. This enables the user to directly integrate CD-GAN with an existing production deployed classifier smoothly. CD-GAN iteratively removes the adversarial noise using a multi-step approach in a modular approach. It performs comparably to the state of the arts with mean accuracy of 33.67 while using minimal compute resources in training.

Learning robust features against adversarial attacks is a challenging task that requires highly complex models, especially on aerial images, because they are subject to environmental and adversarial changes. Embedding hypersphere normalization, along with adversarial settings, causes performance degradation and enables the feature to overlap. To address this, in this article, we propose a dynamic hypersphere embedding scale (DHS) method that remaps the normalized features to a relative scale to learn robust features. The proposed method combines the benefits of hypersphere embedding without scarifying softmax advantages. The DHS aggregates the normalized features and the non-normalized ones. It uses a hypersphere embedding to enforce maximum-margin to the features that yield shorter magnitude and utilizes a dynamic scale to avoid features overlapping in the case of adversarial attacks. We validate the DHS's effectiveness by embedding the adversarial training attacks such as Projected Gradient Descent (PGD), CW, and DeepFool. Empirical experiments revealed that the DHS improves the model performance by 12% when using the PGD attack, with less computation than legacy hypersphere models. Another set of experiments showed that the DHS does not obfuscate the gradient.

With the increased usage of Internet of Things (IoT) devices in recent years, various Machine Learning (ML) algorithms have also developed dramatically for attack detection in this domain. However, the ML models are exposed to different classes of adversarial attacks that aim to fool a model into making an incorrect prediction. For instance, label manipulation or label flipping is an adversarial attack where the adversary attempts to manipulate the label of training data that causes the trained model biased and/or with decreased performance. However, the number of samples to be flipped in this category of attack can be restricted, giving the attacker a limited target selection. Due to the great significance of securing ML models against Adversarial Machine Learning (AML) attacks particularly in the IoT domain, this research presents an extensive review of AML in IoT. Then, a classification of AML attacks is presented based on the literature which sheds light on the future research in this domain. Next, this paper investigates the negative impact levels of applying the malicious label-flipping attacks on IoT data. We devise label-flipping scenarios for training a Support Vector Machine (SVM) model. The experiments demonstrate that the label flipping attacks impact the performance of ML models. These results can lead to designing more effective and powerful attack and defense mechanisms in adversarial settings. Finally, we show the weaknesses of the K-NN defense method against the random label flipping attack.

Adversarial attacks have demonstrated the ability to change image classification labels with minimal alterations to the input image across a wide variety of Deep Neural Network (DNN) algorithms. Specifically the Fast Gradient Attack (FGA) [2] method has been shown to change a model’s output classification label with minimal visual alterations. Goodfellow et. al. [1] conjectured that this may be due to the high dimensional nature of the input data, as well as the linearity of the Deep Learning (DL) algorithms. Our study investigates the efficacy of one mitigation strategy (attack defense), centered on a simple preprocessing method (median de-noising) on input images. The median filter is employed on images input into ResNet models of varying depth, which have been pre-trained on the CIFAR10 dataset. It is hypothesized that by employing this preprocessing mitigation technique to adversarial attacked images, the impact of the FGA method [1] can be reduced. This study is designed to test if de-noising an input image can mitigate adversarial attack effectiveness. Additionally, the median filter removes some information contained within the dataset. Therefore, we seek to characterize and quantify the robustness of the ResNet models with this transformation technique. This research provides insight into the nature of the FGA adversarial attack vector, a simple attack mitigation strategy, as well as the robustness of the Deep Learning models being studied. [1] I. J. Goodfellow, J. Shlens and C. Szegedy, Explaining and Harnessing Adversarial Examples, arXiv e-prints, p. 1412.6572 December 2014 [2] Nicolae, Maria-Irina and Sinn, Mathieu and Tran, Minh Ngoc and Buesser, Beat and Rawat, Ambrish and Wistuba, Martin and Zantedeschi, Valentina and Baracaldo, Nathalie and Chen, Bryant and Ludwig, Heiko and Molloy, Ian and Edwards, Ben, Adversarial Robustness Toolbox v1.2.0 CoRR 1807.01069 https://arxiv.org/pdf/1807.01069 2018

The successful emergence of deep learning (DL) in wireless system applications has raised concerns about new security-related challenges. One such security challenge is adversarial attacks. Although there has been much work demonstrating the susceptibility of DL-based classification tasks to adversarial attacks, regression-based problems in the context of a wireless system have not been studied so far from an attack perspective. The aim of this paper is twofold: (i) we consider a regression problem in a wireless setting and show that adversarial attacks can break the DL-based approach and (ii) we analyze the effectiveness of adversarial training as a defensive technique in adversarial settings and show that the robustness of DL-based wireless system against attacks improves significantly. Specifically, the wireless application considered in this paper is the DL-based power allocation in the downlink of a multicell massive multi-input-multi-output system, where the goal of the attack is to yield an infeasible solution by the DL model. We extend the gradient-based adversarial attacks: fast gradient sign method (FGSM), momentum iterative FGSM, and projected gradient descent method to analyze the susceptibility of the considered wireless application with and without adversarial training. We analyze the deep neural network (DNN) models performance against these attacks, where the adversarial perturbations are crafted using both the white-box and black-box attacks.

A steganography technique hides a secret message stealthily within multimedia files such as images, videos, or even the skin image of an avatar in a metaverse environment. Conversely, a steganalysis technique detects steganographic files containing hidden messages. Recently, with the rapid advancement of Convolutional Neural Network (CNN) architectures, CNN-based image steganalysis models have been proposed to accurately detect steganography in image files. Meanwhile, Deep Learning (DL) models, including CNNs, are known to be vulnerable to evasion attacks such as adversarial example attacks, which can cause a CNN-based classifier to misclassify an input image according to the attacker’s intent. Given the lack of prior research in this domain, this paper investigates how effectively state-of-the-art adversarial example attack methods can evade three representative CNN-based image steganalysis ML models (XuNet, YeNet, and SRNet). Specifically, we first describe a system model consisting of three participating entities—a naïve attacker, a defender (Defender Lv. 1 and Defender Lv. 2), and an adversarial attacker. Next, we present experimental results comparing nine adversarial example attack methods against the three representative CNN models in terms of various metrics, including classification accuracy (CA), missed detection rate (MDR), attack success index (ASI), and adversarial example generation time (AEGT).

Speaker recognition is a task that identifies the speaker from multiple audios. Recently, advances in deep learning have considerably boosted the development of speech signal processing techniques. Speaker or speech recognition has been widely adopted in such applications as smart locks, smart vehicle-mounted systems, and financial services. However, deep neural network-based speaker recognition systems (SRSs) are susceptible to adversarial attacks, which fool the system to make wrong decisions by small perturbations, and this has drawn the attention of researchers to the security of SRSs. Unfortunately, there is no systematic review work in this domain. In this work, we conduct a comprehensive survey to fill this gap, which includes the development of SRSs, adversarial attacks and defenses against SRSs. Specifically, we first introduce the mainstream frameworks of SRSs and some commonly used datasets. Then, from the perspectives of adversarial example generation and evaluation, we introduce different attack tasks, the prior knowledge of attacks, perturbation objects, perturbation constraints, and attack effect evaluation indicators. Next, we focus on some effective defense strategies, including adversarial training, attack detection, and input refactoring against existing attacks, and analyze their strengths and weaknesses in terms of fidelity and robustness. Finally, we discuss the challenges posed by audio adversarial examples in SRSs and some valuable research topics in the future.

Deep learning and machine learning have emerged as a prominent domain which has various applications. On the other hand, these models can easily be fooled by different adversarial attacks by adding insignificant adversarial perturbation which yields misclassifying the image. This becomes a potential threat to the deep learning training, testing, and deployed model, as adversarial attacks are unpreventable and harder to detect than previously assumed. The extenuation of this adversarial permutation input remains a challenging domain problem. A step toward understanding adversarial attack and defense, this review paper specifically focuses on a comprehensive review of different methods to be adopted for the deep learning, machine learning models.KeywordsRecurrent neural networksDeep learningReinforcement learningPerturbationBlack-box attackConvolutional neural networksWhite-box attackAdversarial learningAdversaryDeepFoolQuasi-imperceptibleFast gradient sign methodMiscellaneous attacksAutoencodersGradient regularizationDefensive distillationLoss functionsDeep neural network

Adversarial attacks on a convolutional neural network (CNN)—injecting human-imperceptible perturbations into an input image—could fool a high-performance CNN into making incorrect predictions. The success of adversarial attacks raises serious concerns about the robustness of CNNs, and prevents them from being used in safety-critical applications, such as medical diagnosis and autonomous driving. Our work introduces a visual analytics approach to understanding adversarial attacks by answering two questions: (1) Which neurons are more vulnerable to attacks? and (2) Which image features do these vulnerable neurons capture during the prediction? For the first question, we introduce multiple perturbation-based measures to break down the attacking magnitude into individual CNN neurons and rank the neurons by their vulnerability levels. For the second, we identify image features (e.g., cat ears) that highly stimulate a user-selected neuron to augment and validate the neuron’s responsibility. Furthermore, we support an interactive exploration of a large number of neurons by aiding with hierarchical clustering based on the neurons’ roles in the prediction. To this end, a visual analytics system is designed to incorporate visual reasoning for interpreting adversarial attacks. We validate the effectiveness of our system through multiple case studies as well as feedback from domain experts.

Image processing networks are known to be vulnerable to adversarial examples, where adding carefully crafted adversarial perturbations to the inputs can mislead the model. This paper addresses the problem of robust blind face restoration (BFR) against adversarial attacks. BFR refers to recovering the HQ images from the LQ images, which suffer from diverse unknown degradation, such as noise, blur, artifact removal, low resolution, etc. Although existing BFR methods exhibit good performance, they experience significant degradation when subtle distortions and perturbations are introduced into the input images. This paper is the first to investigate, improve comprehensively, and evaluate BFR methods towards adversarial attacks. Project Gradient Descent (PGD) is employed to generate adversarial examples, and multiple types of attacks were used to thoroughly assess the robustness of various BFR methods across different objectives, regions, and levels. We evaluate the robustness of multiple BFR methods and analyze the advantages of their structures and modules towards adversarial attacks. Experimental results demonstrate that the method utilizing latent feature encoding and pre-trained discrete HQ codebook achieves better robustness than other methods, with the latter outperforming the former. Similarly, multi-scale semantic guidance information also exhibits superior performance in enhancing robustness. Therefore, we propose a powerful BFR method to mitigate this issue while maintaining better performance. Extensive experiments on three real-world datasets demonstrate our method’s state-of-the-art robustness in different scenarios.

Adversarial attacks have exposed serious vulnerabilities in deep neural networks (DNNs), causing misclassifications through human-imperceptible perturbations to DNN inputs. We explore a new direction in the field of adversarial attacks by suggesting attacks that aim to degrade the energy or latency of DNNs rather than their classification accuracy. As a specific embodiment of this new threat vector, we propose and demonstrate adversarial sparsity attacks, which modify a DNN's inputs so as to reduce sparsity (or the incidence of zeros) in its internal activation values. Exploiting sparsity in hardware and software has emerged as a popular approach to improve DNN efficiency in resource-constrained systems. The proposed attack, therefore, increases the execution time and energy consumption of sparsity-optimized DNN implementations, raising concern over their deployment in latency and energy-critical applications. We propose a systematic methodology to generate adversarial inputs for sparsity attacks by formulating an objective function that quantifies the network's activation sparsity and minimizing this function using iterative gradient-descent techniques. To prevent easy detection of the attack, we further ensure that the perturbation magnitude is within a specified constraint and that the perturbation does not affect classification accuracy. We launch both white-box and black-box versions of adversarial sparsity attacks on image recognition DNNs and demonstrate that they decrease activation sparsity by 1.16×-1.82×. On a sparsity-optimized DNN accelerator, the attack results in degradations of 1.12×-1.59× in latency and 1.18×-1.99× in energy-delay product (EDP). Additionally, we analyze the impact of various hyperparameters and constraints on the attack's efficacy. Finally, we evaluate defense techniques, such as activation thresholding and input quantization and demonstrate that the proposed attack is able to withstand them, highlighting the need for further efforts in this new direction within the field of adversarial machine learning.