Evasion Attacks Research Articles

AdverSPAM: Adversarial SPam Account Manipulation in Online Social Networks

In recent years, the widespread adoption of Machine Learning (ML) at the core of complex IT systems has driven researchers to investigate the security and reliability of ML techniques. A very specific kind of threats concerns the adversary mechanisms through which an attacker could induce a classification algorithm to provide the desired output. Such strategies, known as Adversarial Machine Learning (AML), have a twofold purpose: to calculate a perturbation to be applied to the classifier’s input such that the outcome is subverted, while maintaining the underlying intent of the original data. Although any manipulation that accomplishes these goals is theoretically acceptable, in real scenarios perturbations must correspond to a set of permissible manipulations of the input, which is rarely considered in the literature. In this article, we present AdverSPAM , an AML technique designed to fool the spam account detection system of an Online Social Network (OSN). The proposed black-box evasion attack is formulated as an optimization problem that computes the adversarial sample while maintaining two important properties of the feature space, namely statistical correlation and semantic dependency . Although being demonstrated in an OSN security scenario, such an approach might be applied in other context where the aim is to perturb data described by mutually related features. Experiments conducted on a public dataset show the effectiveness of AdverSPAM compared to five state-of-the-art competitors, even in the presence of adversarial defense mechanisms.

Improving Adversarial Robustness of Ensemble Classifiers by Diversified Feature Selection and Stochastic Aggregation

Learning-based classifiers are found to be vulnerable to attacks by adversarial samples. Some works suggested that ensemble classifiers tend to be more robust than single classifiers against evasion attacks. However, recent studies have shown that this is not necessarily the case under more realistic settings of black-box attacks. In this paper, we propose a novel ensemble approach to improve the robustness of classifiers against evasion attacks by using diversified feature selection and a stochastic aggregation strategy. Our proposed scheme includes three stages. Firstly, the adversarial feature selection algorithm is used to select a feature each time that can trade-offbetween classification accuracy and robustness, and add it to the feature vector bank. Secondly, each feature vector in the bank is used to train a base classifier and is added to the base classifier bank. Finally, m classifiers from the classifier bank are randomly selected for decision-making. In this way, it can cause each classifier in the base classifier bank to have good performance in terms of classification accuracy and robustness, and it also makes it difficult to estimate the gradients of the ensemble accurately. Thus, the robustness of classifiers can be improved without reducing the classification accuracy. Experiments performed using both Linear and Kernel SVMs on genuine datasets for spam filtering, malware detection, and handwritten digit recognition demonstrate that our proposed approach significantly improves the classifiers’ robustness against evasion attacks.

PAD: Towards Principled Adversarial Malware Detection Against Evasion Attacks

Machine Learning (ML) techniques can facilitate the automation of <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mal</u> icious soft <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ware</u> (malware for short) detection, but suffer from evasion attacks. Many studies counter such attacks in heuristic manners, lacking theoretical guarantees and defense effectiveness. In this paper, we propose a new adversarial training framework, termed <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</u> rincipled <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> dversarial Malware <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</u> etection (PAD), which offers convergence guarantees for robust optimization methods. PAD lays on a learnable convex measurement that quantifies distribution-wise discrete perturbations to protect malware detectors from adversaries, whereby for smooth detectors, adversarial training can be performed with theoretical treatments. To promote defense effectiveness, we propose a new mixture of attacks to instantiate PAD to enhance deep neural network-based measurements and malware detectors. Experimental results on two Android malware datasets demonstrate: (i) the proposed method significantly outperforms the state-of-the-art defenses; (ii) it can harden ML-based malware detection against 27 evasion attacks with detection accuracies greater than 83.45%, at the price of suffering an accuracy decrease smaller than 2.16% in the absence of attacks; (iii) it matches or outperforms many anti-malware scanners in VirusTotal against realistic adversarial malware.

Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance

Emerging Distributed AI systems are revolutionizing big data computing and data processing capabilities with growing economic and societal impact. However, recent studies have identified new attack surfaces and risks caused by security, privacy, and fairness issues in AI systems. In this paper, we review representative techniques, algorithms, and theoretical foundations for trustworthy distributed AI through robustness guarantee, privacy protection, and fairness awareness in distributed learning. We first provide a brief overview of alternative architectures for distributed learning, discuss inherent vulnerabilities for security, privacy, and fairness of AI algorithms in distributed learning, and analyze why these problems are present in distributed learning regardless of specific architectures. Then we provide a unique taxonomy of countermeasures for trustworthy distributed AI, covering (1) robustness to evasion attacks and irregular queries at inference, and robustness to poisoning attacks, Byzantine attacks, and irregular data distribution during training; (2) privacy protection during distributed learning and model inference at deployment; and (3) AI fairness and governance with respect to both data and models. We conclude with a discussion on open challenges and future research directions toward trustworthy distributed AI, such as the need for trustworthy AI policy guidelines, the AI responsibility-utility co-design, and incentives and compliance.

Explanatory subgraph attacks against Graph Neural Networks

Graph Neural Networks (GNNs) are often viewed as black boxes due to their lack of transparency, which hinders their application in critical fields. Many explanation methods have been proposed to address the interpretability issue of GNNs. These explanation methods reveal explanatory information about graphs from different perspectives. However, the explanatory information may also pose an attack risk to GNN models.In this work, we will explore this problem from the explanatory subgraph perspective. To this end, we utilize a powerful GNN explanation method called SubgraphX and deploy it locally to obtain explanatory subgraphs from given graphs. Then we propose methods for conducting evasion attacks and backdoor attacks based on the local explainer. In evasion attacks, the attacker gets explanatory subgraphs of test graphs from the local explainer and replace their explanatory subgraphs with an explanatory subgraph of other labels, making the target model misclassify test graphs as wrong labels. In backdoor attacks, the attacker employs the local explainer to select an explanatory trigger and locate suitable injection locations. We validate the effectiveness of our proposed attacks on state-of-art GNN models and different datasets. The results also demonstrate that our proposed backdoor attack is more efficient, adaptable, and concealed than previous backdoor attacks.

Visual Prompt Engineering for Enhancing Facial Recognition Systems Robustness Against Evasion Attacks

Visual Prompt Engineering for Enhancing Facial Recognition Systems Robustness Against Evasion Attacks

Evasion Attack and Defense on Machine Learning Models in Cyber-Physical Systems: A Survey

Evasion Attack and Defense on Machine Learning Models in Cyber-Physical Systems: A Survey

MTDroid: A Moving Target Defense-Based Android Malware Detector Against Evasion Attacks

MTDroid: A Moving Target Defense-Based Android Malware Detector Against Evasion Attacks

Evasion Attacks and Defense Mechanisms for Machine Learning-Based Web Phishing Classifiers

Evasion Attacks and Defense Mechanisms for Machine Learning-Based Web Phishing Classifiers

Automatic Evasion of Machine Learning-Based Network Intrusion Detection Systems

Network intrusion detection systems (IDS) are often considered effective to thwart cyber attacks. Currently, state-of-the-art (SOTA) IDSs are mainly based on machine learning (ML) including deep learning (DL) models, which suffer from their own security issues, especially evasion attacks by using adversarial examples. However, previous studies mostly focus on extracted features rather than the traffic sample itself, and/or assume that the adversary knows the information of the target model more or less, which severely restricts attack feasibility in practice. In this paper, we re-investigate this problem in a more realistic label-only black-box scenario and propose a practical evasion attack strategy to solve the above limitations. In this newly considered case that the adversary morphs the traffic sample and only obtains the results accepted or rejected without other knowledge, we successfully leverage the model extraction and transfer attack to evade the detection. The entire attack strategy is automated and a comprehensive evaluation is performed. Final results show that the proposed strategy effectively evades seven typical ML-based IDSs and one SOTA DL-based IDS with an average success rate of over 75%. We also discuss the corresponding countermeasures against our attack, which finally highlight the need for effective defenses against our attack.

Spatio-Temporal Graph-Based Generation and Detection of Adversarial False Data Injection Evasion Attacks in Smart Grids

Spatio-Temporal Graph-Based Generation and Detection of Adversarial False Data Injection Evasion Attacks in Smart Grids

GAN-Based Evasion Attack in Filtered Multicarrier Waveforms Systems

GAN-Based Evasion Attack in Filtered Multicarrier Waveforms Systems

EvadeDroid: A practical evasion attack on machine learning for black-box Android malware detection

Over the last decade, researchers have extensively explored the vulnerabilities of Android malware detectors to adversarial examples through the development of evasion attacks; however, the practicality of these attacks in real-world scenarios remains arguable. The majority of studies have assumed attackers know the details of the target classifiers used for malware detection, while in reality, malicious actors have limited access to the target classifiers. This paper introduces EvadeDroid, a problem-space adversarial attack designed to effectively evade black-box Android malware detectors in real-world scenarios. EvadeDroid constructs a collection of problem-space transformations derived from benign donors that share opcode-level similarity with malware apps by leveraging an n-gram-based approach. These transformations are then used to morph malware instances into benign ones via an iterative and incremental manipulation strategy. The proposed manipulation technique is a query-efficient optimization algorithm that can find and inject optimal sequences of transformations into malware apps. Our empirical evaluations, carried out on 1K malware apps, demonstrate the effectiveness of our approach in generating real-world adversarial examples in both soft- and hard-label settings. Our findings reveal that EvadeDroid can effectively deceive diverse malware detectors that utilize different features with various feature types. Specifically, EvadeDroid achieves evasion rates of 80%–95% against DREBIN, Sec-SVM, ADE-MA, MaMaDroid, and Opcode-SVM with only 1–9 queries. Furthermore, we show that the proposed problem-space adversarial attack is able to preserve its stealthiness against five popular commercial antiviruses with an average of 79% evasion rate, thus demonstrating its feasibility in the real world.

Hybrid Method for the Detection of Evasion Attacks Aimed at Machine Learning Systems

Hybrid Method for the Detection of Evasion Attacks Aimed at Machine Learning Systems

Topological safeguard for evasion attack interpreting the neural networks’ behavior

In the last years, Deep Learning technology has been proposed in different fields, bringing many advances in each of them, but raising new threats in these solutions regarding cybersecurity. Those implemented models have brought several vulnerabilities associated with Deep Learning technology. Moreover, those allow taking advantage of the implemented model, obtaining private information, and even modifying the model’s decision-making. Therefore, interest in studying those vulnerabilities/attacks and designing defenses to avoid or fight them is gaining prominence among researchers. In particular, the widely known evasion attack is being analyzed by researchers; thus, several defenses to avoid such a threat can be found in the literature. Since the presentation of the L-BFG algorithm, this threat concerns the research community. However, it continues developing new and ingenious countermeasures since there is no perfect defense for all the known evasion algorithms. In this work, a novel detector of evasion attacks is developed. It focuses on the information on the activations of the neurons given by the model when an input sample is injected. Moreover, it pays attention to the topology of the targeted deep learning model to analyze the activations according to which neurons are connecting. This approach is motivated from the observation that the literature shows that the targeted model’s topology contains essential information about if the evasion attack occurs. For this purpose, a huge data preprocessing is required to introduce all this information in the detector, which uses the Graph Convolutional Neural Network (GCN) technology. Thus, it understands the topology of the target model, obtaining promising results and improving the outcomes presented in the literature related to similar defenses.

A survey of strategy-driven evasion methods for PE malware: Transformation, concealment, and attack

The continuous proliferation of malware poses a formidable threat to the cyberspace landscape. Researchers have proffered a multitude of sophisticated defense mechanisms aimed at its detection and mitigation. Nevertheless, malware writers persistently pursue pioneering and innovative methods to evade detection by security software, thereby presenting an ever-evolving and dynamic threat to computer systems. Malware evasion refers to the use of certain strategies by malware to evade the detection of security software. Despite numerous surveys on malware evasion techniques, the existing surveys were fragmented and focused on specific types of evasion methods, leading to a lack of systematic and comprehensive research on malware evasion approaches. To fill this gap, this paper proposed a strategy-driven framework from the perspective of malware writers. Based on this framework, we categorize existing evasion detection techniques into transformation (alter the structural and behavioral pattern of the malware), concealment (conceal the behavior of the malware), and attack-based (engage in an attack on the detector to render it inoperable) methods and conduct a comprehensive survey of the relevant research works. In addition, we demonstrate how to integrate existing evasion strategies in the process of generating malware from the perspective of malware writers to subvert the multiple defenses of defenders. Our investigation indicates that: 1) evasion techniques such as packer and code obfuscation remain the foremost selection for attackers, no fewer than 10 off-the-shelf tools provide great assistance to them, 2) environment analysis is the primary concealment-based strategy used by the attacker (48% of the reviewed concealment-based strategy), defenders need greater efforts to counter them, 3) only 3 works discussed techniques for evasion attacks by leveraging fragilities in antivirus engines, meaning that direct attack on the detector is no longer as effective, 4) reinforcement learning algorithm serves as the most popular adversarial attack-based methods and 50% of works based on reinforcement learning are effective against real-world antivirus engines. Furthermore, this paper delves into the development trends in evasive malware and open issues for defenders. The primary objective of this survey is to furnish researchers and practitioners with a thorough comprehension of malware evasion strategies and techniques, thereby fostering the advancement of more potent and efficient approaches to detect and thwart malware.

Adversarial attacks and defenses on ML- and hardware-based IoT device fingerprinting and identification

In the last years, the number of IoT devices deployed has suffered an undoubted explosion, reaching the scale of billions. However, some new cybersecurity issues have appeared together with this development. Some of these issues are the deployment of unauthorized devices, malicious code modification, malware deployment, or vulnerability exploitation. This fact has motivated the requirement for new device identification mechanisms based on behavior monitoring. Besides, these solutions have recently leveraged Machine and Deep Learning (ML/DL) techniques due to the advances in this field and the increase in processing capabilities. In contrast, attackers do not stay stalled and have developed adversarial attacks focused on context modification and ML/DL evaluation evasion applied to IoT device identification solutions. However, literature has not yet analyzed in detail the impact of these attacks on individual identification solutions and their countermeasures. This work explores the performance of hardware behavior-based individual device identification, how it is affected by possible context- and ML/DL-focused attacks, and how its resilience can be improved using defense techniques. In this sense, it proposes an LSTM-CNN architecture based on hardware performance behavior for individual device identification. Then, the most usual ML/DL classification techniques have been compared with the proposed architecture using a hardware performance dataset collected from 45 Raspberry Pi devices running identical software. The LSTM-CNN improves previous solutions achieving a ＋0.96 average F1-Score and 0.8 minimum TPR for all devices. Afterward, context- and ML/DL-focused adversarial attacks were applied against the previous model to test its robustness. A temperature-based context attack was not able to disrupt the identification, but some ML/DL state-of-the-art evasion attacks were successful. Finally, adversarial training and model distillation defense techniques are selected to improve the model resilience to evasion attacks, improving its robustness from up to 0.88 attack success ratio to 0.17 in the worst attack case, without degrading its performance in an impactful manner.

Deep reinforcement learning based Evasion Generative Adversarial Network for botnet detection

Botnet detectors based on machine learning are potential targets for adversarial evasion attacks. Several research works employ adversarial training with samples generated from generative adversarial nets (GANs) to make the botnet detectors adept at recognising adversarial evasions. However, the synthetic evasions may not follow the original semantics of the input samples. This paper proposes a novel GAN model leveraged with deep reinforcement learning (DRL) to explore semantic aware samples and simultaneously harden its detection. A DRL agent is used to attack the discriminator of the GAN that acts as a botnet detector. The agent trains the discriminator on the crafted perturbations during the GAN training, which helps the GAN generator converge earlier than the case without DRL. We name this model RELEVAGAN, i.e. [“relieve a GAN” or deep REinforcement Learning-based Evasion Generative Adversarial Network] because, with the help of DRL, it minimises the GAN’s job by letting its generator explore the evasion samples within the semantic limits. During the GAN training, the attacks are conducted to adjust the discriminator weights for learning crafted perturbations by the agent. RELEVAGAN does not require adversarial training for the ML classifiers since it can act as an adversarial semantic-aware botnet detection model. The code will be available at https://github.com/rhr407/RELEVAGAN.

Adversarial Attacks Against Machine Learning-Based Resource Provisioning Systems

Microarchitectural attacks, such as side-channel, exploit shared resources to leak sensitive information. Performing microarchitectural attacks on the cloud is possible once the attacker’s virtual machine (VM) is co-located with the victim’s VM. Hence, the co-location requirement with the victim limits the practicality of microarchitectural attacks on the cloud. In this work, we demonstrate that resource provisioning systems (RPSs) can be exploited to solve the co-location challenge of microarchitectural attacks in the cloud by deploying adversarial evasion attacks on RPSs to co-locate attackers’ VMs with victims’ VMs. Moreover, we discuss the adaptability of defense techniques proposed against adversarial attacks in the image classification domain on the RPSs.

A Survey and Evaluation of Android-Based Malware Evasion Techniques and Detection Frameworks

Android platform security is an active area of research where malware detection techniques continuously evolve to identify novel malware and improve the timely and accurate detection of existing malware. Adversaries are constantly in charge of employing innovative techniques to avoid or prolong malware detection effectively. Past studies have shown that malware detection systems are susceptible to evasion attacks where adversaries can successfully bypass the existing security defenses and deliver the malware to the target system without being detected. The evolution of escape-resistant systems is an open research problem. This paper presents a detailed taxonomy and evaluation of Android-based malware evasion techniques deployed to circumvent malware detection. The study characterizes such evasion techniques into two broad categories, polymorphism and metamorphism, and analyses techniques used for stealth malware detection based on the malware’s unique characteristics. Furthermore, the article also presents a qualitative and systematic comparison of evasion detection frameworks and their detection methodologies for Android-based malware. Finally, the survey discusses open-ended questions and potential future directions for continued research in mobile malware detection.