Adversarial Environment Research Articles

Translation certification for smart contracts

Compiler correctness is an old problem, but with the emergence of smart contracts on blockchains that problem presents itself in a new light. Smart contracts are self-contained pieces of software that control (valuable) assets in an adversarial environment; once committed to the blockchain, these smart contracts cannot be modified. Smart contracts are typically developed in a high-level contract language and compiled to low-level virtual machine code before being committed to the blockchain. For a smart contract user to trust a given piece of low-level code on the blockchain, they must convince themselves that (a) they are in possession of the matching source code and (b) that the compiler has correctly translated the source code to the given low-level code.Classic approaches to compiler correctness tackle the second point. We argue that translation certification also squarely addresses the first. We describe the proof architecture of a translation certification framework and demonstrate how we can model the compilation pipeline as a sequence of translation relations. We give a detailed account of such relations for most passes of the Plutus Tx compiler, which we formalised in Coq. This approach facilitates a modular verification methodology and is robust in the face of an evolving compiler implementation.

Leveraging Machine Learning for Generating and Utilizing Motion Primitives in Adversarial Environments

Motion primitives enable fast planning for complex and dynamic environments. Adversarial environments pose a particularly challenging and unpredictable scenario. Motion-primitive-based planners have the potential to provide benefit in these types of environments. The key challenge is to design a library of maneuvers that effectively capture the necessary capabilities of the vehicle. This work presents a primitive-based game tree search to solve adversarial games in continuous state and action spaces and applies a reinforcement learning framework to autonomously generate effective primitives for the given task. The results demonstrate the ability of the learning framework to produce maneuvers necessary for competing against adversaries. Furthermore, we propose a method for learning a model to estimate the state-dependent value of each motion primitives and demonstrate how to incorporate this model to increase planning performance under time constraints. Additionally, we compare our primitive-based algorithm against forward simulated methods from existing literature and highlight the benefits of motion primitives.

Developing multi-agent adversarial environment using reinforcement learning and imitation learning

Developing multi-agent adversarial environment using reinforcement learning and imitation learning

Navigation in Adversarial Environments Guided by PRA* and a Local RL Planner

Real-time strategy games require players to respond to short-term challenges (micromanagement) and long-term objectives (macromanagement) simultaneously to win. However, many players excel at one of these skills but not both. This research is motivated by the question of whether the burden of micromanagement can be reduced on human players through delegation of responsibility to autonomous agents. In particular, this research proposes an adversarial navigation architecture that enables units to autonomously navigate through places densely populated with enemies by learning to micromanage itself. Our approach models the adversarial pathfinding problem as a Markov Decision Process (MDP) and trains an agent with reinforcement learning on this MDP. We observed that our approach resulted in the agent taking less damage from adversaries while travelling shorter paths, compared to previous approaches for adversarial navigation. Our approach is also efficient in memory use and computation time. Interestingly, the agent using the proposed approach also outperformed baseline approaches while navigating through environments that are significantly different from the training environments. Furthermore, when the game design is modified, the agent discovers effective alternate strategies considering the updated design without any changes in the learning framework. This property is particularly useful because in game development the design of a game is often updated iteratively.

A soft actor-critic reinforcement learning algorithm for network intrusion detection

Network intrusion detection plays a very important role in network security. Although current deep learning-based intrusion detection algorithms have achieved good detection performance, there are still limitations in dealing with unbalanced datasets and identifying minority attacks and unknown attacks. In this paper, we propose an intrusion detection model AE-SAC based on adversarial environment learning and soft actor-critic reinforcement learning algorithm. First, this paper introduces an environmental agent for training data resampling to solve the imbalance problem of the original data. Second, rewards are redefined in reinforcement learning. In order to improve the recognition rate of few categories of network attacks, we set different reward values for different categories of attacks. The environment agent and classifier agent are trained adversarially around maximizing their respective reward values. Finally, a multi-classification experiment is conducted on the NSL-KDD and AWID datasets to compare with the existed excellent intrusion detection algorithms. AE-SAC achieves excellent classification performance with an accuracy of 84.15% and a f1-score of 83.97% on the NSL-KDD dataset, and an accuracy and a f1-score over 98.9% on the AWID dataset.

A Multi-Agent Intrusion Detection System Optimized by a Deep Reinforcement Learning Approach with a Dataset Enlarged Using a Generative Model to Reduce the Bias Effect

Intrusion detection systems can defectively perform when they are adjusted with datasets that are unbalanced in terms of attack data and non-attack data. Most datasets contain more non-attack data than attack data, and this circumstance can introduce biases in intrusion detection systems, making them vulnerable to cyberattacks. As an approach to remedy this issue, we considered the Conditional Tabular Generative Adversarial Network (CTGAN), with its hyperparameters optimized using the tree-structured Parzen estimator (TPE), to balance an insider threat tabular dataset called the CMU-CERT, which is formed by discrete-value and continuous-value columns. We showed through this method that the mean absolute errors between the probability mass functions (PMFs) of the actual data and the PMFs of the data generated using the CTGAN can be relatively small. Then, from the optimized CTGAN, we generated synthetic insider threat data and combined them with the actual ones to balance the original dataset. We used the resulting dataset for an intrusion detection system implemented with the Adversarial Environment Reinforcement Learning (AE-RL) algorithm in a multi-agent framework formed by an attacker and a defender. We showed that the performance of detecting intrusions using the framework of the CTGAN and the AE-RL is significantly improved with respect to the case where the dataset is not balanced, giving an F1-score of 0.7617.

Geographic-Region Monitoring by Drones in Adversarial Environments

We consider surveillance of a geographic region by a collaborative system of drones. The drones assist each other in identifying and managing activities of interest on the ground. We also consider an adversary who can create both genuine and fake activities on the ground. The objective of the adversary is to use fake activities to maximize the response time to genuine activities. We present two collaboration algorithms and analyze their response times, as well as the adversary’s efforts in terms of the number of fake activities required to achieve a certain response time.

Secure IoU in Adversarial Environments: User Authentication and Performance Optimization

The emerging UAV network suffers from severe security risks from attackers. If the attackers sneak into an Internet of UAV (IoU) network, they would occupy the scarce resources which should be allocated among authenticated users, degrading the communication performance. Therefore, a two-stage scheme is proposed to both enhance IoU security and optimize network resource allocation. In the first stage, a distributed radio-frequency fingerprinting-based authentication model with consensus voting is designed to enhance the IoU security. Moreover, in the second stage, a performance optimization problem in consideration of authentication accuracy is formulated to maximize the data rate. It is proved that the problem can be transformed into a potential game, which implies that the optimization of the overall network's utility equals the optimization of each individual's utility. Also, the existence of this game's Nash equilibrium (NE) is proved and an iterative algorithm is used to achieve its NE. Finally, simulation results show that that our proposed consensus-based authentication model improves recognition accuracy by 40\%compared to the conventional Adaboost method under typical parameter settings, and our proposed algorithm can improve the network utility significantly. The effect of various parameters on the utility is also numerically verified.

Joint task offloading and UAVs deployment for UAV-assisted mobile edge computing

Recent advances in artificial intelligence (AI) have impacts on the development of communication networks. Diversified applications and scenarios put forward AI-native requirement for communication network. The network is envisioned to offer high data rate and low-latency communications, pervasive artificial intelligence, full coverage and programmable service. With the aid of AI, autonomous unmanned aerial vehicles (UAVs) play an indispensable role in the emerging scenarios. Owing to their flexibility and scalability, UAVs can be utilized as the mobile platform to deliver various kinds of services in dynamic and adversarial environments, especially the emergency scenarios. In this article, we deal with the post-disaster scenarios that UAVs, equipped with computing servers, work with the remaining infrastructures to provide communication and computing services for the ground mobile users (GMUs). With the aim of minimizing the energy cost and computing delay of task offloading, a two-layer optimization mechanism that can dynamically orchestrate the task offloading decision and the UAVs deployment is proposed. In lower layer, the problem of making offloading decisions is formulated into an evolutionary game. Replicator dynamics is exploited to help users to make decisions. In upper layer, a clustering approach for users is introduced to avoid unbalanced load distribution. By suggesting the locations and number of UAVs based the clusters, system cost is further reduced. The fairness of service load among UAVs is also achieved. Numerical results demonstrate the effectiveness of the proposed two-layer optimization mechanism.

Robust Planning over Restless Groups: Engagement Interventions for a Large-Scale Maternal Telehealth Program

In 2020, maternal mortality in India was estimated to be as high as 130 deaths per 100K live births, nearly twice the UN's target. To improve health outcomes, the non-profit ARMMAN sends automated voice messages to expecting and new mothers across India. However, 38% of mothers stop listening to these calls, missing critical preventative care information. To improve engagement, ARMMAN employs health workers to intervene by making service calls, but workers can only call a fraction of the 100K enrolled mothers. Partnering with ARMMAN, we model the problem of allocating limited interventions across mothers as a restless multi-armed bandit (RMAB), where the realities of large scale and model uncertainty present key new technical challenges. We address these with GROUPS, a double oracle–based algorithm for robust planning in RMABs with scalable grouped arms. Robustness over grouped arms requires several methodological advances. First, to adversarially select stochastic group dynamics, we develop a new method to optimize Whittle indices over transition probability intervals. Second, to learn group-level RMAB policy best responses to these adversarial environments, we introduce a weighted index heuristic. Third, we prove a key theoretical result that planning over grouped arms achieves the same minimax regret--optimal strategy as planning over individual arms, under a technical condition. Finally, using real-world data from ARMMAN, we show that GROUPS produces robust policies that reduce minimax regret by up to 50%, halving the number of preventable missed voice messages to connect more mothers with life-saving maternal health information.

Enhancing federated learning robustness in adversarial environment through clustering Non-IID features

Federated Learning (FL) enables many clients to train a joint model without sharing the raw data. While many byzantine-robust FL methods have been proposed, FL remains vulnerable to security attacks such as poisoning attacks and evasion attacks due to its distributed adversarial environment. Additionally, real-world training data used in FL are usually Non-Independent and Identically Distributed (Non-IID), which further weakens the robustness of the existing FL methods (such as Krum, Median, Trimmed-Mean, etc.), thereby making it possible for a global model in FL to be broken in extreme Non-IID scenarios. In this work, we mitigate the aforementioned weaknesses of existing FL methods in Non-IID and adversarial scenarios by proposing a new FL framework called Mini-Federated Learning (Mini-FL). Mini-FL follows the general FL approach but considers the Non-IID sources of FL and aggregates the gradients by groups. Specifically, Mini-FL first performs unsupervised learning for the gradients received to define the grouping policy. Then, the server divides the gradients received into different groups according to the grouping policy defined and performs byzantine-robust aggregation. Finally, the server calculates the weighted mean of gradients from each group to update the global model. Owning the strong generality, Mini-FL can utilize the most existing byzantine-robust method. We demonstrate that Mini-FL effectively enhances FL robustness and achieves greater global accuracy than existing FL methods when against security attacks and in Non-IID settings.

Robust Multiple-Path Orienteering Problem: Securing Against Adversarial Attacks

The multiple-path orienteering problem asks for paths for a team of robots that maximize the total reward collected while satisfying budget constraints on the path length. This problem models many multirobot routing tasks, such as exploring unknown environments and information gathering for environmental monitoring. In this article, we focus on how to make the robot team robust to failures when operating in adversarial environments. We introduce the robust multiple-path orienteering problem (RMOP), where we seek worst case guarantees against an adversary that is capable of attacking at most <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\alpha$</tex-math></inline-formula> robots. We consider two versions of this problem: RMOP offline and RMOP online. In the offline version, there is no communication or replanning when robots execute their plans, and our main contribution is a general approximation scheme with a bounded approximation guarantee that depends on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\alpha$</tex-math></inline-formula> and the approximation factor for single-robot orienteering. In particular, we show that the algorithm yields a: 1) constant-factor approximation when the cost function is modular; 2) <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\log$</tex-math></inline-formula> factor approximation when the cost function is submodular; and 3) constant-factor approximation when the cost function is submodular, but the robots are allowed to exceed their path budgets by a bounded amount. In the online version, the RMOP is modeled as a two-player sequential game and solved adaptively in a receding horizon fashion based on Monte Carlo tree search. In addition to theoretical analysis, we perform simulation studies for ocean monitoring and tunnel information-gathering applications to demonstrate the efficacy of our approach.

Reactive Symbolic Planning and Control in Dynamic Adversarial Environments

Satisfying both safety and reachability requirements in dynamic adversarial environments is very challenging, particularly when little or no information about the dynamics and intentions of the adversarial objects is available. Therefore, this paper addresses the problem of path planning and control of autonomous vehicles in a dynamic adversarial reach-avoid scenario with two non-cooperative vehicles and their competitive objectives: &#x201C;reaching a target and avoiding the other vehicle&#x201D; for one of them, called attacker, and &#x201C;protecting the target and capturing the opponent vehicle&#x201D; for the other one, called defender. In the proposed solution, first, a discrete version of the problem is formulated and solved using Linear Temporal Logic, temporal games, and <inline-formula><tex-math notation="LaTeX">$\mu$</tex-math></inline-formula>-calculus to construct winning discrete strategies that guarantee safety and reachability. A comprehensive discussion on the existence, correctness, and complexity analysis of the solution is also provided. Finally, a novel correct-by-design hybrid controller is designed to generate smooth control signals that preserve the satisfaction of safety and reachability.

Fate of rubber bush (Calotropis procera (Aiton) W. T. Aiton) in adversary environment modulated by microstructural and functional attributes

Fate of rubber bush (Calotropis procera (Aiton) W. T. Aiton) in adversary environment modulated by microstructural and functional attributes

Online Submodular Coordination With Bounded Tracking Regret: Theory, Algorithm, and Applications to Multi-Robot Coordination

We enable efficient and effective coordination in unpredictable environments, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , in environments whose future evolution is unknown a priori and even adversarial. We are motivated by the future of autonomy that involves multiple robots coordinating in dynamic, unstructured, and adversarial environments to complete complex tasks such as target tracking, environmental mapping, and area monitoring. Such tasks are often modeled as submodular maximization coordination problems. We introduce the first submodular coordination algorithm with bounded <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tracking regret</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , with bounded suboptimality with respect to optimal time-varying actions that know the future a priori. The bound gracefully degrades with the environments' capacity to change adversarially. It also quantifies how often the robots must re-select actions to “learn” to coordinate as if they knew the future a priori. The algorithm requires the robots to select actions sequentially based on the actions selected by the previous robots in the sequence. Particularly, the algorithm generalizes the seminal Sequential Greedy algorithm by Fisher et al. to unpredictable environments, leveraging submodularity and algorithms for the problem of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tracking the best expert</i> . We validate our algorithm in simulated scenarios of target tracking.

Three level intrusion detection system based on conditional generative adversarial network

&lt;span lang="EN-US"&gt;Security threat protection is important in the internet of things (IoT) applications since both the connected device and the captured data can be hacked or hijacked or both at the same time. To tackle the above-mentioned problem, we proposed three-level intrusion detection system conditional generative adversarial network (3LIDS-CGAN) model which includes four phases such as first-level intrusion detection system (IDS), second-level IDS, third-level IDS, and attack type classification. In first-level IDS, features of the incoming packets are extracted by the firewall. Based on the extracted features the packets are classified into three classes such as normal, malicious, and suspicious using support vector machine and golden eagle optimization. Suspicious packets are forwarded to the second-level IDS which classified the suspicious packets as normal or malicious. Here, signature-based intrusions are detected using attack history information, and anomaly-based intrusions are detected using event-based semantic mapping. In third-level IDS, adversary packets are detected using CGAN which automatically learns the adversarial environment and detects adversary packets accurately. Finally, proximal policy optimization is proposed to detect the attack type. Experiments are conducted using the NS-3.26 network simulator and performance is evaluated by various performance metrics which results that the proposed 3LIDS-CGAN model outperforming other existing works.&lt;/span&gt;

Robust Satisfaction of Metric Interval Temporal Logic Objectives in Adversarial Environments

This paper studies the synthesis of controllers for cyber-physical systems (CPSs) that are required to carry out complex time-sensitive tasks in the presence of an adversary. The time-sensitive task is specified as a formula in the metric interval temporal logic (MITL). CPSs that operate in adversarial environments have typically been abstracted as stochastic games (SGs); however, because traditional SG models do not incorporate a notion of time, they cannot be used in a setting where the objective is time-sensitive. To address this, we introduce durational stochastic games (DSGs). DSGs generalize SGs to incorporate a notion of time and model the adversary’s abilities to tamper with the control input (actuator attack) and manipulate the timing information that is perceived by the CPS (timing attack). We define notions of spatial, temporal, and spatio-temporal robustness to quantify the amounts by which system trajectories under the synthesized policy can be perturbed in space and time without affecting satisfaction of the MITL objective. In the case of an actuator attack, we design computational procedures to synthesize controllers that will satisfy the MITL task along with a guarantee of its robustness. In the presence of a timing attack, we relax the robustness constraint to develop a value iteration-based procedure to compute the CPS policy as a finite-state controller to maximize the probability of satisfying the MITL task. A numerical evaluation of our approach is presented on a signalized traffic network to illustrate our results.

GPS-free synchronized pseudo-random number generators for internet-of-things

IntroductionSecuring wireless communications in internet-of-things (IoT) requires both generation and synchronization of random numbers in real-time. However, resource constraints on an IoT device limit the use of computationally intensive random number generators and the use of global positioning systems (GPS) for synchronization. In this paper, we propose a synchronized pseudo-random number generator (SPRNG) that uses a combination of a fast, low-complexity linear-feedback-shift-register (LFSR) based PRNG and a slow but secure, synchronized seed generator based on self-powered timers.MethodsA prototype synchronized self-powered timer (SSPT) array was fabricated in a standard silicon process and was used to generate dynamic random seeds for the LFSR. The SSPTs use quantum-mechanical tunneling of electrons to operate without any external power and are practically secure against tampering, snooping, and side-channel attacks (both power and electromagnetic).ResultsIn this work, we explore protocols to periodically and securely generate random bits using the self-powered timers for seeding the LFSR. We also show that the time-varying random seeds extend and break the LFSR periodic cycles, thus making it difficult for an attacker to predict the random output or the random seed. Using the National Institute of Standards and Technology (NIST) test suite we verify the randomness of the measured seeds from the fabricated ensemble of SSPTs together with the random bit sequences generated by a software-seeded LFSR.DiscussionsIn this modality, the proposed SPRNG could be used as a trusted platform module (TPM) on IoTs and used for verifying and authenticating secure transactions (e.g., software upgrades). Since the SPRNG system does not require access to GPS for synchronization, therefore it could be used in many resource-constrained and adversarial environments.

EIFDAA: Evaluation of an IDS with function-discarding adversarial attacks in the IIoT

The complexity of the Industrial Internet of Things (IIoT) presents higher requirements for intrusion detection systems (IDSs). An adversarial attack is a threat to the security of machine learning-based IDSs. For such a complex situation, this paper analyses adversarial attackers’ ability to deceive IDSs used in the IIoT and proposes the evaluation of an IDS with function-discarding adversarial attacks in the IIoT (EIFDAA), a framework that can evaluate the defence performance of machine learning-based IDSs against various adversarial attack algorithms. This framework is composed of two main processes: adversarial evaluation and adversarial training. Adversarial evaluation can diagnose IDS that is unfitting in adversarial environments. Then, adversarial training is used to treat the weak IDS. In this framework, five well-known adversarial attacks, the fast-gradient sign method (FGSM), basic iterative method (BIM), projected gradient descent (PGD), DeepFool and Wasserstein generative adversarial network with gradient penalty (WGAN-GP) are used to convert attack samples into adversarial samples to simulate the adversarial environment. This study evaluates the capability of mainstream machine learning techniques as intrusion detection models to defend against adversarial attacks, and retrains these detectors to improve the robustness of IDSs through adversarial training. In addition, the framework includes an adversarial attack model that discards the attack function of the attack samples in the IIoT. Through the experimental results on the X-IIoTID dataset, the dropped adversarial detection rate of these detectors to nearly zero demonstrates that an adversarial attack has black-box attack capabilities for these IDSs. Additionally, the improved IDSs retrained with adversarial samples can effectively defend against adversarial attackers while maintaining the original detection rate for the attack samples. EIFDAA is expected to be a solution that can be applied to IDS for improving the robustness in the IIoT.

Fault detection method based on adversarial reinforcement learning

Fault detection is an essential task for large-scale industrial maintenance. However, in practical applications, due to the possible harm caused by the collection of fault data, the fault samples that lead to the labeling are usually very few. Most existing methods consider training unsupervised models with a large amount of unlabeled data while ignoring the rich knowledge that existed in a small amount of labeled data. To make full use of this prior knowledge, this article proposes a reinforcement learning model, namely, adversarial reinforcement learning in weakly supervised (WS-ARL), which performs significantly better by jointly learning small labeled anomaly data and large unlabeled data. We use an agent of the reinforcement learning model as a fault detector and add a new environment agent as a sample selector, by providing an opposite reward for two agents, and they learn in an adversarial environment. The feasibility and effectiveness of the model are verified by experimental analysis and compared the performance of the model with five state-of-the-art weakly/un-supervised methods in the hydraulic press fault detection task.