Adversarial Environment Research Articles

HGEE: Learning for Trajectory Prediction with Heterogeneous Graph Interaction and External Embedding of Unmanned Swarm Systems in Adversarial Environment

Trajectory prediction of unmanned swarm systems, serving as the foundation for behavioral and intentional cognition, has attracted extensive attention and made considerable progress in adversarial research. The influence of heterogeneous interaction relationships and external factors is crucial for trajectory prediction. Consequently, this paper proposes the Heterogeneous Graph with External Embedding (HGEE) network. We model the latent variables as multi-layer heterogeneous graphs based on prior knowledge of different interaction relationships and propose a method for calculating edge embeddings for heterogeneous graphs. Furthermore, we introduce a method that combines external environmental feature with historical observational trajectory data as the input for the decoder, enabling the model to learn the impacts of obstacles, targets, and desired formations on trajectories. We demonstrate that our approach surpasses state-of-the-art models in interaction inference and trajectory prediction through experiments on our proposed formation datasets based on consensus theory, across five evaluation metrics.

A generic approach for network defense strategies generation based on evolutionary game theory

The generation of optimal defense strategies in dynamic adversarial environments is crucial for cybersecurity. Recently, defense approaches based on evolutionary game theory have gained significant achievements. However, they would fail when facing complex networks and sophisticated attack strategies, due to the fatal drawbacks of defense strategy generation considering atomic attacks only. To relieve this issue, a generic approach for generating defense strategies using evolutionary game theory is proposed in this paper. Initially, a novel payoff quantification method for network attack-defense games based on attack graphs is designed. Innovatively, two factors concerning the decision-maker's degree of irrationality (DI) and the level of environmental security (LES) are introduced into the replicator dynamics equation to model the impacts on equilibrium solutions. Noting that Active Directory (AD) domain service is one of the most used and representative information security management system in Windows domains, from which attack graphs and paths can be plainly extracted and analyzed. Therefore, it is necessary and imperative to anchor AD to unfold the theoretical analyses and experiments validation based on a real environment. Case studies on a real-world AD network demonstrate that the proposed approach is effective and can generate stable and efficient defense strategies.

An adversarial environment reinforcement learning-driven intrusion detection algorithm for Internet of Things

The increasing prevalence of Internet of Things (IoT) systems has made them attractive targets for malicious actors. To address the evolving threats and the growing complexity of detection, there is a critical need to search for and develop new algorithms that are fast and robust in detecting and classifying dangerous network traffic. In this context, deep reinforcement learning (DRL) is gaining recognition as a prospective solution in numerous fields as it enables autonomous agents to cooperate with their environment for decision-making without relying on human experts. This article presents an innovative approach to intrusion detection in IoT systems using an adversarial reinforcement learning (RL) algorithm known for its exceptional predictive capabilities. The predictive process relies on a classifier, implemented as a streamlined and highly efficient neural network. Embedded within this classifier is a policy function meticulously trained using an innovative RL model. Importantly, this model ensures that the environment’s behavior is dynamically fine-tuned simultaneously with the learning process, improving the overall effectiveness of the intrusion detection approach. The efficiency of our proposal was assessed using the Bot-IoT database, consisting of a mixture of legitimate IoT network traffic and simulated attack scenarios. Our scheme shows superior performance compared to existing ones. Therefore, our approach to IoT intrusion detection can be considered a valuable alternative to existing methods, capable of significantly improving the IoT systems’ security.

Resilient distributed optimization for cyber–physical systems under adversarial environments: An event-based method

This work presents a resilient distributed optimization algorithm based on the event-triggering mechanism for cyber–physical systems (CPSs) to optimize an average of convex cost functions corresponding to multiple agents under adversarial environments. Two attack scenarios, including the f-total (each agent is affected by at most f malicious agents in the whole network) and the f-local (each agent is affected by at most f malicious agents in its in-neighbor set) attacks are considered. Subsequently, the convergence conditions under these two attack scenarios are provided, respectively, both of which guarantee that the state values of benign agents converge to a bounded error range. The optimality conditions are also presented by theoretical analysis, which guarantee that the state values of benign agents converge to a safety interval constructed by local optimal values under certain graph conditions, despite the misbehavior of malicious agents. In addition, four numerical examples are presented to show the effectiveness and superiority of the event-triggering resilient distributed optimization (RDO-E) algorithm. Compared to existing resilient algorithms, the proposed method achieves resilient distributed optimization with higher accuracy and less demanding communication overheads. Finally, by applying the proposed method to the multi-microgrid system, a resilient economic dispatch problem (REDP) is successfully solved, which validates the practical viability of the RDO-E algorithm.

Topology-Preserving Motion Coordination for Multi-Robot Systems in Adversarial Environments

Topology-Preserving Motion Coordination for Multi-Robot Systems in Adversarial Environments

Extracting the explore‐exploit intelligence of Physarum to manage the sustainability of an enterprise network

AbstractIn this work, we enhance the sustainability of an enterprise network (EN) by complementing it with an expert system that apprehends the explore‐exploit behavioural intelligence of Physarum to survive against the attractive‐adversarial nutritional environment. EN sustainability is dynamic since it depends on how well EN can react to an adversarial environment. We capture a reverse analogy to characterize EN's workload‐environment with Physarum's nutritive‐environment, where the high volume of workloads at the backbone network corresponds to a poor‐nutrient environment. The expert system explores EN to find out how to manage the workloads as Physarum handles its survivability, and exploits the users' workload patterns by grouping the highly communicating users together to redesign the network structure as Physarum's intelligence to exploit energy from rich‐ and poor‐nutrient food sources through redesigned tubular structures. We define two factors, such as nutrient‐intensity and chemo‐attractant to aid the redesign process. EN evolves through a set of redesigned clusters with an objective function to maximize its sustainability for a given set of explored workloads by minimizing the workloads through the backbone. EN evolution terminates when there is no change in the backbone utilization, resembling the organism's stay in a dormant state until it experiences a favourable environment. Our experimental results on an EN with a higher volume of workloads at the backbone producing 14.26 kWh energy consumption demonstrated that the developed expert system reduced the energy consumption to 11.27 kWh, thus enhanced the sustainability from 21% to 61%.

Optimization of high-speed fixed-wing UAV penetration strategy based on deep reinforcement learning

The penetration decision of unmanned aerial vehicles (UAVs) is one of the key components for UAVs to detect or strike crucial targets in adversarial environments. Presently, most methods lack research on penetration decisions under unknown interception information. The executing UAV for decision-making is generally a simplified three-degree-of-freedom model. The decision format is often simplified to path point optimization, lacking control over the stability of complex model attitudes and exhibiting relatively poor robustness. This paper proposes a penetration relative motion theory and aircraft control method based on the six-degrees-of-freedom UAV model to reduce response errors between algorithm control decisions and actual flight control. Using a Markov decision process, a UAV penetration decision control method based on an attitude-overload control loop is designed. This method combines reinforcement learning techniques to enable autonomous penetration decision-making and control for UAVs facing active defense scenarios, enhancing the autonomy, accuracy, and generalization of UAV online decision-making. The paper concludes by implementing autonomous decision-making and control for UAV penetration through the integration of interceptor trajectory prediction with proximal policy optimization. This method reduces the dependency of penetration strategies on interceptor information, ensuring both the generalization ability of penetration strategy models under uncertain information and the consistency of decision stability. Through simulations and experiments, we verified that the penetration strategy based on the pre-sampling PPO method improves the penetration effect by 15.44 % on average and the hit rate by 18.48 %. Moreover, the probability of penetration in different scenarios exceeds 70 %, and the penetration effect is improved by more than 10 % in the case of multiple interceptors. This paper also analyzes the impact of interceptor spatial distribution and the quantity of interceptors on the effectiveness of UAV penetration decision-making.

A semi-supervised learning approach for semantic parsing boosted by BERT word embedding

In the modern world, structured and semi-structured knowledge bases hold a considerable amount of data. There-fore, people who are familiar with formal query languages should not be the only ones who can efficiently and clearly query them. Semantic Parsing (SP) is converting natural language utterances into formal meaning representations. The paper suggests a model for SP that uses a novel method of utilizing the Semi-Supervised Generative Adversarial Network (SS-GAN) to enhance the classifier performance. The proposed SS-GAN extends the fine-tuning of word embedding architectures using unlabeled examples in a generative adversarial environment. We provide a regularization strategy for addressing the mode missing problem and unstable training in SS-GAN. The main viewpoint is to use the extracted feature vectors from the discriminator. Hence, the generator produces outputs by aiding the discriminator’s learned features. A reconstruction loss is added to the loss function of the SS-GAN to drive the genera-tor to reconstruct outputs from the discriminator’s features, hence steering the generator toward actual data configurations. The proposed reconstruction loss improves the performance of SS-GAN, produces high-quality outputs, and may be combined with other regularization loss functions to improve the performance of diverse GANs. We employ BERT word embedding for our model, which can be included in a downstream task and fine-tuned as a model, while the pre-trained BERT model can capture various linguistic properties. We examine the suggested model using the WikiSQL and SparC datasets, and the analysis findings reveal our model outperforms its rivals. The findings from our experiments indicate that the need for labeled samples can be minimized, down to as few as 100 instances, while still achieving commendable classification outcomes.

Resilient Time-Varying Formation Tracking for Heterogeneous High-Order Multiagent Systems With Multiple Dynamic Leaders

This paper investigates time-varying output formation tracking problems for heterogeneous multiagent systems in an adversarial environment, where both leaders and followers are vulnerable to cyber-attacks. Despite the presence of adversarial agents, the normal followers not suffering from cyber-attacks seek to achieve a desired for-mation configuration and track a safe trajectory generated by normal leaders' outputs simultaneously. To this end, a resilient time-varying output formation (RTVOF) tracking scheme is presented in this work. Firstly, to counteract the effect of adversarial agents, a safety strategy is proposed for general linear heterogeneous systems, which is a syn-thesis of the mean-subsequence-reduced family algorithms and an attack detection procedure. Secondly, using the neighboring interaction, a resilient distributed observer and a local formation controller are designed for each normal follower to estimate the state and track the output of the safe formation reference. Then, it is proved that the hetero-geneous multiagent system with multiple dynamic leaders and time-varying topologies can accomplish the RTVOF tracking if specific graph-theoretic properties and forma-tion feasibility conditions are satisfied. Finally, simulation examples are given to demonstrate the effectiveness of obtained results.

RRIoT: Recurrent reinforcement learning for cyber threat detection on IoT devices

To address the recent worldwide proliferation of cybersecurity attacks across computing systems, especially internet-of-things devices, new robust and automated methods are needed to detect and mitigate the attacks in real time, ensuring the confidentiality, integrity, and availability of systems. Machine Learning (ML) techniques have shown promise in detecting some types of attacks. However, they are not universally successful in detecting sequential attacks. In this study, we propose RRIoT, to mitigate this issue. RRIoT applies a Deep Deterministic Policy Gradient reinforcement learning (RL) algorithm in conjunction with an LSTM layer within an adversarial environment to detect and identify attacks. We evaluate our method against novel and state-of-the-art ML/RL algorithms which build upon previous RL algorithms such as deep Q-networks (DQN), dueling deep Q-networks (DDQN), and deep deterministic policy gradient (DDPG). Our results indicate that our proposed RRIoT generally performs better than existing ML algorithms and performs as well as or better than novel RL algorithms with new network architectures. We leverage Shapley Additive Global Importance (SAGE) to provide additional insight into which features contribute most to a model's performance and verify feature importance through the implementation of an ablation study across three IoT telemetry datasets.

Editorial: Cooperative control and team behaviors in adversarial environments

Editorial: Cooperative control and team behaviors in adversarial environments

Identity concealment games: How I learned to stop revealing and love the coincidences

In an adversarial environment, a hostile player performing a task may behave like a non-hostile one in order not to reveal its identity to an opponent. To model such a scenario, we define identity concealment games: zero-sum stochastic reachability games with a zero-sum objective of identity concealment. To measure the identity concealment of the player, we introduce the notion of an average player. The average player’s policy represents the expected behavior of a non-hostile player. We show that there exists an equilibrium policy pair for every identity concealment game and give the optimality equations to synthesize an equilibrium policy pair. If the player’s opponent follows a non-equilibrium policy, the player can hide its identity better. For this reason, we study how the hostile player may learn the opponent’s policy. Since learning via exploration policies would quickly reveal the hostile player’s identity to the opponent, we consider the problem of learning a near-optimal policy for the hostile player using the game runs collected under the average player’s policy. Consequently, we propose an algorithm that provably learns a near-optimal policy and give an upper bound on the number of sample runs to be collected.

Image-Level Adaptive Adversarial Ranking for Person Re-Identification.

The potential vulnerability of deep neural networks and the complexity of pedestrian images, greatly limits the application of person re-identification techniques in the field of smart security. Current attack methods often focus on generating carefully crafted adversarial samples or only disrupting the metric distances between targets and similar pedestrians. However, both aspects are crucial for evaluating the security of methods adapted for person re-identification tasks. For this reason, we propose an image-level adaptive adversarial ranking method that comprehensively considers two aspects to adapt to changes in pedestrians in the real world and effectively evaluate the robustness of models in adversarial environments. To generate more refined adversarial samples, our image representation enhancement module leverages channel-wise information entropy, assigning varying weights to different channels to produce images with richer information content, along with a generative adversarial network to create adversarial samples. Subsequently, for adaptive perturbation of ranking, the adaptive weight confusion ranking loss is presented to calculate the weights of distances between positive or negative samples and query samples. It endeavors to push positive samples away from query samples and bring negative samples closer, thereby interfering with the ranking of system. Notably, this method requires no additional hyperparameter tuning or extra data training, making it an adaptive attack strategy. Experimental results on large-scale datasets such as Market1501, CUHK03, and DukeMTMC demonstrate the effectiveness of our method in attacking ReID systems.

The impact of mental health and psychosocial support programmes on children and young people's mental health in the context of humanitarian emergencies in low- and middle-income countries: A systematic review and meta-analysis.

Humanitarian emergencies pose a significant global health challenge for children and young people's mental and psychological health. This systematic review investigates the effectiveness of mental health and psychosocial support (MHPSS) programmes delivered to children and young people affected by humanitarian emergencies in low- and middle-income countries (LMICs). Twelve electronic databases, key websites and citation checking were undertaken. Forty-three randomised controlled trials (RCTs) published in English between January 1980 and May 2023 were included in the review. Overall, the findings suggest that cognitive behavioural therapy may improve depression symptoms in children and young people affected by humanitarian emergencies. Narrative exposure therapy may reduce feelings of guilt. However, the impact of the other MHPSS modalities across outcomes is inconsistent. In some contexts, providing psychosocial programmes involving creative activities may increase the symptoms of depression in children and young people. These findings emphasise the need for the development of MHPSS programmes that can safely and effectively address the diverse needs of children and young people living in adversarial environments.

Byzantine-Robust Distributed Online Learning: Taming Adversarial Participants in An Adversarial Environment

Byzantine-Robust Distributed Online Learning: Taming Adversarial Participants in An Adversarial Environment

Quantum Bandit With Amplitude Amplification Exploration in an Adversarial Environment

Quantum Bandit With Amplitude Amplification Exploration in an Adversarial Environment

Differential Game-Based Deep Reinforcement Learning in Underwater Target Hunting Task.

To meet requirements for real-time trajectory scheduling and distributed coordination, underwater target hunting task is challenging in terms of turbulent ocean environments and dynamic adversarial environment. Despite the existing research in game-based target hunting area, few approaches have considered dynamic environmental factors, such as sea currents, winds, and communication delay. In this article, we focus on a target hunting system consisted of multiple unmanned underwater vehicles (UUVs) and a target with high maneuverability. Besides, differential game theory is leveraged to analyze adversarial behaviors between hunters and the escapee. However, it is intractable that UUVs have to deploy an adaptive scheme to guarantee the consistency and avoid the escape of the target without collision. Therefore, we conceive the Hamiltonian function with Leibniz's formula to obtain feedback control policies. In addition, it proves that the target hunting system is asymptotically stable in the mean, and the system can satisfy Nash equilibrium relying on the proposed control policies. Furthermore, we design a modified multiagent reinforcement learning (MARL) to facilitate the underwater target hunting task under the constraints of energetic flows and acoustic propagation delay. Simulation results show that the proposed scheme is superior to the typical MARL algorithm in terms of reward and success rate.

A dynamic resilience evaluation method for cross-domain swarms in confrontation

As a typical Internet of Things system, unmanned swarms can execute missions efficiently, while reduce risks and ensure personnel safety during operations. Because swarms in different physical domains have different advantages, the cross-domain swarms received lots of attentions recently. Since swarm has the ability to resist damage and recover from damage through self-organization and self-reconfiguration, resilience has become an important indicator for swarm design and mission planning in adversarial environments. Existing resilience evaluation methods require a complete degradation-recovery-stabilization process of system performance, which is not proper for the cross-domain swarm under dynamic and uncertain environment. This paper proposes a dynamic resilience evaluation method for cross-domain swarms in confrontation scenario. First, a cross-domain swarm confrontation model is developed and the swarm confrontation performance indicator is proposed. Second, a dynamic resilience evaluation method is proposed for cross-domain swarm. Then, a confrontation strategy selection model is given based on the resilience measurement. A case study of air-ground swarm in confrontations is provided for illustration and analysis. As can be seen from the simulation results, the proposed dynamic resilience method can better capture the dynamic and uncertain nature of cross-domain swarms in confrontation. The proposed strategy selection model can improve the overall resilience of the swarm, and ultimately improve the winning percentage of the cross-domain swarm in confrontation.

Optimal resilient sensor placement problem for secure state estimation

In this paper, we consider a sensor placement problem for secure state estimation in an adversarial environment. Specifically, this paper deals with a system consisting of n agents where up to l sensor measurements are potentially compromised by malicious adversaries and tackles the problem of sensor placement such that the system state can be reconstructed even if the measurements are compromised. However, since it is costly to deploy sensors for all agents, we also aim to place sensors at the lowest possible cost. We call this problem the optimal resilient sensor placement problem (ORSPP) and show that this problem is coNP-hard. Therefore, it is, in general, impossible to compute the ORSPP in polynomial time unless P=coNP. On the positive side, however, we also provide that this problem can be solved in polynomial time if the network structure of the agents satisfies certain conditions. The concrete algorithm for the ORSPP is provided, and further, in the process of providing the coNP-hardness, we also derive a new form of a necessary and sufficient condition for secure state estimation. Finally, we show the results of numerical simulations using a random graph and a diffusion process.

Securing blockchain-based timed data release against adversarial attacks1

Timed data release refers to protecting sensitive data that can be accessed only after a pre-determined amount of time has passed. While blockchain-based solutions for timed data release provide a promising approach for decentralizing the process, designing an attack-resilient timed-release service that is resilient to malicious adversaries in a blockchain network is inherently challenging. A timed-release service on a blockchain network is inevitably exposed to the risk of post-facto attacks where adversaries may launch attacks after the data is released in the blockchain network. Existing incentive-based solutions for timed data release in Ethereum blockchains guarantee protection under the assumption of a fully rational adversarial environment in which every peer acts rationally. However, these schemes fail invariably when even a single participating peer node in the protocol starts acting maliciously and deviates from the rational behavior. In this paper, we propose a systematic solution for attack-resilient and practical blockchain-based timed data release in a mixed adversarial environment, where both malicious adversaries and rational adversaries exist. We first propose an effective uncertainty-aware reputation measure to capture the behaviors of the peer involved in timed data release activities in the network. In light of such a measure, we present the design of a basic protocol that consists of two critical ingredients, namely reputation-aware peer recruitment and verifiable enforcement protocols. The former, prior to the start of the enforcement protocols, performs peer recruitment based on the reputation measure to make the design probabilistically attack-resilient to the post-facto attacks. The latter is responsible for contractually guarding the recruited peers at runtime by transparently reporting observed adversarial behaviors. However, the basic recruitment design is only aware of the reputation of the peers and it does not consider the working time schedule of the participating peers and as a result, it results in lower attack-resilience. To enhance the attack resilience further without impacting the verifiable enforcement protocols, we propose a temporal graph-based reputation-aware peer recruitment algorithm that carefully determines the peer recruitment plan to make the service more attack-resilient. In our proposed approach, we formally capture the timed data release service as a temporal graph and we develop a novel maximal attack-resilient path-finding algorithm on the temporal graph for the participating peers. We implement a prototype of the proposed approach using Smart Contracts and deploy it on the Ethereum official test network, Rinkeby. For extensively evaluating the proposed techniques, we perform simulation experiments to validate the effectiveness of the reputation-aware timed data release protocols as well as our proposed temporal-graph-based improvements. The results demonstrate the effectiveness and strong attack resilience of the proposed mechanisms and our approach incurs only a modest gas cost.