Multi-agent Coordination Research Articles

Determining What’s Next: Visual Analytics for Evaluating Human-Automation Coordination

As advanced automated systems become more complex, there is a greater need to analyze the difficulty of human-automation coordination. We developed a novel visual analytics tool to evaluate human-automation coordination by examining the relationship between projection, explanation (retrojection), and the understanding of current events through time. The What’s Next Diagram visualizes demands on human sensemaking as operators answer Wiener’s key automation surprise question: what will automation do next. The tool can assess whether operators can answer the question directly and quickly given the design and architecture of human-automation systems. It can be used as a model of multi-agent coordination, as a tool for post-mortem analysis, or an aid for design. We illustrate how the tool can be used to identify system requirements for anomaly response in the aviation domain.

Target-Oriented Multi-Agent Coordination with Hierarchical Reinforcement Learning

Target-Oriented Multi-Agent Coordination with Hierarchical Reinforcement Learning

New Distributed Consensus Schemes With Time Delays and Output Saturation

Estimates of agents in a distributed consensus control are aligned with a particular value through interacting on communication graphs, which are of particular interest for researchers in the field of multi-agent coordination. Following this pattern, a discrete-time constrained consensus issue with generalized time delays is first established in this article, which is manipulated into an optimization problem via a quadratic performance index introduced as a global objective function. Then, a novel consensus scheme is investigated and proposed for handling this problem, enabling the consensus to approach the desired state globally and rapidly with optimal system property ensured. Besides, to advance the convergence speed and stability in resisting constant bias or oscillation, an adjustment control method is explored to construct a modified consensus scheme; further, to enhance the scene adaptability, fix topologies are extended to switching ones, and the latter is involved to develop another scheme on this basis. Moreover, the convergence and robustness of these three proposed consensus schemes are substantiated by theoretical analysis and numerical simulations. To highlight the practical implementations, the proposed consensus schemes are incorporated with a winner-take-all operation to accomplish multi-agent competitive coordination in a distributed way, and the results embody their effectiveness and superior consensus control ability, along with strong plasticity. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is dedicated to investigating and optimizing distributed consensus schemes with time delays and output saturation with application to the competitive coordination of multi-agent systems. On the one hand, the consensus problem with output saturation receives limited attention, and few consensus algorithms consider both saturation limitation and channel gain. On the other hand, most existing studies on consensus with saturation consider simply the dynamic behaviour of agents without employing optimization, thus limiting further improvement of system performance. In this paper, a consensus algorithm is built from an optimization perspective that guarantees the state consensus of a multi-agent system with output constraints and generalized time delays. In addition, a consensus scheme is designed in a discrete-time framework and further modified and perfected along with the idea of considering time delays and expanding diverse topologies. Finally, experiments are conducted by applying the consensus scheme to a winner-take-all operation, with contributions of this paper verified.

Depthwise Convolution for Multi-Agent Communication With Enhanced Mean-Field Approximation.

Multi-Agent settings remain a fundamental challenge in the reinforcement learning (RL) domain due to the partial observability and the lack of accurate real-time interactions across agents. In this article, we propose a new method based on local communication learning to tackle the multi-agent RL (MARL) challenge within a large number of agents coexisting. First, we design a new communication protocol that exploits the ability of depthwise convolution to efficiently extract local relations and learn local communication between neighboring agents. To facilitate multi-agent coordination, we explicitly learn the effect of joint actions by taking the policies of neighboring agents as inputs. Second, we introduce the mean-field approximation into our method to reduce the scale of agent interactions. To more effectively coordinate behaviors of neighboring agents, we enhance the mean-field approximation by a supervised policy rectification network (PRN) for rectifying real-time agent interactions and by a learnable compensation term for correcting the approximation bias. The proposed method enables efficient coordination as well as outperforms several baseline approaches on the adaptive traffic signal control (ATSC) task and the StarCraft II multi-agent challenge (SMAC).

Entropy Enhanced Multiagent Coordination Based on Hierarchical Graph Learning for Continuous Action Space

Entropy Enhanced Multiagent Coordination Based on Hierarchical Graph Learning for Continuous Action Space

Robust Multi-Agent Communication With Graph Information Bottleneck Optimization.

Recent research on multi-agent reinforcement learning (MARL) has shown that action coordination of multi-agents can be significantly enhanced by introducing communication learning mechanisms. Meanwhile, graph neural network (GNN) provides a promising paradigm for communication learning of MARL. Under this paradigm, agents and communication channels can be regarded as nodes and edges in the graph, and agents can aggregate information from neighboring agents through GNN. However, this GNN-based communication paradigm is susceptible to adversarial attacks and noise perturbations, and how to achieve robust communication learning under perturbations has been largely neglected. To this end, this paper explores this problem and introduces a robust communication learning mechanism with graph information bottleneck optimization, which can optimally realize the robustness and effectiveness of communication learning. We introduce two information-theoretic regularizers to learn the minimal sufficient message representation for multi-agent communication. The regularizers aim at maximizing the mutual information (MI) between the message representation and action selection while minimizing the MI between the agent feature and message representation. Besides, we present a MARL framework that can integrate the proposed communication mechanism with existing value decomposition methods. Experimental results demonstrate that the proposed method is more robust and efficient than state-of-the-art GNN-based MARL methods.

An inertial control method for large-scale wind farm based on hierarchical distributed hybrid deep-reinforcement learning

Wind farm (WF) is gradually required to provide inertial support during frequency events nowadays. Existing WF inertial control methods have exhibited limitations in parameter estimation and optimal control parameter tuning under variable operating conditions. Moreover, traditional model-based methods suffer from computational efficiency problems due to their reliance on repeated iterations or derivative computations in optimization process. Thus, this paper proposes an inertial control method for large-scale WF based on hierarchical distributed hybrid deep-reinforcement learning (HDH-DRL). Firstly, the control objectives for the inertial control are defined, and are decomposed on the basis of wind turbines (WTs) division. Next, this paper presents a HDH-DRL based inertial control framework consisting of two levels. The upper-level control is achieved by the hybrid multi-agent DRL (HMA-DRL) algorithm with hybrid action exploration mechanism and multi-agent coordination. The consensus based lower-level control aims to achieve the consensus convergence of the control through the locally distributed interaction among WTs. Finally, the inertial control processes of the model are exhibited; the influences of DRL algorithms on the control are discussed; the computational speed and solution accuracy of the proposed method are compared with the model-based methods; the multi-scenario applicability and performance in local communication failures of the model are analyzed. The results demonstrate that the proposed method significantly improves the active power response and frequency stability, achieving rapid consensus convergence with a power deviation below 0.3%, surpassing traditional control methods by at least 0.2%; also, it demonstrates superior performance across various scenarios, including transient and steady-state conditions, with frequency enhancements up to 0.30% and exceptional stability under different wind conditions and communication disruptions.

Multiagent Coordination with Relative Measurement via Cooperative and Individual Control

Multiagent Coordination with Relative Measurement via Cooperative and Individual Control

A Framework-Based Multi-Agent Coordination for Enhanced Microgrid Energy Management at the Secondary Control Layer

A Framework-Based Multi-Agent Coordination for Enhanced Microgrid Energy Management at the Secondary Control Layer

Graph-Time Convolutional Neural Networks: Architecture and Theoretical Analysis.

Devising and analysing learning models for spatiotemporal network data is of importance for tasks including forecasting, anomaly detection, and multi-agent coordination, among others. Graph Convolutional Neural Networks (GCNNs) are an established approach to learn from time-invariant network data. The graph convolution operation offers a principled approach to aggregate multi-resolution information in each layer and offers some degree of mathematical analysis by exploring tools from graph signal processing. This analysis provides insights on the equivariance properties of GCNNs; spectral behaviour of the learned filters; and the stability to perturbations in the graph topology, which arises because of support perturbations or uncertainties. However, extending the convolution-principled learning and respective analysis to the spatiotemporal domain is challenging because spatiotemporal data have more intrinsic dependencies. Hence, a higher flexibility to capture jointly the spatial and the temporal dependencies is required to learn meaningful higher-order representations. Here, we leverage product graphs to represent the spatiotemporal dependencies in the data and introduce Graph-Time Convolutional Neural Networks (GTCNNs) as a principled architecture to aid learning. The proposed approach can work with any type of product graph and we also introduce a parametric product graph to learn also the spatiotemporal coupling. The convolution principle further allows a similar mathematical tractability as for GCNNs. In particular, the stability result shows GTCNNs are stable to spatial perturbations but there is an implicit trade-off between discriminability and robustness; i.e., the more complex the model, the less stable. Extensive numerical results on benchmark datasets corroborate our findings and show the GTCNN compares favourably with state-of-the-art solutions. We anticipate the GTCNN to be a starting point for more sophisticated models that achieve good performance but are also fundamentally grounded.

The Home Office’s Racism Studies Before the Macpherson Inquiry: Revisiting Administrative Criminology’s Corpus

Abstract This article examines the history of Home Office research on racism between 1976 and 1997. By analysing archival materials, the article focusses on the methodological orientations and bureaucratic negotiations associated with this corpus of administrative criminological studies. The article describes how Home Office criminologists adopted a stance that privileged quantification, restudies and aligning its research with pre-existing policy paradigms of Situational Crime Prevention and multi-agency coordination. In response to campaigners’ calls for racial justice, Home Office policy officials used this research to demonstrate governmental concern, manage media coverage and coordinate community stakeholders. The article argues that British criminology should move beyond dismissing administrative criminology as ‘theoretical empiricism’ to enable a more critical appraisal of the entangled histories of racism, criminology and crime policy.

Live Traffic Video Multicasting Services in UAV-Assisted Intelligent Transport Systems: A Multiactor Attention Critic Approach

Live traffic video is vitally important for vehicles in future intelligent transport systems (ITS). Due to the limitation of onboard sensors, vehicles may not be able to obtain a full view of the traffic situations which endangers safety for autonomous driving vehicles. In this paper, we propose a traffic video multicasting scheme by using video splitting and group splitting techniques for unmanned aerial vehicles (UAVs) assisted ITS, in which UAVs are considered as the eyes in the sky to capture real-time traffic videos. We aim to maximize the long-term video quality received by vehicles by jointly optimizing vehicle grouping and spectrum allocation. Considering the interactions among UAVs, the above optimization problem is formulated as a multi-agent coordination problem in the form of a Markov game (MG). The MG is subsequently solved by leveraging a state-of-the-art multi-agent deep reinforcement learning (MADRL) algorithm, namely, multi-actor attention critic (MAAC), in which an attention mechanism is utilized to pay attention to other agents to make the learning process more effective and scalable. Extensive simulation results show that the MAAC-based algorithm has better performance in terms of video quality and spectrum efficiency compared with the baseline methods.

A Hybrid Optimization Algorithm for Efficient Virtual Machine Migration and Task Scheduling Using a Cloud-Based Adaptive Multi-Agent Deep Deterministic Policy Gradient Technique

This To achieve optimal system performance in the quickly developing field of cloud computing, efficient resource management—which includes accurate job scheduling and optimized Virtual Machine (VM) migration—is essential. The Adaptive Multi-Agent System with Deep Deterministic Policy Gradient (AMS-DDPG) Algorithm is used in this study to propose a cutting-edge hybrid optimization algorithm for effective virtual machine migration and task scheduling. An sophisticated combination of the War Strategy Optimization (WSO) and Rat Swarm Optimizer (RSO) algorithms, the Iterative Concept of War and Rat Swarm (ICWRS) algorithm is the foundation of this technique. Notably, ICWRS optimizes the system with an amazing 93% accuracy, especially for load balancing, job scheduling, and virtual machine migration. The VM migration and task scheduling flexibility and efficiency are greatly improved by the AMS-DDPG technology, which uses a powerful combination of deterministic policy gradient and deep reinforcement learning. By assuring the best possible resource allocation, the Adaptive Multi-Agent System method enhances decision-making even more. Performance in cloud-based virtualized systems is significantly enhanced by our hybrid method, which combines deep learning and multi-agent coordination. Extensive tests that include a detailed comparison with conventional techniques verify the effectiveness of the suggested strategy. As a consequence, our hybrid optimization approach is successful. The findings show significant improvements in system efficiency, shorter job completion times, and optimum resource utilization. Cloud-based systems have unrealized potential for synergistic optimization, as shown by the integration of ICWRS inside the AMS-DDPG framework. Enabling a high-performing and sustainable cloud computing infrastructure that can adapt to the changing needs of modern computing paradigms is made possible by this strategic resource allocation, which is attained via careful computational utilization.

A multi-agency coordination resource allocation and routing decision-making problem: A coordinated truck-and-drone DSS for improved wildfire detection coverage

This study proposes a novel coordinated truck-drone system as a decision support system to improve fire suppression operations. The problem in this study is formulated as a bi-objective mathematical model to minimize total monitoring cost and time, considering trucks as mobile drone depots and that drones can fly at various altitudes and access hard-to-reach areas. In addition, time and cost parameters in the mathematical model are deemed uncertain, rendering the model more realistic. Accelerated Benders' decomposition (ABD) is utilized to solve the model rapidly. Furthermore, a column-and-constraint generation (CCG) algorithm is employed, which is an effective method for solving scenario-based models under uncertainty. Two criteria are subsequently used to evaluate and compare proposed robust optimization approaches (risk averse and min-max relative regret). The results indicate that the proposed system can assist firefighters in determining the optimal number of drones and trucks to patrol and the time and cost required to visit all areas. Moreover, the findings demonstrate that the min-max relative regret approach can outperform other methods during the hot season when the risk of wildfire is elevated. In contrast, when the fire risk is lower during the cold season, time and cost can be effectively managed, and a risk-averse strategy can be implemented. Finally, the proposed solution framework can facilitate optimal strategic organizational decision planning.

Fully decentralized multi-agent coordination scheme in smart distribution restoration: Multilevel consensus

Multi-agent systems (MAS) are emerging as a promising approach for smart power system restoration. After a fault has been detected and isolated, the maximum out-of-service loads should be restored in a timely manner. This study aims to develop reliable restoration through autonomous, rapid, and stable communication. For this purpose, a multilevel consensus is suggested to coordinate MAS in a fully decentralized fashion. System dynamics emerge in a distributed manner based on the state vectors of the agent and its proximate neighbors. The information matrix (IM) is used by the agents to accumulate and exchange state vectors encompassing common variable interests. In order to weight the influence of agents on voting, an optimized weighting model (Optw) is proposed. The parameters are bound by Lyapunov's assessment to ensure consensus and then optimized for quick convergence. The proposed multilevel topology supports parallel information discovery in local bunches and provides extremely quick convergence. The suggested one-by-one communication architecture facilitates the exchange of information since each agent only interacts with four of its neighbors. The redundant communication prevents a single point of failure. These achievements make it suitable for application in large-scale power systems. The effectiveness of the affirmation method is revealed through numerical simulation in various scenarios.

Robust Multi-Agent Coordination via Evolutionary Generation of Auxiliary Adversarial Attackers

Cooperative Multi-agent Reinforcement Learning (CMARL) has shown to be promising for many real-world applications. Previous works mainly focus on improving coordination ability via solving MARL-specific challenges (e.g., non-stationarity, credit assignment, scalability), but ignore the policy perturbation issue when testing in a different environment. This issue hasn't been considered in problem formulation or efficient algorithm design. To address this issue, we firstly model the problem as a Limited Policy Adversary Dec-POMDP (LPA-Dec-POMDP), where some coordinators from a team might accidentally and unpredictably encounter a limited number of malicious action attacks, but the regular coordinators still strive for the intended goal. Then, we propose Robust Multi-Agent Coordination via Evolutionary Generation of Auxiliary Adversarial Attackers (ROMANCE), which enables the trained policy to encounter diversified and strong auxiliary adversarial attacks during training, thus achieving high robustness under various policy perturbations. Concretely, to avoid the ego-system overfitting to a specific attacker, we maintain a set of attackers, which is optimized to guarantee the attackers high attacking quality and behavior diversity. The goal of quality is to minimize the ego-system coordination effect, and a novel diversity regularizer based on sparse action is applied to diversify the behaviors among attackers. The ego-system is then paired with a population of attackers selected from the maintained attacker set, and alternately trained against the constantly evolving attackers. Extensive experiments on multiple scenarios from SMAC indicate our ROMANCE provides comparable or better robustness and generalization ability than other baselines.

Robust Near-Optimal Coordination in Uncertain Multiagent Networks With Motion Constraints.

This article addresses the robust coordination problem for nonlinear uncertain second-order multiagent networks with motion constraints, including velocity saturation and collision avoidance. A single-critic neural network-based approximate dynamic programming approach and exact estimation of unknown dynamics are employed to learn online the optimal value function and controller. By incorporating avoidance penalties into tracking variable, constructing a novel value function, and designing of suitable learning algorithms, multiagent coordination and collision avoidance are achieved simultaneously. We show that the developed feedback-based coordination strategy guarantees uniformly ultimately bounded convergence of the closed-loop dynamical stability and all underlying motion constraints are always strictly obeyed. The effectiveness of the proposed collision-free coordination law is finally illustrated using numerical simulations.

IBERO Protocol: A Multi-agency Coordination Framework for the Tactical Emergency Casualty Care Spanish System

Introduction:Despite Spanish experience in counterterrorism, jihadism means new modus operandi, based on active mass casualty incidents, involving new injury patterns as mass bleeding ones. Therefore, the environment is completely different to first responders' structure. This change forced the revision of the tactical response model, led by the Spanish Association for Emergency Medicine's Tactical Care Secretary, who provides a discussion forum to its members, reaching a joint framework document, IBERO, adapted to Spanish tactical emergency capabilities.Method:Methodology was based on three parameters: 1)Literature revision, selecting three sources: Books and academic papers, TCCC and TECC guidelines, and Hartford Consensus, and AARs from the main active incidents with similarities to jihadist ones.2)Formation of discussion groups: Threat definition, tactile response, and prehospital care, triage and evacuation systems.3)A final group formed by representatives of each discussion group wrote the final document.Results:The protocol defines a staged plan of action, adapted to Spanish legislation and institutions, following the acronym IBERO: 1)Information: threat intelligence and information transmitted to responders from the incident location and dispatchers.2)Block the threat by perimeter, zonification and suppression of the threat.3)Escalation of resources. Definition of areas of action according to direct threat, indirect threat and safe areas of action.4)Response and Rescue. Extrication to safe areas, including different stages of triage and protection against secondary attacks.5)Order and evacuation. Access to other emergency services, and psychological first aid.Conclusion:The document proves the need for a coordinating framework of the Spanish emergency system to be fully adapted to these new threats. The discussion groups have identified the need for regular training on threat identification, zonification setting, mass bleeding control, extrication, and evacuation techniques during hostile situations. To achieve this goal, realistic training is mandatory.

Deep hierarchical generalized transformation models for spatio-temporal data with discrepancy errors

“Discrepancy error covariance” refers to the cross-covariance between the signal and the noise terms in an additive model. Traditionally, the signal and noise are assumed independent in additive models to avoid issues with confounding and non-identifiable expressions in the marginal likelihood. This assumption is made even in settings where it is known that discrepancy error covariances exists. Recently, a model has been proposed that allows for discrepancy error covariances that avoids issues with confounding. These models introduce a telescoping sum within the additive model’s expression such that the latent process of interest is dependent on other terms of the telescoping sum that are included as part of the noise. However, when evaluating the telescoping sum one obtains signal and noise terms that are independent, which avoids such concerns with confounding. The current model that allows for discrepancy error covariances only includes two terms in this telescoping sum, and consequently, a natural extension is to include more terms within the telescoping sum, which leads to a deep architecture to the statistical model. We refer to this model as the “deep hierarchical generalized transformation” (DHGT) model due to a relationship with the recently introduced hierarchical generalized transformation model. We show that the DHGT is extremely efficient to implement, and can allow for exact Bayesian implementation without the use of MCMC (i.e., we can sample directly from its posterior distribution). We illustrate the DHGT using a simulation and an analysis of the 2017 Haypress wildfire downloaded from the Geospatial Multi-Agency Coordination (GeoMAC) database. These illustrations show that discrepancy errors that arise from common model misspecifications in the spatio-temporal setting can be leveraged to improve prediction.

Mapping and review of self-neglect policies and procedures from safeguarding adults boards in England

PurposeLocal Safeguarding Adults Board (SAB) policies, procedures, guidance and related documents on self-neglect were gathered and analysed, to map what approaches are being taken across England. This paper aims to identify areas of divergence to highlight innovations or challenges faced by SABs.Design/methodology/approachSelf-neglect documents were identified by searching SAB websites. Data were extracted into a framework enabling synthesis and comparison between documents.FindingsThis paper reports on how English SAB documentation defines self-neglect, treats executive capacity, lays out pathways for self-neglect cases, advises on refusal of service input and multi-agency coordination and draws on theories or tools. Greater coherence in understanding self-neglect has developed since it was brought within safeguarding in 2014; however, variation remains regarding scope, referral pathways and threshold criteria.Research limitations/implicationsThis review was limited to published SAB documentation at one point in time and could not consider either the wider context of safeguarding guidance and training or implementation in practice.Practical implicationsThis review provides an overview of how SABs are interpreting national guidance and guiding practitioners. The trends and areas of uncertainty identified offer a resource for informed research and policy-making.Originality/valueTo the best of the author’s knowledge, this is the first systematic survey of SAB self-neglect policies, procedures and guidance since self-neglect was included under safeguarding.