Multi-agent Coordination Research Articles

Hurricane Maria Response in Puerto Rico

TPS 662: Climate change effects on labour, migration and infections, Exhibition Hall, Ground floor, August 28, 2019, 3:00 PM - 4:30 PM During disasters, large amounts of data and information about the impacted areas are collected by numerous response agencies. The repetitive nature of data collection efforts, combined with the lack of standardized methods and efforts, results in data that are often not suitable for identifying immediate needs and remedial actions. As part of the Hurricane Maria response activities, environmental assessments were conducted for health care facilities across Puerto Rico. After determining the significant amount of time required for data entry and cleaning and analysis and reporting of key findings, the assessment team leadership, in collaboration with the Puerto Rico Junta de Planificación GIS group, determined the feasibility of utilizing an app-based platform for assessment administration during the response activities. Pilot testing of the app, with geospatial capabilities, included assessing timing of survey administration and required procedures for manual and app-based data entry to determine feasibility and efficiency. A dashboard allowing real-time field data to be viewed by response leadership. The pilot work indicates using the app instead of paper-based assessments saved a minimum of 1 hour per survey that was previously required for data entry and cleaning. The errors encountered during transcription of paper-survey information into the electronic database were eliminated. Use of the app provides a unique opportunity for multi-agency coordination and the potential to decrease redundancy and duplication of efforts, decrease respondent fatigue, and increase efficiency and data quality while allowing for real-time presentation of key information to response leadership during the disaster response phase. While the app was used during the hurricane response, there are a multitude of other applications. For example, the current work as part of the Hurricane Recovery includes incorporated environmental contamination data into the app and dashboard platform to reduce the potential for environmental exposures to vulnerable populations.

Cooperative formation control of multiple aerial vehicles based on guidance route in a complex task environment

In recent years, formation control of multi-agent has been a significant research subject in the field of cooperative control. However, previous works have mainly concentrated on formation control for simple point-mass model and linear model. In contrast, this paper presents a novel cooperative algorithm for multiple air vehicles formation control, which aims to devise a control strategy based on guidance route to achieve precisely coordinated formation control for a group of fixed-wing aircraft in a complex task environment. The proposed method introduces the leader-follower structure for effective organization of the multi-agent coordination. Moreover, the Partial Integrated Formation and Control (PIFC) is adopted to design the control law for Guidance-Route based Formation Control (GRFC). Additionally, the proposed approach designs two guidance-route generation strategies for two special situations to demonstrate the effectiveness of GRFC in complex task environments. Theoretical analysis reveals that the proposed control protocol for guidance command can ensure the overall stability and tracking accuracy of the system. Numerical simulations are performed to illustrate the theoretical results, and verify that the proposed approach can achieve coordinated formation control precisely in a complex task environment.

Stochastic Semistability for Nonlinear Dynamical Systems With Application to Consensus on Networks With Communication Uncertainty

This article focuses on semistability and finite time semistability analysis and synthesis of stochastic dynamical systems having a continuum of equilibria. Stochastic semistability is the property whereby the solutions of a stochastic dynamical system almost surely converge to Lyapunov stable in probability equilibrium points determined by the system initial conditions. In this article, we extend the theories of semistability and finite-time semistability for deterministic dynamical systems to develop a rigorous framework for stochastic semistability and stochastic finite-time semistability. Specifically, Lyapunov and converse Lyapunov theorems for stochastic semistability are developed for dynamical systems driven by Markov diffusion processes. These results are then used to develop a general framework for designing semistable consensus protocols for dynamical networks in the face of stochastic communication uncertainty for achieving multiagent coordination tasks in finite time. The proposed controller architectures involve the exchange of generalized charge or energy state information between agents guaranteeing that the closed-loop dynamical network is stochastically semistable to an equipartitioned equilibrium representing a state of almost sure consensus consistent with basic thermodynamic principles.

Multi-Agent Coordination under Uncertain Communication

Multi-agent coordination is not a simple problem. While significant research has gone into computing plans efficiently and managing competing preferences, the execution of multiagent plans can still fail even when the plan space is small and agent goals are universally aligned. The reason for this difficulty is that in order to guarantee successful execution of a plan, effective multi-agent coordination requires communication to ensure that all actors have accurate beliefs about the state of the world. My thesis will focus on the problem of characterizing, modeling, and providing efficient algorithms for addressing planning and execution when there agents cannot maintain perfect communication.

An Optimal Rewiring Strategy for Cooperative Multiagent Social Learning

Multiagent coordination in cooperative multiagent systems (MASs) has been widely studied in both fixed-agent repeated interaction setting and static social learning framework. However, two aspects of dynamics in real-world MASs are currently missing. First, the network topologies can dynamically change during the course of interaction. Second, the interaction utilities between each pair of agents may not be identical and not known as a prior. Both issues mentioned above increase the difficulty of coordination. In this paper, we consider the multiagent social learning in a dynamic environment in which agents can alter their connections and interact with randomly chosen neighbors with unknown utilities beforehand. We propose an optimal rewiring strategy to select most beneficial peers to maximize the accumulated payoffs in long-run interactions. We empirically demonstrate the effects of our approach in large-scale MASs.

Learning to Teach in Cooperative Multiagent Reinforcement Learning

Collective human knowledge has clearly benefited from the fact that innovations by individuals are taught to others through communication. Similar to human social groups, agents in distributed learning systems would likely benefit from communication to share knowledge and teach skills. The problem of teaching to improve agent learning has been investigated by prior works, but these approaches make assumptions that prevent application of teaching to general multiagent problems, or require domain expertise for problems they can apply to. This learning to teach problem has inherent complexities related to measuring long-term impacts of teaching that compound the standard multiagent coordination challenges. In contrast to existing works, this paper presents the first general framework and algorithm for intelligent agents to learn to teach in a multiagent environment. Our algorithm, Learning to Coordinate and Teach Reinforcement (LeCTR), addresses peer-to-peer teaching in cooperative multiagent reinforcement learning. Each agent in our approach learns both when and what to advise, then uses the received advice to improve local learning. Importantly, these roles are not fixed; these agents learn to assume the role of student and/or teacher at the appropriate moments, requesting and providing advice in order to improve teamwide performance and learning. Empirical comparisons against state-of-the-art teaching methods show that our teaching agents not only learn significantly faster, but also learn to coordinate in tasks where existing methods fail.

The development and reform of public health in China from 1949 to 2019

BackgroundPublic health system plays a vital role in the development of health sector in China and protects the health of Chinese people. However, there are few comprehensive reviews and studies focusing on its evolution and reform. It is worthwhile to pay attention to the public health development in China, given that the history and structure of public health system have their own characteristics in China.MethodsThe study is a retrospective review of the development public health over seven decades in China. It presents the findings from some national or provincial survey data, interviews with key informants, reviews of relevant published papers and policy contents.ResultsThis study identified four key stages that public health experienced in China: the initial stage centering on prevention, the stage of deviation with more attention to treatment but little to prevention, the recovery stage after SARS(Severe Acute Respiratory Syndromes) Crisis, and the new stage to an equitable and people-centered system. In the latest stage, the National Basic Public Health Service Program (NBPHSP) is implemented to respond the threat of noncommunicable diseases (NCDs) and has achieved some initial results, while there are still many challenges including service quality, poor integration among service items and IT system, lack of quality professionals and insufficient intersectoral endeavor.DiscussionThere are unique Chinese wisdom and remarkable achievements as well as twists and turns on the development of China’s public health. Prevention-first, flexible structure of the system, multi-agency collaboration and mass mobilization and society participation are the main experience of public health in early stage. Despite twists and turns since 1980s, public health system in China shows substantial resilience which may be from the government’s continuous commitment to social development and people’s livelihoods and its flexible governance. In 2010s, in order to achieve the well-off society, Chinese government pays unprecedented attention to health sector, which bring a new wave of opportunities to public health such as remaining the NBPHSP for priority. The evolution and reform of China’s public health is based on its national condition, accumulates rich experience but also faces many common worldwide challenges. Getting this development and reform right is important to China’s social and economic development in future, and China’s experience in public health may provide many lessons for other countries.ConclusionPublic health in China needs to focus on prevention, strengthen multi-agency coordination mechanism, improve the quality of public health services in the future.

Cooperative Learning for Spectrum Management in Railway Cognitive Radio Network

High-speed rail public broadband wireless access requires communication terminals with high mobility to perform seamless cell switching in a harsh and fast time-varying environment. However, frequent switching caused by fast motion inevitably leads to unpredictable high volatility of the available spectrum, resulting in inefficient wireless communication. The application of highly mobile cognitive radio technology may address this challenge. This paper first analyzes the physical structure of the existing railway wireless communication network, and determines the characteristics of chainlike distribution and cascade operation of the base stations along the railway. Furthermore, using Bayesian reinforcement learning and multi-agent theory, a distributed cognitive base station model suitable for railway wireless environment is constructed. Based on the joint spectrum management of cascaded base station groups, this model proposes a multi-agent coordination algorithm with the goal of global optimal communication performance of the entire rail line. Finally, this paper evaluates the network performance of various test scenarios, and proves that the cognitive base station multi-agent cascade cooperative system can significantly increase the probability of successful data transmission and greatly reduce wireless spectrum handovers. This proposed scheme provides a brand-new solution for solving the problem of low wireless spectrum efficiency in the typical railway wireless network.

Multi-Agent Pursuit-Evasion Game Based on Organizational Architecture

Multi-agent coordination mechanisms are frequently used in pursuit-evasion games with the aim of enabling the coalitions of the pursuers and unifying their individual skills to deal with the complex tasks encountered. In this paper, we propose a coalition formation algorithm based on organizational principles and applied to the pursuit-evasion problem. In order to allow the alliances of the pursuers in different pursuit groups, we have used the concepts forming an organizational modeling framework known as YAMAM (Yet Another Multi Agent Model). Specifically, we have used the concepts Agent, Role, Task, and Skill, proposed in this model to develop a coalition formation algorithm to allow the optimal task sharing. To control the pursuers' path planning in the environment as well as their internal development during the pursuit, we have used a Reinforcement learning method (Q-learning). Computer simulations reflect the impact of the proposed techniques.

Distributed Gibbs: A Linear-Space Sampling-Based DCOP Algorithm


 
 
 Researchers have used distributed constraint optimization problems (DCOPs) to model various multi-agent coordination and resource allocation problems. Very recently, Ottens et al. proposed a promising new approach to solve DCOPs that is based on confidence bounds via their Distributed UCT (DUCT) sampling-based algorithm. Unfortunately, its memory requirement per agent is exponential in the number of agents in the problem, which prohibits it from scaling up to large problems. Thus, in this article, we introduce two new sampling-based DCOP algorithms called Sequential Distributed Gibbs (SD-Gibbs) and Parallel Distributed Gibbs (PD-Gibbs). Both algorithms have memory requirements per agent that is linear in the number of agents in the problem. Our empirical results show that our algorithms can find solutions that are better than DUCT, run faster than DUCT, and solve some large problems that DUCT failed to solve due to memory limitations.
 
 

A Passivity-Based Approach to Nash Equilibrium Seeking Over Networks

In this paper we consider the problem of distributed Nash equilibrium (NE) seeking over networks, a setting in which players have limited local information. We start from a continuous-time gradient-play dynamics that converges to an NE under strict monotonicity of the pseudo-gradient and assumes perfect information, i.e., instantaneous all-to-all player communication. We consider how to modify this gradient-play dynamics in the case of partial, or networked information between players. We propose an augmented gradient-play dynamics with correction in which players communicate locally only with their neighbours to compute an estimate of the other players' actions. We derive the new dynamics based on the reformulation as a multi-agent coordination problem over an undirected graph. We exploit incremental passivity properties and show that a synchronizing, distributed Laplacian feedback can be designed using relative estimates of the neighbours. Under a strict monotonicity property of the pseudo-gradient, we show that the augmented gradient-play dynamics converges to consensus on the NE of the game. We further discuss two cases that highlight the tradeoff between properties of the game and the communication graph.

Human social motor solutions for human–machine interaction in dynamical task contexts

Multiagent activity is commonplace in everyday life and can improve the behavioral efficiency of task performance and learning. Thus, augmenting social contexts with the use of interactive virtual and robotic agents is of great interest across health, sport, and industry domains. However, the effectiveness of human-machine interaction (HMI) to effectively train humans for future social encounters depends on the ability of artificial agents to respond to human coactors in a natural, human-like manner. One way to achieve effective HMI is by developing dynamical models utilizing dynamical motor primitives (DMPs) of human multiagent coordination that not only capture the behavioral dynamics of successful human performance but also, provide a tractable control architecture for computerized agents. Previous research has demonstrated how DMPs can successfully capture human-like dynamics of simple nonsocial, single-actor movements. However, it is unclear whether DMPs can be used to model more complex multiagent task scenarios. This study tested this human-centered approach to HMI using a complex dyadic shepherding task, in which pairs of coacting agents had to work together to corral and contain small herds of virtual sheep. Human-human and human-artificial agent dyads were tested across two different task contexts. The results revealed (i) that the performance of human-human dyads was equivalent to those composed of a human and the artificial agent and (ii) that, using a "Turing-like" methodology, most participants in the HMI condition were unaware that they were working alongside an artificial agent, further validating the isomorphism of human and artificial agent behavior.

Identification of Hessian matrix in distributed gradient-based multi-agent coordination control systems

Multi-agent coordination control usually involves a potential function that encodes information of a global control task, while the control input for individual agents is often designed by a gradient-based control law. The property of Hessian matrix associated with a potential function plays an important role in the stability analysis of equilibrium points in gradient-based coordination control systems. Therefore, the identification of Hessian matrix in gradient-based multi-agent coordination systems becomes a key step in multi-agent equilibrium analysis. However, very often the identification of Hessian matrix via the entry-wise calculation is a very tedious task and can easily introduce calculation errors. In this paper we present some general and fast approaches for the identification of Hessian matrix based on matrix differentials and calculus rules, which can easily derive a compact form of Hessian matrix for multi-agent coordination systems. We also present several examples on Hessian identification for certain typical potential functions involving edge-tension distance functions and triangular-area functions, and illustrate their applications in the context of distributed coordination and formation control.

Allocating structured tasks in heterogeneous agent teams

AbstractTask allocation is an important aspect of multiagent coordination. However, there are many challenges in developing appropriate strategies for multiagent teams so that they operate efficiently. Real‐world scenarios such as flooding disasters usually require the use of heterogeneous robots and the execution of tasks with different structures and complexities. In this paper, we propose a decentralized task allocation mechanism considering different types of tasks for heterogeneous agent teams where agents play different roles and carry out tasks according to their own capabilities. We have run several experiments to evaluate the proposed mechanism. The results show that the proposed mechanism appears to scale well and provides near‐optimal allocations.

SwarmGrid: Demand-Side Management with Distributed Energy Resources Based on Multifrequency Agent Coordination

This paper focuses on a multi-agent coordination for demand-side management in electrical grids with high penetration rates of distributed generation, in particular photovoltaic generation. This coordination is done by the use of swarm intelligence and coupled oscillators, proposing a novel methodology, which is implemented by the so-call SwarmGrid algorithm. SwarmGrid seeks to smooth the aggregated consumption by considering distributed and local generation by the development of a self-organized algorithm based on multifrequency agent coordination. The objective of this algorithm is to increase stability and reduce stress of the electrical grid by the aggregated consumption smoothing based on a frequency domain approach. The algorithm allows not only improvements in the electrical grid, but also increases the penetration of distributed and renewable sources. Contrary to other approaches, this objective is achieved anonymously without the need for information exchange between the users; it only takes into account the aggregated consumption of the whole grid.

Distributed Average Tracking in Multi-Agent Coordination: Extensions and Experiments

This paper addresses the distributed average tracking (DAT) problem for a group of agents to track the average of multiple time-varying reference signals, each of which is available to only one agent, under local interaction with neighbors and an undirected graph. We consider three cases: 1) DAT for single-integrator dynamics with a chain of integrators in algorithm design; 2) DAT with swarm behavior for single-integrator dynamics; and 3) DAT with swarm behavior for double-integrator dynamics. First, a continuous distributed algorithm with a chain of two integrators is proposed for single-integrator dynamics, where each agent needs its own and its neighbors’ filter outputs obtained through communication besides its absolute position, local relative positions, reference signal, and reference velocity. The introduced algorithm can deal with a wide class of reference signals with steady deviations among reference velocities. Then, two swarm-based DAT algorithms for, respectively, single- and double-integrator dynamics are presented. In the first swarm-based algorithm for single-integrator dynamics, each agent needs to measure the relative positions between itself and its neighbors, while in the second swarm-based algorithm for double-integrator dynamics, relative velocity information is required too. In both cases, the information can be obtained through local sensing. If correct initialization constraints can be achieved, the center of the agents will track the average of the reference signals and the agents will maintain connectivity and avoid interagent collision. Numerical simulations are also presented to illustrate the theoretical results. Finally, the algorithms presented in this paper are experimentally implemented and validated on a multirobot platform.

Field Team Syndromic Surveillance for Mass Gatherings: NCAA Final Four 2017

ObjectiveTo describe and present results of field-based near-real time syndromic surveillance conducted at first aid stations during the 2017 National Collegiate Athletic Association Division I Men’s College Basketball Championship (Final Four) events, and the use of field team data to improve situational awareness for Mass Gathering events.IntroductionFinal Four-associated events culminated in four days of intense activity from 3/31/17-4/3/17, which attracted an estimated 400,000 visitors to Maricopa County (population 4.2 million). Field teams of staff and volunteers were deployed to three days of Music Fest, four days of Fan Fest, and three Final Four games (Games) as part of an enhanced epidemiologic surveillance system.MethodsAttendees presenting to first aid stations were requested to complete an electronic questionnaire which captured illness and injury syndromes (after needed care was given). Emergency Medical Services technicians and nurses (EMS) conducted patient care. These were submitted and epidemiologically assessed in near-real-time to rapidly identify threats. Syndrome-specific data were mapped during events to identify spatial clustering. Field Teams were provided with case contact log sheets, suspicious substance investigation and exposure registry forms to allow rapid investigation of significant public health events. Patient Presentation Rates (PPR) and Transport to Hospital Rates (TTHR) were calculated per 10,000 attendees. Patients presenting per hour of event and transports per hour of event were calculated. Field reports were included in daily reports to inter-disciplinary partners, and shared during regular Multi-Agency Coordination Center briefings.Results301 field questionnaires were completed, including 146 from Final Four games (Games), 127 from the Music Fest, and 28 from the Fan Fest (see Figure).Among the 153,780 attendees of the three Games, there were 146 cases who presented to one of five first aid stations (over 12 hours). There were 27 illness cases who sought care (18.5% of Games cases), among whom 21 (78%) were assessed by EMS. Illness cases not assessed by EMS (n=6) included mostly allergy symptoms/medication needs. There were 50 injury cases who sought care (34.2% of Games cases), among whom 10 (20%) were assessed by EMS. Sixty (41.1%) persons presented seeking a pain reliever, and 9 (6.2%) presented seeking an antacid. Games experienced a PPR of 9.5, and a TTHR of 0.52. Patients presented at 12.2 per hour on average and there were eight transports to medical facilities (0.66 per hour).There were 127 cases among an estimated 135,000 Music Fest attendees who presented to one of two first aid stations (and at times 2 roving teams) over 3 days (22.5 hours) from 3/31 to 4/2. Illnesses accounted for 29 cases (22.8% of Music Fest cases) and 28 of 29 were assessed by EMS. There were 68 injury cases who sought care (53.5% of Music Fest cases), among whom 22 (32.4%) were assessed by EMS. Twenty-seven persons (21.3%) presented seeking a pain reliever and 2 (1.6%) sought an antacid. Music Fest results included a PPR of 9.4, and a TTHR of 0.15. There were 5.6 patients presenting per hour on average, and there were two transports to the hospital (0.09 per hour).At the Fan Fest there were 28 cases among an estimated 50,803 attendees presenting to the first aid station (or roving teams) from 3/31-4/3 (over 37 hours). Most cases sought care for an injury (n=22, 78.6% of cases). Four persons sought care for an illness (14.3%), all with relatively minor complaints. For the Fan Fest, there was a PPR of 5.5, and a TTHR of 0 (there were no transports to the hospital). There were 0.76 patient presentations per hour on average.No geographic clustering or public health threats requiring investigation were identified at any of the three sites. Interdisciplinary partners requested additional field data during the response.ConclusionsInjuries were more common than illnesses at all three sites. Visits requiring pain relievers only were more common at Games (41.1%) than at Music Fest (21.3%) or Fan Fest (3.6%).A greater percentage of visits requiring EMS assessment were seen at the Fan Fest (78.6%) than at the Music Fest (40.2%) or the Games (21.2%).The PPRs per 10,000 attendees were highest at the Games (9.5) and Music Fest (9.4), compared to the Fan Fest (5.5). The TTHR per 10,000 attendees was highest at the Games (0.52), compared to the Music Fest (0.15) and Fan Fest (0.0). The Music Fest field team reported greater effectiveness at fixed first aid stations compared to traveling with roving EMS teams. Field reports enhanced health and medical situational awareness and information sharing as evidenced by requests from interdisciplinary partners for additional field data.

A disaster multiagent coordination simulation system to evaluate the design of a first‐response team

AbstractIdentifying the best design configuration for a first‐response team is important for minimizing total operation time and reducing the human cost of natural and manmade disasters. This paper presents ongoing research that focuses on a disaster multiagent coordination simulation (DMCsim) system that is able to optimally design the first‐response team and evaluate the team design configuration before initiation of a search and rescue operation. We developed an agent‐based simulation system that uses machine learning techniques and design of experiments methods to test different configuration setups and determine the effects of various factors on operation completion time. The evaluation of a team design for a disaster‐response operation revealed that some design factors have a significant effect on operation outcome. Removing the effects of uncontrollable factors, such as damage level and robot reliability, yielded a robust team design that could function in a particular disaster environment regardless of the effects of such factors. The DMCsim assists decision makers to evaluate an emergency‐response operation, revise the current strategy based on resources on hand, redesign the available team, and visually track operation performance before launching the actual team in the disaster field. This research extends previous disaster response coordination systems by proposing a new simulation model and evaluating a first‐response team design.

An overview of the consensus problem in the control of multi-agent systems

ABSTRACTAs a solution to the distributed multi-agent coordination, the problems of consensus or agreement have been widely explored and studied in the literature. This document provides an overview of consensus problems in multi-agent cooperative control with the goal of exposing the related literature and promote the research in this area. The document presents the theoretical results concerning the search for consensus in the involved topologies with information exchange that is invariant in time and change dynamically. Applications related to consensus protocols are studied for the cooperation of multi-agent systems. The presentation includes as well open problems and offers future research direction.

An Adaptive Learning Based Network Selection Approach for 5G Dynamic Environments.

Networks will continue to become increasingly heterogeneous as we move toward 5G. Meanwhile, the intelligent programming of the core network makes the available radio resource be more changeable rather than static. In such a dynamic and heterogeneous network environment, how to help terminal users select optimal networks to access is challenging. Prior implementations of network selection are usually applicable for the environment with static radio resources, while they cannot handle the unpredictable dynamics in 5G network environments. To this end, this paper considers both the fluctuation of radio resources and the variation of user demand. We model the access network selection scenario as a multiagent coordination problem, in which a bunch of rationally terminal users compete to maximize their benefits with incomplete information about the environment (no prior knowledge of network resource and other users’ choices). Then, an adaptive learning based strategy is proposed, which enables users to adaptively adjust their selections in response to the gradually or abruptly changing environment. The system is experimentally shown to converge to Nash equilibrium, which also turns out to be both Pareto optimal and socially optimal. Extensive simulation results show that our approach achieves significantly better performance compared with two learning and non-learning based approaches in terms of load balancing, user payoff and the overall bandwidth utilization efficiency. In addition, the system has a good robustness performance under the condition with non-compliant terminal users.