Multi-agent Teams Research Articles

Knowledge-based strategies for multi-agent teams playing against Nature

We study teams of agents that play against Nature towards achieving a common objective. The agents are assumed to have imperfect information due to partial observability, and have no communication during the play of the game. We propose a natural notion of higher-order knowledge of agents. Based on this notion, we define a class of knowledge-based strategies, and consider the problem of synthesis of strategies of this class. We introduce a multi-agent extension, MKBSC, of the well-known knowledge-based subset construction applied to such games. Its iterative applications turn out to compute higher-order knowledge of the agents. We show how the MKBSC can be used for the design of knowledge-based strategy profiles, and investigate the transfer of existence of such strategies between the original game and in the iterated applications of the MKBSC, under some natural assumptions. We also relate and compare the “intensional” view on knowledge-based strategies based on explicit knowledge representation and update, with the “extensional” view on finite memory strategies based on finite transducers and show that, in a certain sense, these are equivalent.

Onto4MAT: A Swarm Shepherding Ontology for Generalized Multiagent Teaming

Research in multi-agent teaming has increased substantially over recent years, with knowledge-based systems to support teaming processes typically focused on delivering functional (communicative) solutions for a team to act meaningfully in response to direction. Enabling humans to effectively interact and team with a swarm of autonomous cognitive agents is an open research challenge in Human-Swarm Teaming research, partially due to the focus on developing the enabling architectures to support these systems. Typically, bi-directional transparency and shared semantic understanding between agents has not prioritised a designed mechanism in Human-Swarm Teaming, potentially limiting how a human and a swarm team can share understanding and information\textemdash data through concepts and contexts\textemdash to achieve a goal. To address this, we provide a formal knowledge representation design that enables the swarm Artificial Intelligence to reason about its environment and system, ultimately achieving a shared goal. We propose the Ontology for Generalised Multi-Agent Teaming, Onto4MAT, to enable more effective teaming between humans and teams through the biologically-inspired approach of shepherding.

Impact of Heterogeneity and Risk Aversion on Task Allocation in Multi-Agent Teams

Cooperative multi-agent decision-making is a ubiquitous problem with many real-world applications. In many practical applications, it is desirable to design a multi-agent team with a heterogeneous composition where the agents can have different capabilities and levels of risk tolerance to address diverse requirements. While heterogeneity in multi-agent teams offers benefits, new challenges arise including how to find optimal heterogeneous team compositions and how to dynamically distribute tasks among agents in complex operations. In this work, we develop an artificial intelligence framework for multi-agent heterogeneous teams to dynamically learn task distributions among agents through reinforcement learning. The framework extends Decentralized Partially Observable Markov Decision Processes (Dec-POMDP) to be compatible to model various types of heterogeneity. We demonstrate our approach with a benchmark problem on a disaster relief scenario. The effect of heterogeneity and risk aversion in agent capabilities and decision-making strategies on the performance of multi-agent teams in uncertain environments is analyzed. Results show that a well-designed heterogeneous team outperforms its homogeneous counterpart and possesses higher adaptivity in uncertain environments.

Characterizing Multi-Agent Team Behavior from Partial Team Tracings: Evidence from the English Premier League

Real-world AI systems have been recently deployed which can automatically analyze the plan and tactics of tennis players. As the game-state is updated regularly at short intervals (i.e. point-level), a library of successful and unsuccessful plans of a player can be learnt over time. Given the relative strengths and weaknesses of a player’s plans, a set of proven plans or tactics from the library that characterize a player can be identified. For low-scoring, continuous team sports like soccer, such analysis for multi-agent teams does not exist as the game is not segmented into “discretized” plays (i.e. plans), making it difficult to obtain a library that characterizes a team’s behavior. Additionally, as player tracking data is costly and difficult to obtain, we only have partial team tracings in the form of ball actions which makes this problem even more difficult. In this paper, we propose a method to overcome these issues by representing team behavior via play-segments, which are spatio-temporal descriptions of ball movement over fixed windows of time. Using these representations we can characterize team behavior from entropy maps, which give a measure of predictability of team behaviors across the field. We show the efficacy and applicability of our method on the 2010-2011 English Premier League soccer data.

Hierarchical control of multi-agent reinforcement learning team in real-time strategy (RTS) games

Coordinated control of multi-agent teams is an important task in many real-time strategy (RTS) games. In most prior work, micromanagement is the commonly used strategy whereby individual agents operate independently and make their own combat decisions. On the other extreme, some employ a macromanagement strategy whereby all agents are controlled by a single decision model. In this paper, we propose a hierarchical command and control architecture, consisting of a single high-level and multiple low-level reinforcement learning agents operating in a dynamic environment. This hierarchical model enables the low-level unit agents to make individual decisions while taking commands from the high-level commander agent. Compared with prior approaches, the proposed model provides the benefits of both flexibility and coordinated control. The performance of such hierarchical control model is demonstrated through empirical experiments in a real-time strategy game known as StarCraft: Brood War (SCBW).

Toward competitive multi-agents in Polo game based on reinforcement learning

The learning of agents in a competitive space such as a game is a challenging task. The aim of the proposed research is to improve the reinforcement learning techniques in a competitive multi-agent for the Polo game. First, the video dataset is prepared. Then, the rules of the Polo game are extracted as a class diagram. An architecture is designed for multi-agent team in the Polo game. Therefore, an algorithm is proposed for the temporal difference in the game belief space for improving reward catching. The reward function is implemented in the agent team. Finally, the research improvement is evaluated by increasing 31 units in comparison with previous work. Therefore, competitive learning in the agent team has been improved.

Greedy Decentralized Auction-based Task Allocation for Multi-Agent Systems

We propose a decentralized auction-based algorithm for the solution of dynamic task allocation problems for spatially distributed multi-agent systems. In our approach, each member of the multi-agent team is assigned to at most one task from a set of spatially distributed tasks, while several agents can be allocated to the same task. The task assignment is dynamic since it is updated at discrete time stages (iterations) to account for the current states of the agents as the latter move towards the tasks assigned to them at the previous stage. Our proposed methods can find applications in problems of resource allocation by intelligent machines such as the delivery of packages by a fleet of unmanned or semi-autonomous aerial vehicles. In our approach, the task allocation accounts for both the cost incurred by the agents for the completion of their assigned tasks (e.g., energy or fuel consumption) and the rewards earned for their completion (which may reflect, for instance, the agents' satisfaction). We propose a Greedy Coalition Auction Algorithm (GCAA) in which the agents possess bid vectors representing their best evaluations of the task utilities. The agents propose bids, deduce an allocation based on their bid vectors and update them after each iteration. The solution estimate of the proposed task allocation algorithm converges after a finite number of iterations which cannot exceed the number of agents. Finally, we use numerical simulations to illustrate the effectiveness of the proposed task allocation algorithm (in terms of performance and computation time) in several scenarios involving multiple agents and tasks distributed over a spatial 2D domain.

Communication-Aware Multi-Agent Metareasoning for Decentralized Task Allocation

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Metareasoning</i> refers to reasoning about one’s own decision making process. This paper considers metareasoning about the decision making process in multi-agent settings. We present a multi-agent metareasoning approach that enables a multi-agent team to select which task allocation algorithm to use as a function of changing communication quality level. Given a set of multi-agent task allocation algorithms, we synthesize a policy that prescribes the best algorithm to use among a predefined set of algorithms for a given communication level. Since each agent in the team runs the same policy, the team (or a part of the team) will collectively switch between task allocation algorithms as a function of the observed level of communication. We apply reactive synthesis to generate the policy from high-level specifications written in Linear Temporal Logic encoding the agents’ switching behavior with respect to the state of the environment. We perform experiments in simulation to identify the best performing algorithms under different communication levels. The communication environment is modeled using the Rayleigh fading model and communication estimation is done through the exchange of heartbeat messages among agents. We test our metareasoning policy in three types of scenarios: search & rescue, fire monitoring, and ship protection scenarios. For each scenario, we demonstrate that our policy achieved better performance with respect to either max distance traveled, max number of transmitted messages or both compared to running any single algorithm.

Time-Aware Multi-Agent Symbiosis.

Contemporary research in human-machine symbiosis has mainly concentrated on enhancing relevant sensory, perceptual, and motor capacities, assuming short-term and nearly momentary interaction sessions. Still, human-machine confluence encompasses an inherent temporal dimension that is typically overlooked. The present work shifts the focus on the temporal and long-lasting aspects of symbiotic human-robot interaction (sHRI). We explore the integration of three time-aware modules, each one focusing on a diverse part of the sHRI timeline. Specifically, the Episodic Memory considers past experiences, the Generative Time Models estimate the progress of ongoing activities, and the Daisy Planner devices plans for the timely accomplishment of goals. The integrated system is employed to coordinate the activities of a multi-agent team. Accordingly, the proposed system (i) predicts human preferences based on past experience, (ii) estimates performance profile and task completion time, by monitoring human activity, and (iii) dynamically adapts multi-agent activity plans to changes in expectation and Human-Robot Interaction (HRI) performance. The system is deployed and extensively assessed in real-world and simulated environments. The obtained results suggest that building upon the unfolding and the temporal properties of team tasks can significantly enhance the fluency of sHRI.

Optimal composition of heterogeneous multi-agent teams for coverage problems with performance bound guarantees

We consider the problem of determining the optimal composition of a heterogeneous multi-agent team for coverage problems by including costs associated with different agents and subject to an upper bound on the maximal allowable number of agents. We formulate a resource allocation problem without introducing additional non-convexities to the original problem. We develop a distributed Projected Gradient Ascent (PGA) algorithm to solve the optimal team composition problem. To deal with non-convexity, we initialize the algorithm using a greedy method and exploit the submodularity and curvature properties of the coverage objective function to derive novel tighter performance bound guarantees on the optimization problem solution. Numerical examples are included to validate the effectiveness of this approach in diverse mission space configurations and different heterogeneous multi-agent collections. Comparative results obtained using a commercial mixed-integer nonlinear programming problem solver demonstrate both the accuracy and computational efficiency of the distributed PGA algorithm.

The impact of agent definitions and interactions on multiagent learning for coordination in traffic management domains

The state-action space of an individual agent in a multiagent team fundamentally dictates how the individual interacts with the rest of the team. Thus, how an agent is defined in the context of its domain has a significant effect on team performance when learning to coordinate. In this work we explore the trade-offs associated with these design choices, for example, having fewer agents in the team that individually are able to process and act on a wider scope of information about the world versus a larger team of agents where each agent observes and acts in a more local region of the domain. We focus our study on a traffic management domain and highlight the trends in learning performance when applying different agent definitions. In addition, we analyze the impact of agent failure for different agent definitions and investigate the ability of the team to learn new coordination strategies when individual agents become unresponsive.

Market Clearing–based Dynamic Multi-agent Task Allocation

Realistic multi-agent team applications often feature dynamic environments with soft deadlines that penalize late execution of tasks. This puts a premium on quickly allocating tasks to agents. However, when such problems include temporal and spatial constraints that require tasks to be executed sequentially by agents, they are NP-hard, and thus are commonly solved using general and specifically designed incomplete heuristic algorithms. We propose FMC_TA, a novel such incomplete task allocation algorithm that allows tasks to be easily sequenced to yield high-quality solutions. FMC_TA first finds allocations that are fair (envy-free), balancing the load and sharing important tasks among agents, and efficient (Pareto optimal) in a simplified version of the problem. It computes such allocations in polynomial or pseudo-polynomial time (centrally or distributedly, respectively) using a Fisher market with agents as buyers and tasks as goods. It then heuristically schedules the allocations, taking into account inter-agent constraints on shared tasks. We empirically compare our algorithm to state-of-the-art incomplete methods, both centralized and distributed, on law enforcement problems inspired by real police logs. We present a novel formalization of the law enforcement problem, which we use to perform our empirical study. The results show a clear advantage for FMC_TA in total utility and in measures in which law enforcement authorities measure their own performance. Besides problems with realistic properties, the algorithms were compared on synthetic problems in which we increased the size of different elements of the problem to investigate the algorithm’s behavior when the problem scales. The domination of the proposed algorithm was found to be consistent.

A Receding Horizon Scheduling Approach for Search & Rescue Scenarios

Many applications involving complex multi-task problems such as disaster relief, logistics and manufacturing necessitate the deployment and coordination of heterogeneous multi-agent systems due to the sheer number of tasks that must be executed simultaneously. A fundamental requirement for the successful coordination of such systems is leveraging the specialization of each agent within the team. This work presents a Receding Horizon Planning (RHP) framework aimed at scheduling tasks for heterogeneous multi-agent teams in a robust manner. In order to allow for the modular addition and removal of different types of agents to the team, the proposed framework accounts for the capabilities that each agent exhibits (e.g. quadrotors are agile and agnostic to rough terrain but are not suited to transport heavy payloads). An instantiation of the proposed RHP is developed and tested for a search and rescue scenario. Moreover, we present an abstracted search and rescue simulation environment, where a heterogeneous team of agents is deployed to simultaneously explore the environment, find and rescue trapped victims, and extinguish spreading fires as quickly as possible. We validate the effectiveness of our approach through extensive simulations comparing the presented framework with various planning horizons to a greedy task allocation scheme.

Using Human-Agent Teams to Purposefully Design Multi-Agent Systems

With multi-agent teams becoming more of a reality every day, it is important to create a common design model for multi-agent teams. These teams need to be able to function in dynamic environments and still communicate with any humans that may need a problem solved. Existing human-agent research can be used to purposefully create multi-agent teams that are interdependent but can still interact with humans. Rather than creating dynamic agents, the most effective way to overcome the dynamic nature of modern workloads is to create a dynamic team configuration, rather than individual member-agents that can change their roles. Multi-agent teams will require a variety of agents to be designed to cover a diverse subset of problems that need to be solved in the modern workforce. A model based on existing multi-agent teams that satisfies the needs of human-agent teams has been created to serve as a baseline for human-interactive multi-agent teams.

Intention-based fairness preferences in multi-partner project teams

We propose a stylized model of cooperative effort expenditure in multi-agent teams which is driven by intention-based preferences for fairness. Differing from preferences about material payoffs, we consider agents with intention-based preferences for fairness who also care about the motivation that drives other agents’ actions. In the spirit of Rabin (1993), we define the agent's kindness to other agents in terms of how much more or less than other agents’ equitable payoff that agent believes she is giving to the other agent. In a series of propositions, we compare to each other two kinds of agents: “selfish” agents who are only motivated to maximize their material payoffs, and “unselfish” agents who are also motivated by considerations of intention-based fairness to their co-workers. Our analysis shows that the effect of intention-based fairness preferences on the agents’ cooperative effort expenditure depends critically on the effort cost of the agents and on major attributes of the project. Additionally, the intention-based fairness preference may change the way that the project attributes influence the agents’ cooperative effort choices. We conclude the paper by proposing recommendations for enhancing cooperation of the agents in project teams.

Personalized change awareness: Reducing information overload in loosely-coupled teamwork

Complex tasks such as treating patients with chronic conditions and developing software products are typically accomplished by teams that collaborate over an extended time duration. To remain coordinated, team members need to be aware of others' activities if those activities are likely to affect their own actions. However, identifying such interactions and sharing information appropriately is challenging, especially when the activities of team members are loosely-coupled. In practice, team members often either lack important information about others' activities, or are overwhelmed by the need to review too much information. This paper presents Personalized Change Awareness, a new approach for supporting team coordination which aims to automatically identify and share the subset of information about others' activities that is most relevant to each of the team members. The paper formally defines the computational problem of information sharing in loosely-coupled teamwork, which underlies the personalized change awareness approach. It defines a new representation, Mutual Influence Potential Networks (MIP-Nets) and an algorithm, MIP-DOI, that uses this representation to determine the information that is most relevant to each team member. In contrast to existing information sharing algorithms in multi-agent teams, MIP-DOI does not assume the availability of a priori knowledge of a team's possible plans, because human teams rarely explicitly define detailed long-term plans in advance. We demonstrate the ability of MIP-DOI to identify relevant information using simulations of collaborative activities. We further evaluated the contribution of personalized change awareness to team performance in a controlled user study. To this end, we developed a personalized change awareness mechanism for collaborative writing, which used MIP-DOI to determine which changes to share with each author. This evaluation demonstrates that the Personalized Change Awareness approach resulted in higher productivity and lower perceived workload without any change in final quality compared to the currently prevalent approach of sharing all change information. Our results also demonstrate that merely reducing the amount of information shared with co-authors is not enough: sharing a random subset of changes resulted in significantly lower quality of work than sharing a personalized subset of changes.

Intentional Forgetting in Distributed Artificial Intelligence

In times of Big Data and Industry 4.0, organizational information as well as knowledge availability and quantity are driving complex decision-making tasks. Especially for AI systems, increasing knowledge-bases for elaborate computations lead to a lower efficiency of their inference mechanisms. In contrast to AI, bounded cognitive capacity is a well-known problem in psychology. When humans receive too much information, a state of information overload emerges. In order to cope with limited capacity and prevent information overload, humans adapt their knowledge and delete, override, suppress, or sort out outdated information, i.e., they forget. By transferring theories from human cognition to multiagent systems, the AdaptPRO project adopts intentional forgetting as a strategy for coping with information overload in both human and multiagent teams. This article gives an overview of an interdisciplinary research project with a strong focus on knowledge distributions and knowledge dynamics from a distributed AI perspective. Its core contribution is a formal model for distributing and adapting (meta-) knowledge by intentional forgetting to enable efficient and resilient teamwork.

Dynamic Team Heterogeneity in Cooperative Coevolutionary Algorithms

We propose Hyb-CCEA, a cooperative coevolutionary algorithm for the evolution of genetically heterogeneous multiagent teams. The proposed approach extends the cooperative coevolution architecture with operators that put the number of coevolving populations under evolutionary control. Populations are dynamically merged based on behavioral similarity, thus decreasing team heterogeneity, and stochastic population splits are used to explore increased team heterogeneity. Hyb-CCEA is capable of converging to suitable team compositions for the given task, be it a completely homogeneous team where all agents share the same control logic, a heterogeneous team where each agent has distinct control logic, or a partially heterogeneous team. By placing both team composition and agent controllers under evolutionary control, Hyb-CCEA can be applied to domains for which the experimenter has limited or no knowledge about possible solutions. We study Hyb-CCEA extensively in an abstract domain, and conduct a series of validation experiments with four simulated multirobot tasks: two multirover foraging tasks and two robotic soccer tasks. The results show that Hyb-CCEA takes advantage of partial heterogeneity and frequently outperforms the standard cooperative coevolution approach, both in terms of fitness scores achieved and number of evaluations needed to evolve solutions.

Allocating structured tasks in heterogeneous agent teams

AbstractTask allocation is an important aspect of multiagent coordination. However, there are many challenges in developing appropriate strategies for multiagent teams so that they operate efficiently. Real‐world scenarios such as flooding disasters usually require the use of heterogeneous robots and the execution of tasks with different structures and complexities. In this paper, we propose a decentralized task allocation mechanism considering different types of tasks for heterogeneous agent teams where agents play different roles and carry out tasks according to their own capabilities. We have run several experiments to evaluate the proposed mechanism. The results show that the proposed mechanism appears to scale well and provides near‐optimal allocations.

The privileged sensing framework: A principled approach to improved human-autonomy integration

ABSTRACTA primary goal for human-autonomy integration (HAI) is to balance the strengths of human and autonomy in order to achieve performance objectives more efficiently and robustly than either the human or autonomous agents would independently. This paper proposes the Privileged Sensing Framework (PSF) as a novel approach to HAI. This approach is based on the concept of dynamically ‘privileging’ information during the process of integration by dynamically bestowing special rights based on the characteristics of each individual agent, the task context, and the performance goals. The proposed framework is tested through a series of simulation experiments that provide a clear demonstration of increased accuracy and throughput of human-autonomy performance. These proof-of-concept simulations provide initial evidence of the utility of the PSF. Continued development of this approach has the potential to revolutionise capabilities of multi-agent cooperative teams across a broad range of applications.