Multi-agent Planning Research Articles

SafeMAP: Safe Multi-Agent Planning framework based on Dynamic Probabilistic Risk Assessment

This paper proposes a risk-aware framework for Safe Multi-Agent Planning (SafeMAP) that unifies heterogeneous models for multi-agent systems in a Markovian process that allows for simultaneous system health monitoring, decision making under uncertainty, and multi-agent system collaboration. As operations beyond low earth orbit mature, there is an increased need for autonomous cyber-physical systems with onboard decision making capabilities. Multi-agent cyber-physical systems in particular offer the potential of increased efficiency, resiliency, and mission capabilities for future applications such as multi-rover terrain operations, distributed satellite operations, and management of smart lunar habitats. SafeMAP utilizes physics-based models of each agent and the relevant components, probability models of the environment and component operational states, and reward models for mission-specific objectives such as scientific task completion or resource consumption. The output of SafeMAP is a set of mission plans that satisfy the mission objective under specified risk/reward constraints. A readable interpretation of each of these generated mission plans is provided as an additional output. SafeMAP has been demonstrated on a simulated case study involving a four-rover system performing surface mapping operations and science tasks. The results of this paper demonstrate SafeMAP’s ability to generate explainable mission plans that satisfy the mission objective while minimizing risk under nominal and off-nominal conditions.

Decentralized Multi-Agent Planning Using Model Predictive Control and Time-Aware Safe Corridors

Multi-agent planning has numerous real-world applications, such as autonomous agriculture, search and rescue, and infrastructure inspection. It is thus of tremendous benefit to improve on the state-of-the-art in multiple metrics to make autonomous multi-agent systems more efficient and a more viable real-world solution. In this letter, we propose a new framework for multi-agent planning in a static environment that improves upon the existing state-of-the-art in multiple areas such as computation time, trajectory smoothness, and feasibility. The proposed method first generates a global path that only avoids static obstacles and then generates a Safe Corridor around it. It then extends the notion of Safe Corridors and makes them time-aware in order to account for the future positions of other agents. The time-aware Safe Corridor, along with a local reference trajectory generated from the global path, are used in an Mixed-Integer Quadratic Program/Model Predictive Control formulation that generates a collision-free optimal trajectory. The proposed framework is real-time, decentralized, and synchronous. It is compared to two recent state-of-the-art methods in simulations. It outperforms both methods in robustness as well as feasibility and computation time.

Remuneration Sensitivity Analysis in Prosumer and Aggregator Strategies by Controlling Electric Vehicle Chargers

The efficient use of energy resources is profoundly changing power grid regulation and policy. New forms of power generation coupled with storage and the presence of new, increasingly flexible loads such as electric vehicles enable the development of multi-agent planning systems based on new forms of interaction. For instance, consumers can take advantage of flexibility by interacting directly with the grid or through aggregators that bridge the gap between these end-users and traditional centralised markets. This paper aims to provide insight into the benefits for aggregators and end-users from a financial perspective by proposing a methodology that can be applied to different scenarios. End-users may provide flexibility services related to private vehicle charging stations or battery storage systems. The paper will analyse different remuneration levels for end-users by highlighting the most beneficial scenarios for aggregators and end-users and providing evidence on potential conflict of interests. The numerical results show that some consumers may benefit more from aggregation. This is because if taken individually, consumption habits do not allow the same flexibility when considering clusters of consumers with different behaviour. It is also shown that there are cases in which consumers do not seem to benefit from the presence of intermediate parties. We provide extensive numerical results to gain insight for better decision making.

On the Use of Width-Based Search for Multi Agent Privacy-Preserving Planning (Extended Abstract)

The aim of decentralised multi-agent (DMA) planning is to coordinate a set of agents to jointly achieve a goal while preserving their privacy. Blind search algorithms, such as width-based search, have recently proved to be very effective in the context of centralised automated planning, especially when combined with goal-oriented techniques. In this paper, we discuss a recent line of research in which the usage of width-based search has been extensively studied in the context of DMA planning, addressing the challenges related to the agents' privacy and performance.

Privacy leakage of search-based multi-agent planning algorithms

Privacy preservation has become one of the crucial research topics in multi-agent planning. A number of techniques to preserve private information throughout the planning process have emerged. One major difficulty of such research is the comparison of properties related to privacy among such techniques. A metric allowing for comparison of such privacy preservation was introduced only recently, having a number of drawbacks such as prohibitive computational complexity. In this work we strengthen the theoretical foundations and simplify the metric in order to be practically usable. Moreover, we test the usability of the metric in an analysis of various techniques in multi-agent heuristic computation and search, determining which are the most beneficial in terms of privacy preservation. We also evaluate the techniques in terms of the classical IPC score to assess their impact on the overall planning performance. The results are somewhat surprising and show that extracting any privacy-related information even from the simplest variant of heuristic search is a very complicated task. Existing techniques such as distributed heuristic and sending only relevant states is shown to reduce the privacy leakage even more.

Fault Tolerance for Composite Cloud Services: A MAS-based Novel Approach

Several Cloud services may be composed in order to respond quickly to the users’ needs. Unfortunately, when running such a service some faults may occur. The outcome of fault control is a big challenge. In this paper, the authors propose a new approach based back recovery and multi-agent planning methods. The proposed architecture-based MAS (Multi-Agent System) is composed of two main types of Agents: a Composition Manager Agent (CMA) and a Supervisor Agent (SA). The role of the CMA is to create a set of plans as an oriented graph where the nodes are the Cloud services and the valued arcs represent the composition order of these services. This agent saves checkpoints (nodes) in a stable memory so that there is at least one possible path. However, the SA ensures that the running plan is working properly; otherwise, it informs the CMA to select another sub-plan. Experimental results show the performance of the proposed approach in terms of different metrics such as response time, cost, and total cost of different alternatives of cloud services.

Multi-Agent Tree Search with Dynamic Reward Shaping

Sparse rewards and their representation in multi-agent domains remains a challenge for the development of multi-agent planning systems. While techniques from formal methods can be adopted to represent the underlying planning objectives, their use in facilitating and accelerating learning has witnessed limited attention in multi-agent settings. Reward shaping methods that leverage such formal representations in single-agent settings are typically static in the sense that the artificial rewards remain the same throughout the entire learning process. In contrast, we investigate the use of such formal objective representations to define novel reward shaping functions that capture the learned experience of the agents. More specifically, we leverage the automaton representation of the underlying team objectives in mixed cooperative-competitive domains such that each automaton transition is assigned an expected value proportional to the frequency with which it was observed in successful trajectories of past behavior. This form of dynamic reward shaping is proposed within a multi-agent tree search architecture wherein agents can simultaneously reason about the future behavior of other agents as well as their own future behavior.

GamePlan: Game-Theoretic Multi-Agent Planning With Human Drivers at Intersections, Roundabouts, and Merging

We present a new method for multi-agent planning involving human drivers and autonomous vehicles (AVs) in unsignaled intersections, roundabouts, and during merging. In multi-agent planning, the main challenge is to predict the actions of other agents, especially human drivers, as their intentions are hidden from other agents. Our algorithm uses game theory to develop a new auction, called <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GamePlan</small> , that directly determines the optimal action for each agent based on their driving style (which is observable via commonly available sensors). <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GamePlan</small> assigns a higher priority to more aggressive or impatient drivers and a lower priority to more conservative or patient drivers; we theoretically prove that such an approach is game-theoretically optimal prevents collisions and deadlocks. We compare our approach with prior state-of-the-art auction techniques including economic auctions, time-based auctions (first-in first-out), and random bidding and show that each of these methods result in collisions among agents when taking into account driver behavior. We compare with methods based on DRL, deep learning, and game theory and present our benefits over these approaches. Finally, we show that our approach can be implemented in the real-world with human drivers.

REVIEW OF THEORETICAL APPROACHES TO USING OF ARTIFICIAL INTELLIGENCE FOR PLANNING PROBLEMS IN ECONOMICS

Artificial intelligence methods and technologies are increasingly included in human's everyday life. Managing actors in the context of their activities, from the planning stage to the decision-making stage, are faced with the need to operate with big data, non-linear, exponentially growing, critically overloaded data scenarios. In these conditions, the need to introduce artificial intelligence technologies is due to the exhaustion of the intellectual and analytical capabilities of a person. The article discusses a variety of methods and approaches of artificial intelligence, examines the content of key algorithms, models and theories, their strengths and weaknesses in such important areas of the economy as planning and decision-making. The focus is on their classification. Due to the dependence of the planning process on environmental factors, both classical and non-classical planning environments are discussed. If the environment is fully observable, deterministic and static (external changes are ignored) and discrete in terms of time and action, then we are dealing with a classical planning environment. In the case of a partially observable or stochastic environment, we get a non-classical planning environment. The simplest and most intuitive approach to the planning process algorithms is a Total Order Planning. A scheduling algorithm with parallel execution of actions or without specifying the sequence of their execution is a Partial Order Planning algorithm. Recent research into the development of efficient algorithms has sparked interest in one of the earliest planning approaches – Prepositional Logic Planning. With the Critical Path Method, a schedule of activities is drawn up as part of a plan with zero critical travel time margin for each activity, taking into account the calculation of the time margin for each activity and sequence of activities. A forward-looking planning method for solving complex problems is a hierarchical decomposition based on a Hierarchical Task Networks. The influence of time and resource factors on planning procedures is separately highlighted. Approaches and methods used in a non-classical planning environment: compatible planning, conditional planning, continuous planning, multi-agent planning. Special attention is paid to the issues of constructing planning models in conditions of uncertainty based on the theoretical-probabilistic (stochastic) approaches. Bayesian networks are used to represent vagueness. The Relational Probability Model includes certain constraints on the presentation means, thereby guaranteeing a fully defined probability distributions. The main tasks of probabilistic representation in temporal models are: filtering, forecasting, smoothing, determining a probabilistic explanation. By combining these algorithms and additional enhancements, three large blocks of temporal models can be obtained: Hidden Markov Models, Kalman Filter, and Dynamic Bayesian Network. Decision theory allows the agent to determine the sequence of actions to be performed. A simpler formal system for solving decision-making problems is decision-making networks. The use of expert systems containing information about utility creates additional opportunities. Sequential multiple decision problems in an uncertain environment, such as Markov Decision Processes, are defined using transition models. When several agents interact simultaneously, game theory is used to describe the rational behavior of agents. As we can see, planning has recently become one of the most interesting and relevant directions in the field of artificial intelligence research. There is still a long way to go: it is necessary to develop a clear vision of the problem of choosing the appropriate specific methods depending on the type of task, perhaps by creating completely new methods and approaches.

An efficient lightweight coordination model to multi-agent planning

An efficient lightweight coordination model to multi-agent planning

Privacy-aware Distributed Diagnosis of Multi-Agent Plans

In many multi-agent systems (MAS), agents are assigned to perform tasks. These tasks require designing plans for the agents, which are called multi-agent plan (MAP). Such systems are prone to failures due to variety of reasons. When a failure occurs, there is a need to diagnose it and identify the faulty agents. This problem has been traditionally addressed in a centralized manner. However, in some systems, agents might not be able to share plans due to privacy or single point of failure reasons. To address this challenge, we propose Distributed Diagnosis of Multi-Agent Plans (DDMAP) algorithms, which identify faulty agents of a MAS in a distributed manner without sharing plans. The contributions of this paper are: (1) formalizing DDMAP as a model-based diagnosis problem, and (2) presenting synchronous, asynchronous and semi-asynchronous distributed algorithms to diagnose the faulty agents. Experiments show that the semi-asynchronous algorithm performs better in terms of run-time while the performance in terms of communication overhead is comparable.

When Advocates are Forced to go Virtual: The Long-Term Care Ombudsman Response to COVID 19

Animal-assisted activities (AAA) and therapy standards of practice have been published to protect the well-being of animals, animal handlers, and the special populations of patients that benefit from this mode of treatment. Inconsistencies among practice standards with concerns surrounding the topics of dog welfare, human well-being, and zoonotic transmission have been reported. The purpose of this qualitative research study was to review published AAA and therapy standards with older adult populations for best practices, conduct focus group sessions with caregivers from long-term care facilities that allow therapy dog visitation, and synthesize findings into an AAA checklist to be used by long-term care facility decision-makers when interviewing or bringing in therapy dog teams. Comparative analyses utilizing a systematic and sequential approach was used to analyze the data from the focus group sessions. Due to the COVID-19 pandemic, only two focus group sessions at one long-term care facility were conducted resulting in a total of 15 caregivers. Four themes emerged from the data: promotes positive mood, essential resident screenings, caregiver roles, and memory aides. Relevant themes and AAA and therapy standards and guidelines were then combined in the development of the AAA/Therapy Dog Checklist. Administrators may find having a user-friendly AAA/therapy dog checklist a useful tool that can be used when interviewing therapy dog teams to ensure future dog therapy experiences will be positive and safe. The safety and well-being of residents in long-term care facilities and animals are essential to promote positive health outcomes for both populations.

Mission Planning Based on Dynamic Agent Interaction Graph for Deep Space Probes

Facing the increasingly complex deep-space exploration missions and the dynamic space environment, deep-space probes need efficient planning methods for the fast generation of plans. The distribution and concurrency of subsystems make a probe suitable to be modeled as a multi-agent system. Existing multi-agent planners, however, cannot be used directly in mission planning of deep space probes that involve handling numeric constraints such as time resources. To solve the above problem, a multi-agent mission plan-space planning method based on distributed refinement search was proposed. A dynamic agent interaction graph (DAIG) was designed to coordinate interactions between agents during planning. Temporal constraints and resource constraints were modeled as constraint satisfaction problems and were handled by graph theory methods. Experiments show that the method proposed in this paper can save computing time of mission planning problems for a probe with multiple subsystems.

Following the Death of Ms A: A Local Authority Response to Transitional Safeguarding

This article discusses work done over several years in the London Borough of Havering (LBH) to develop and strengthen services for young people transitioning from Children’s Services to adulthood, care-experienced young people in particular. The impetus for this development work was a Safeguarding Adult Review (SAR) published in 2017 concerning a care-experienced young adult, Ms A, who was receiving Leaving Care services at the time of her death at the age of 20. Following a successful funding bid to the Department for Education, LBH made significant investment in transitions services, co-producing these with young adults themselves in conjunction with a work-support mental health charity MAC-UK. This work resulted in the creation of: The Cocoon, a new ‘one-stop shop’ onsite multi-agency service provision; and the Transitions Panel, a new multi-agency planning forum. This article describes the development of The Cocoon and the Transitions Panel, and summarises the impact of these developments.

Trial-Based Heuristic Tree-Search for Distributed Multi-Agent Planning

We present a novel search scheme for privacy-preserving multi-agent planning. Inspired by UCT search, the scheme is based on growing an asynchronous search tree by running repeated trials through the tree. We describe key differences to classical multi-agent forward search, discuss theoretical properties of the presented approach, and evaluate it based on benchmarks from the CoDMAP competition. As a secondary contribution, we describe a technique that extends the regular search approach by small explorative trials which are performed subsequent to each node expansion. We show that this technique significantly increases the number of problems solved for all algorithms considered, including MAFS.

Trial-Based Heuristic Tree-search for Distributed Multi-Agent Planning

We present a novel search scheme for privacy-preserving multi-agent planning, inspired by UCT search. We compare the presented approach to classical multi-agent forward search and evaluate it based on benchmarks from the CoDMAP competition.

Novelty Messages Filtering for Multi Agent Privacy-Preserving Plannin

In multi-agent planning, agents jointly compute a plan that achieves mutual goals, keeping certain information private to the individual agents. Agents' coordination is achieved through the transmission of messages, but they can be a source of privacy leakage as they can permit a malicious agent to collect information about other agents' search processes and states. In this paper, we investigate the usage of novelty techniques in the context of (decentralised) multi-agent privacy preserving planning, addressing the challenges related to the agents' privacy and performance. In particular, we show that novelty based techniques allow a significant reduction on the number of messages transmitted among agents, increasing their privacy levels and also their performances. An experimental study analyses the effectiveness of our techniques and compares them with the state of-the-art. Finally, we examine the robustness of our approach considering different delays in the messages transmission as would occur in overloaded networks, due for example to massive attacks or critical situations.

Stubborn Sets Pruning for Privacy Preserving Planning

We adapt a partial order reduction technique based on stubborn sets to the setting of privacy-preserving multi-agent planning. We prove that the presented approach preserves optimality and show experimentally that it can significantly improve search performance on some domains.

Fast Random Genetic Search for Large-Scale RTS Combat Scenarios

This paper makes a contribution to the advancement of artificial intelligence in the context of multi-agent planning for large-scale combat scenarios in RTS games. This paper introduces Fast Random Genetic Search (FRGS), a genetic algorithm which is characterized by a small active population, a crossover technique which produces only one child, dynamic mutation rates, elitism, and restrictions on revisiting solutions. This paper demonstrates the effectiveness of FRGS against a static AI and a dynamic AI using the Portfolio Greedy Search (PGS) algorithm. In the context of the popular Real-Time Strategy (RTS) game, StarCraft, this paper shows the advantages of FRGS in combat scenarios up to the maximum size of 200 vs. 200 units under a 40 ms time constraint.

A Factored Approach to Deterministic Contingent Multi-Agent Planning

Collaborative Multi-Agent Planning (MAP) under uncertainty with partial observability is a notoriously difficult problem. Such MAP problems are often modeled as DecPOMDPs, or its qualitative variant, QDec-POMDP, which is essentially a MAP version of contingent planning. The QDecPOMDP model was introduced with the hope that its simpler, non-probabilistic structure will allow for better scalability. Indeed, at least with deterministic actions, the recent IMAP algorithm scales much better than comparable DecPOMDP algorithms (Bazinin and Shani 2018). In this work we suggest a new approach to solving Deterministic QDecPOMDPs based on problem factoring. First, we find a solution to a MAP problem where the results of any observation is available to all agents. This is essentially a single-agent planning problem for the entire team. Then, we project the solution tree into sub-trees, one per agent, and let each agent transform its projected tree into a legal local tree. If all agents succeed, we combine the trees into a valid joint-plan. Otherwise, we continue to explore the space of team solutions. This approach is sound, complete, and as our empirical evaluation demonstrates, scales much better than the IMAP algorithm.