Multi-agent Planning Research Articles

On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)

Automated Planning and Scheduling is among the growing areas in Artificial Intelligence (AI) where mention of LLMs has gained popularity. Based on a comprehensive review of 126 papers, this paper investigates eight categories based on the unique applications of LLMs in addressing various aspects of planning problems: language translation, plan generation, model construction, multi-agent planning, interactive planning, heuristics optimization, tool integration, and brain-inspired planning. For each category, we articulate the issues considered and existing gaps. A critical insight resulting from our review is that the true potential of LLMs unfolds when they are integrated with traditional symbolic planners, pointing towards a promising neuro-symbolic approach. This approach effectively combines the generative aspects of LLMs with the precision of classical planning methods. By synthesizing insights from existing literature, we underline the potential of this integration to address complex planning challenges. Our goal is to encourage the ICAPS community to recognize the complementary strengths of LLMs and symbolic planners, advocating for a direction in automated planning that leverages these synergistic capabilities to develop more advanced and intelligent planning systems. We aim to keep the categorization of papers updated on https://ai4society.github.io/LLM-Planning-Viz/, a collaborative resource that allows researchers to contribute and add new literature to the categorization.

Economic and low-carbon planning for interconnected integrated energy systems considering emerging technologies and future development trends

The integrated energy system facilitates the synergistic and efficient utilization of diverse energy. Establishing interconnected regional energy internets by linking multiple integrated energy systems enables the realization of cross-sector, cross-temporal, and cross-subject energy coupling, consequently enhancing economic benefits, driving profound decarbonization, and bolstering energy resilience. This study develops a multi-agent planning model grounded in an electricity-heat-hydrogen sharing framework, collaboratively optimizing capacity configurations and energy scheduling strategies across various integrated energy systems containing emerging technologies. Furthermore, this study integrates the trading volume and prices of various energy sources to distinguish the energy sharing contribution of each participant, then combines with the Nash bargaining game, forming a fair cooperative benefit allocation incentive mechanism. The proposed model is adeptly solved using the alternating directions method of multipliers, ensuring the protection of participants’ private information. To reflect seasonal characteristics of energy supply and demand, this study uses the daily data of three integrated energy systems across four seasons to simulate. The research results reveal an annual cost saving of 233.73 thousand RMB and a reduction of 290.85 tCO2. Additionally, the impact of variations in electricity operation modes, uncertainties, and emerging technical advances are evaluated to further validate the sustainability of the multi-energy trading mechanism.

Offline policy reuse-guided anytime online collective multiagent planning and its application to mobility-on-demand systems

Offline policy reuse-guided anytime online collective multiagent planning and its application to mobility-on-demand systems

Domain Engineering to Represent Human Behavior Using Multi-Agent Planning and Inductive Methodologies

This research combines multi agent planning, the psycholinguistics of question asking, procedural grounded theory, and hierarchical task networks to represent domains for automated planning.

Leveraging active queries in collaborative robotic mission planning

This paper focuses on the high-level specification and generation of 3D models for operational environments using the idea of active queries as a basis for specifying and generating multi-agent plans for acquiring such models. Assuming an underlying multi-agent system, an operator can specify a request for a particular type of model from a specific region by specifying an active query. This declarative query is then interpreted and executed by collecting already existing data/information in agent systems or, in the active case, by automatically generating high-level mission plans for agents to retrieve and generate parts of the model that do not already exist. The purpose of an active query is to hide the complexity of multi-agent mission plan generation, data transformations, and distributed collection of data/information in underlying multi-agent systems. A description of an active query system, its integration with an existing multi-agent system and validation of the active query system in field robotics experimentation using Unmanned Aerial Vehicles and simulations are provided.

Electricity–gas multi-agent planning method considering users’ comprehensive energy consumption behavior

With the advent of the energy Internet and the swift growth of unified energy systems, the comprehensive energy demand of users has gradually become a problem that cannot be ignored for the planning of integrated energy systems. Aiming at this problem, this paper suggests a multi-agent planning approach for electricity and gas, considering users’ holistic energy consumption behavior. First, utilizing a combined subjective and objective weighting method, this study establishes a utility model for users’ energy consumption characteristics. The analysis of comprehensive energy consumption behavior is conducted through an evolutionary game. On this basis, the planning revenue model for electricity grid and gas network investors is formulated, and the game mechanism of different investors is analyzed. A dynamic game model of electricity–gas multi-agent planning considering comprehensive energy consumption behavior is proposed. Ultimately, the model is resolved using an iterative exploration approach. The validity and efficacy of the proposed method are confirmed through a simulation example.

Distributed, Dynamic and Recursive Planning for Holonic Multi-Agent Systems: A Behavioural Model-Based Approach

In this work, we propose a new distributed, dynamic, and recursive planning approach able to consider the hierarchical nature of the holonic agent and the unpredictable evolution of its behaviour. For each new version of the holonic agent, introduced because of the agent members obtaining new roles to achieve new goals and adapt to the changing environment, the approach generates a new plan that can solve the new planning problem associated with this new version against which the plans, executed by the holonic agent, become obsolete. To do this, the approach starts by generating sub-plans capable of solving the planning subproblems associated with the groups of the holonic agent at its different levels. It then recursively links the sub-plans, according to their hierarchical and behavioural dependency, to obtain a global plan. To generate the sub-plans, the approach exploits the behavioural model of the holonic agent’s groups, thereby minimising the computation rate imposed by other multi-agent planning methods. In our work, we have used a concrete case to show and illustrate the usefulness of our approach.

MultiStyle: Characterizing Multiplayer Cooperative Gameplay by Incorporating Distinct Player Playstyles in a Multi-Agent Planner

This paper presents MultiStyle, a multi-agent centralized heuristic search planner that incorporates distinct agent playstyles to generate solution plans where characters express individual preferences while cooperating to reach a goal. We include algorithmic details, an example domain, and multiple different solution plans generated with unique agent playstyle sets. We discuss our intent to incorporate this planner in a tool for game level designers to help them anticipate and understand how teams of players with distinct playstyles may play through their levels. Ultimately, MultiStyle generates solution plans with a novel and increased expressive range by attempting to satisfy sets of action and proposition preferences for each agent.

Preference-Based Multi-Robot Planning for Nuclear Power Plant Online Monitoring and Diagnostics

Current preventative maintenance paradigms in nuclear power plants carry several costly risks and challenges associated with component downtime and the need for human data collection. Preventative maintenance may be enabled by an online monitoring system that accurately assesses component condition and identifies potential faults. We present an approach for autonomous online monitoring and multiagent planning for robotic data collection. Under the occurrence of a fault, we utilize a machine learning model to form an initial guess of its nature, which we then refine by selectively measuring certain variables to gain additional information via a situation-aware variable selection model. To generate a multi-robot plan to conduct these measurements, we develop a preference-based planning framework within a linear temporal logic–based planning approach that prioritizes collecting data from the most important features. Finally, we demonstrate our approach on a case study using a simulated nuclear power plant circulating water system, showing fault diagnostic performance as well as simulated robot data collection.

Improving Zero-Shot Coordination Performance Based on Policy Similarity

Over these years, multi-agent reinforcement learning have achieved remarkable performance in multi-agent planning and scheduling tasks. It typically follows the self-play setting, where agents are trained by playing with a fixed group of agents. However, in the face of zero-shot coordination, where an agent must coordinate with unseen partners, self-play agents may fail. Several methods have been proposed to handle this problem, but they either take a lot of time or lack generalizability. In this paper, we firstly reveal an important phenomenon: the zero-shot coordination performance is strongly linearly correlated with the similarity between an agent's training partner and testing partner. Inspired by it, we put forward a Similarity-Based Robust Training (SBRT) scheme that improves agents' zero-shot coordination performance by disturbing their partners' actions during training according to a pre-defined policy similarity value. To validate its effectiveness, we apply our scheme to three multi-agent reinforcement learning frameworks and achieve better performance compared with previous methods.

Second-Order Quantified Boolean Logic

Second-order quantified Boolean formulas (SOQBFs) generalize quantified Boolean formulas (QBFs) by admitting second-order quantifiers on function variables in addition to first-order quantifiers on atomic variables. Recent endeavors establish that the complexity of SOQBF satisfiability corresponds to the exponential-time hierarchy (EXPH), similar to that of QBF satisfiability corresponding to the polynomial-time hierarchy (PH). This fact reveals the succinct expression power of SOQBFs in encoding decision problems not efficiently doable by QBFs. In this paper, we investigate the second-order quantified Boolean logic with the following main results: First, we present a procedure of quantifier elimination converting SOQBFs to QBFs and a game interpretation of SOQBF semantics. Second, we devise a sound and complete refutation-proof system for SOQBF. Third, we develop an algorithm for countermodel extraction from a refutation proof. Finally, we show potential applications of SOQBFs in system design and multi-agent planning. With these advances, we anticipate practical tools for development.

Multi-agent planning and coordination for automated aircraft ground handling

Inspired by the vision of fully autonomous airside operations at Schiphol airport, this study aims to contribute to the short-term goal of automated aircraft ground handling. In this research, we design and evaluate a multi-agent system for planning of automated ground handling. There are two main components in the system: task allocation optimization and multi-agent path planning. To allocate tasks to ground support equipment (GSE) vehicles, an auction mechanism inspired by temporal sequential single item (TeSSI) auction is proposed. Ground handling tasks scheduling for GSE vehicles is modeled as several single-vehicle pickup and delivery optimization problems (SPDP), and the values of the objective functions are used to generate bids for GSE vehicle agents in the auction. Prioritized safe interval path planning for large agents (LA-SIPP) is used to plan collision-free paths for GSE vehicle agents in the model to execute tasks. The aim is to increase the success rates of allocating tasks and finding collision free paths without causing flight delays, given the limited resources such as a small number of available GSE vehicles, time windows constraints and conflicting interests of different agents. Due to the results, even for the instances with frequent flights and the most limited resources, the success rates of allocation and path planning were higher than 81% and 98%, respectively. Furthermore, periodic task allocation and path planning of the ground handling tasks for flights in three aircraft stands during a planning time window of the day, as well as replanning in case of disruptions were performed in a short CPU time. There is a lack of research dealing with the complete process of ground handling, since existing studies concerning the automation of ground handling operations involve fleet assignment or task scheduling models without an integration of detailed path planning. Our main contribution is to present a framework that combines task allocation and path planning for automation of ground handling operations and provides solutions using a multi-agent perspective.

Multiagent Systems on Virtual Games: A Systematic Mapping Study

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Context</i> : Games are a well-established scenario to test AI and Multi-Agent Systems (MAS) proposals due to their popularity and defiance. However, there is no big picture of the application of this technology to games, the evolution of the kind of problem tackled, or the game scenarios in which agents have been experimented. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective</i> : To perform a systematic mapping to characterise the state of the art in the field of MAS applied to virtual games and to identify trends, strengths, and gaps for further research. Method: A Systematic Mapping Study has been conducted to find primary studies in the field. A search was performed on title, abstracts, and keywords, whilst classification, data extraction, and further analysis were performed according to specific criteria focused on MAS papers with experimentation and evidence in a game scenario. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results</i> : 78 studies published between 1998 and 2021 were found. Studies have been classified according to the MAS problem faced and the agent reasoning strategy. We detect that Machine Learning is the most common AI technique for MAS in games, considering both reinforcement learning and evolutionary techniques. MAS are used in a variety of gaming genres, especially in Real-Time Strategy (RTS), Sports and Simulation. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusions</i> : RTS and Sports games are well-suited for concrete MAS problems such as multi-agent planning and task allocation. Expanding evidence and experimentation on other aspects related to scalability and usability issues is discussed. Those MAS problems and experiments that remain slightly modelled on games or are not thoroughly studied yet have been also identified.

Best-response planning for urban fleet coordination

The modeling of fleet vehicles as self-interested agents brings a realistic perspective to open fleet transportation research. This feature allows us to model the fleet operation from a non-cooperative point of view. In this work, we study parcel delivery in a city with limited resources (roads and charging stations). We designed and implemented a system composed of a multi-agent planner and a game-theoretic coordination algorithm: a Best-Response Fleet Planner. The system allows for the self-organization of the transportation system by coordinating a fleet of self-interested electric vehicles. The system’s operation is optimized together with resource usage while preserving the agents’ private interests, allowing each agent to plan its actions. The results show that our system has higher scalability than similar approaches, allowing it to function for a considerable number of agents in settings that feature congestion and conflicts. Additionally, overall solution quality is improved compared to other coordination systems, reducing congestion and avoiding unnecessary waiting times.

Automatic parameter learning method for agent activation spreading network by evolutionary computation

A variety of planning research is being actively conducted in multiple research fields. The focus of these studies is to flexibly utilize both immediate and deliberative planning in response to the environment and to adaptively prioritize multiple goals and actions in a human-like manner. To achieve this, a method that applies active propagation to multi-agent planning (agent activation spreading network) has been proposed and is being utilized in various research fields. Furthermore, with the recent development of large-scale artificial intelligence models, we should soon be able to incorporate tacit human knowledge into this architecture. However, there is not yet a method for adjusting the parameters in this architecture which creates a barrier to future extension. In response, we have developed a method for automatically adjusting the parameters using evolutionary computation. Our experimental results showed that (1) the proposed method enables a higher degree of adaptation, thanks to taking the agent’s semantics into account, and (2) it is possible to obtain parameters that are appropriate to the environment even when the experimental environment is changed.

A Conflict-Based Search Framework for Multiobjective Multiagent Path Finding

Conventional multi-agent path planners typically compute an ensemble of paths while optimizing a single objective, such as path length. However, many applications may require multiple objectives, say fuel consumption and completion time, to be simultaneously optimized during planning and these criteria may not be readily compared and sometimes lie in competition with each other. The goal of the problem is thus to find a Pareto-optimal set of solutions instead of a single optimal solution. Naively applying existing multi-objective search algorithms, such as multi-objective A* (MOA*), to multi-agent path finding may prove to be inefficient as the dimensionality of the search space grows exponentially with the number of agents. This article presents an approach named Multi-Objective Conflict-Based Search (MO-CBS) that attempts to address this so-called curse of dimensionality by leveraging prior Conflict-Based Search (CBS), a well-known algorithm for single-objective multi-agent path finding, and principles of dominance from multi-objective optimization literature. We also develop several variants of MO-CBS to improve its performance. We prove that MO-CBS and its variants can compute the entire Pareto-optimal set. Numerical results show that MO-CBS outperforms MOM*, a recently developed state-of-the-art multi-objective multi-agent planner.

Width-based search for multi agent privacy-preserving planning

In multi-agent planning, preserving the agents' privacy has become an increasingly popular research topic. For preserving the agents' privacy, agents jointly compute a plan that achieves mutual goals by keeping certain information private to the individual agents. Unfortunately, this can severely restrict the accuracy of the heuristic functions used while searching for solutions. It has been recently shown that, for centralized planning, blind search algorithms such as width-based search can solve instances of many existing domains in low polynomial time when they feature atomic goals. Moreover, the performance of goal-oriented search can be improved by combining it with width-based search. In this paper, we investigate the usage of width-based search in the context of (decentralised) collaborative multi-agent privacy-preserving planning, addressing the challenges related to the agents' privacy and performance. In particular, we show that width-based search is a very effective approach over several benchmark domains, even when the search is driven by heuristics that roughly estimate the distance from goal states, computed without using the private information of other involved agents. Moreover, we show that the use of width-based techniques can significantly reduce the number of messages transmitted among the agents, better preserving their privacy and improving their performance. An experimental study presented in the paper analyses the effectiveness of our techniques, and compares them with the state-of-the-art of collaborative multi-agent planning.

Multi-Agent Exploration-Based Search for an Unknown Number of Targets

This paper presents an active sensor fusion technique for multiple mobile agents (robots) to detect an unknown number of static targets at unknown positions. To process and fuse sensor measurements from the agents, we use a random finite set formulation with an iterated-corrector probability hypothesis density filter. Our main contribution is to introduce two different multi-agent planners to quickly find the targets. The planners make greedy decisions for the next state of each agent by maximizing an objective function consisting of target refinement and exploration components. We demonstrate the performance of our approach through a series of simulations using homogeneous and heterogeneous agents. The results show that our framework works better than a lawnmower baseline, and that a centralized version of the planner works best.

Hierarchical multi-agent planning for flexible assembly of large-scale lunar facilities

Several national space agencies and commercial aerospace companies plan to set up lunar bases with large-scale facilities that rely on multiple lunar robots’ assembly. Mission planning is necessary to achieve efficient multi-robot cooperation. This paper aims at autonomous multi-robot planning for the flexible assembly of the large-scale lunar facility, considering the harsh lunar environment, mission time optimization, and joint actions. The lunar robots and modules are scattered around the mission area without fixed assembly lines. Thus, the traditional assembly planning methods ignoring the optimal selection of modules are unable to handle this problem. We propose a hierarchical multi-agent planning method based on two-stage two-sided matching (HMAP-TTM) to solve this critical problem. First, the distributed planning framework with multi-replica public agents is introduced, ensuring robot plan knowledge consistency through public agents’ communication. Second, the hierarchical task graph (HTG) divides the mission into task layers based on task dependency knowledge. Third, we develop a novel two-stage two-sided matching algorithm. Time-optimal plans emerge from the matching games among public and private agents in each layer of HTG. Agents make decisions in the game based on action knowledge updated during planning. Finally, an assembly mission is presented to prove the method’s effectiveness. The simulation results show that the HMAP-TTM can generate plans with shorter mission time and require smaller communication costs than the baseline methods.

Correctness-guaranteed strategy synthesis and compression for multi-agent autonomous systems

Planning is a critical function of multi-agent autonomous systems, which includes path finding and task scheduling. Exhaustive search-based methods such as model checking and algorithmic game theory can solve simple instances of multi-agent planning. However, these methods suffer from state-space explosion when the number of agents is large. Learning-based methods can alleviate this problem, but lack a guarantee of correctness of the results. In this paper, we introduce MoCReL, a new version of our previously proposed method that combines model checking with reinforcement learning in solving the planning problem. The approach takes advantage of reinforcement learning to synthesize path plans and task schedules for large numbers of autonomous agents, and of model checking to verify the correctness of the synthesized strategies. Further, MoCReL can compress large strategies into smaller ones that have down to 0.05% of the original sizes, while preserving their correctness, which we show in this paper. MoCReL is integrated into a new version of Uppaal Stratego that supports calling external libraries when running learning and verification of timed games models.