Multi-agent Path Planning Research Articles

Path planning and collision avoidance approach for a multi-agent system in grid environments

Abstract In multi-robot systems, ensuring efficient, safe, and conflict-free navigation for each robot is crucial, especially in scenarios requiring the coordination of complex tasks and interactions with the environment. Currently, most related research focuses on simplified grid environments, making it difficult to apply these findings to real-world scenarios. This paper introduces actual physical constraints into traditional grid environment path planning and designs a set of motion rules suitable for grid environments and a collision avoidance algorithm. We also propose a novel multi-agent path planning approach that integrates deep reinforcement learning with collision avoidance algorithms. Through a series of experiments, we have demonstrated that this method effectively guides agents in navigating from their starting points to their targets within grid environments that possess physical constraints, and significantly reduces the probability of collisions when used in conjunction with our collision avoidance algorithm. These findings not only showcase the efficacy of our approach but also open new avenues for deploying multi-agent systems in more complex real-world scenarios.

A hybrid optimization algorithm for multi-agent dynamic planning with guaranteed convergence in probability

The paper aims to solve the problem of multi-agent path planning in complex environment using optimization algorithm. To address the issue of local optimum and premature convergence, a new method is proposed based on the whale optimization algorithm, combining the chaotic initialization, the reverse search and the differential evolution methods. It is theoretically proved that this algorithm is globally convergent in probability. When applied to path planning problems, the proposed optimization algorithm can effectively find a globally optimal and smoother path. Through simulation experiments with multi-UAVs, it is demonstrated that the proposed algorithm has better performance than the state-of-the-art methods in environment with both static and dynamic obstacles, reflecting the global convergence and robustness of the proposed algorithm.

Multi-UAV Path Planning and Following Based on Multi-Agent Reinforcement Learning

Dedicated to meeting the growing demand for multi-agent collaboration in complex scenarios, this paper introduces a parameter-sharing off-policy multi-agent path planning and the following approach. Current multi-agent path planning predominantly relies on grid-based maps, whereas our proposed approach utilizes laser scan data as input, providing a closer simulation of real-world applications. In this approach, the unmanned aerial vehicle (UAV) uses the soft actor–critic (SAC) algorithm as a planner and trains its policy to converge. This policy enables end-to-end processing of laser scan data, guiding the UAV to avoid obstacles and reach the goal. At the same time, the planner incorporates paths generated by a sampling-based method as following points. The following points are continuously updated as the UAV progresses. Multi-UAV path planning tasks are facilitated, and policy convergence is accelerated through sharing experiences among agents. To address the challenge of UAVs that are initially stationary and overly cautious near the goal, a reward function is designed to encourage UAV movement. Additionally, a multi-UAV simulation environment is established to simulate real-world UAV scenarios to support training and validation of the proposed approach. The simulation results highlight the effectiveness of the presented approach in both the training process and task performance. The presented algorithm achieves an 80% success rate to guarantee that three UAVs reach the goal points.

Evolutionary optimization for risk-aware heterogeneous multi-agent path planning in uncertain environments.

Cooperative multi-agent systems make it possible to employ miniature robots in order to perform different experiments for data collection in wide open areas to physical interactions with test subjects in confined environments such as a hive. This paper proposes a new multi-agent path-planning approach to determine a set of trajectories where the agents do not collide with each other or any obstacle. The proposed algorithm leverages a risk-aware probabilistic roadmap algorithm to generate a map, employs node classification to delineate exploration regions, and incorporates a customized genetic framework to address the combinatorial optimization, with the ultimate goal of computing safe trajectories for the team. Furthermore, the proposed planning algorithm makes the agents explore all subdomains in the workspace together as a formation to allow the team to perform different tasks or collect multiple datasets for reliable localization or hazard detection. The objective function for minimization includes two major parts, the traveling distance of all the agents in the entire mission and the probability of collisions between the agents or agents with obstacles. A sampling method is used to determine the objective function considering the agents' dynamic behavior influenced by environmental disturbances and uncertainties. The algorithm's performance is evaluated for different group sizes by using a simulation environment, and two different benchmark scenarios are introduced to compare the exploration behavior. The proposed optimization method establishes stable and convergent properties regardless of the group size.

A coordinated scheduling approach for task assignment and multi-agent path planning

Path finding is an essential problem in multi-agent systems, widely employed in warehousing and logistics. However, most of the current studies focus on the problem of assigning one agent with one task in a period, which may hinder the efficiency of path planning of the systems when facing multi-tasks. To address this problem, we propose a multi-layer planner, under which a co-scheduling method for multi-agent with batch tasks and path planning in a continuous workspace is proposed. In the task allocation layer of the framework, a standard Genetic Algorithm (GA) is adopted, which optimizes the allocation of task sets, and minimizes the path lengths and the probability of conflicts. Secondly, an Improved Car-Like Conflict-Based Search (ICL-CBS) algorithm is presented as the conflict resolution layer to reduce the runtime. Finally, ICL-CBS and the Spatiotemporal Hybrid-State A* (SHA*) algorithm are jointly used to determine multi-agent paths in the path planning layer. We conduct experiments and compare our method with the baseline algorithm on random obstacles and practical scenarios. The results show that our method effectively improves the efficiency of multi-task completion, and alleviates the computational burden of underlying path planning. Specifically, our method has less runtime.

Path Planning Technique for Mobile Robots: A Review

Mobile robot path planning involves designing optimal routes from starting points to destinations within specific environmental conditions. Even though there are well-established autonomous navigation solutions, it is worth noting that comprehensive, systematically differentiated examinations of the critical technologies underpinning both single-robot and multi-robot path planning are notably scarce. These technologies encompass aspects such as environmental modeling, criteria for evaluating path quality, the techniques employed in path planning and so on. This paper presents a thorough exploration of techniques within the realm of mobile robot path planning. Initially, we provide an overview of eight diverse methods for mapping, each mirroring the varying levels of abstraction that robots employ to interpret their surroundings. Furthermore, we furnish open-source map datasets suited for both Single-Agent Path Planning (SAPF) and Multi-Agent Path Planning (MAPF) scenarios, accompanied by an analysis of prevalent evaluation metrics for path planning. Subsequently, focusing on the distinctive features of SAPF algorithms, we categorize them into three classes: classical algorithms, intelligent optimization algorithms, and artificial intelligence algorithms. Within the classical algorithms category, we introduce graph search algorithms, random sampling algorithms, and potential field algorithms. In the intelligent optimization algorithms domain, we introduce ant colony optimization, particle swarm optimization, and genetic algorithms. Within the domain of artificial intelligence algorithms, we discuss neural network algorithms and fuzzy logic algorithms. Following this, we delve into the different approaches to MAPF planning, examining centralized planning which emphasizes decoupling conflicts, and distributed planning which prioritizes task execution. Based on these categorizations, we comprehensively compare the characteristics and applicability of both SAPF and MAPF algorithms, while highlighting the challenges that this field is currently grappling with.

Path Planning Algorithm for Dual-Arm Robot Based on Depth Deterministic Gradient Strategy Algorithm

In recent years, the utilization of dual-arm robots has gained substantial prominence across various industries owing to their collaborative operational capabilities. In order to achieve collision avoidance and facilitate cooperative task completion, efficient path planning plays a pivotal role. The high dimensionality associated with collaborative task execution in dual-arm robots renders existing path planning methods ineffective for conducting efficient exploration. This paper introduces a multi-agent path planning reinforcement learning algorithm that integrates an experience replay strategy, a shortest-path constraint, and the policy gradient method. To foster collaboration and avoid competition between the robot arms, the proposed approach incorporates a mechanism known as “reward cooperation, punishment competition” during the training process. Our algorithm demonstrates strong performance in the control of dual-arm robots and exhibits the potential to mitigate the challenge of reward sparsity encountered during the training process. The effectiveness of the proposed algorithm is validated through simulations and experiments, comparing the results with existing methods and showcasing its superiority in dual-arm robot path planning.

Research on path planning techniques based on deep reinforcement learning

Path planning problem stands for a sort of problem that find a secure path from the departure position to the destination in the unique application area without conflicts with other objects under the premise of minimum time or distance cost. Path planning has many applications in real life. This paper summarizes and classifies the current main research results on path planning, which are classified according to conventional techniques, single-agent path planning techniques and multi-agent path planning techniques. Conventional techniques are classified into three different algorithms: Traditional techniques, Graphics techniques and Intelligent bionics techniques. For the single-agent path planning techniques, it is primarily broken down into two categories: value-based and strategy-based. For multi-agent techniques, according to the improvement techniques, it is split in three categories: expert demonstration type, improved communication type and task decomposition type. Based on above classification, the mechanisms, advantages, and disadvantages of the existing algorithms are summarized and compared, and the future work in this field is prospected.

Offline Time-Independent Multiagent Path Planning

This study examines a novel planning problem for multiple agents that cannot share holding resources, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Offline Time-Independent Multiagent Path Planning (OTIMAPP)</i> . Given a graph and a set of start-goal pairs, the problem to be addressed is assigning a path to each agent, such that every agent eventually reaches its destination without blocking others, regardless of when each agent starts and finishes each own action. This motivation stems from timing uncertainties, including the reality gaps between planning and robot execution. In contrast to conventional solution, concepts of multirobot path planning that rely on timings, once OTIMAPP solutions are obtained, they can be executed without any synchronization between robot actions. Moreover, there is a theoretical guarantee that all robots eventually reach their destinations, provided they avoid interrobot collisions. This study attempts to establish OTIMAPP both theoretically and practically. Specifically, we present a formalization of the problem, solution conditions based on a categorization of deadlocks, computational complexities showing that OTIMAPP is computationally intractable, practical relaxation of the solution concept, two algorithms to solve OTIMAPP based on multiagent pathfinding algorithms, empirical results showing large OTIMAPP instances can be solved to some extent, as well as robot demonstrations of asynchronous OTIMAPP execution.

Periodic Multi-Agent Path Planning

Multi-agent path planning (MAPP) is the problem of planning collision-free trajectories from start to goal locations for a team of agents. This work explores a relatively unexplored setting of MAPP where streams of agents have to go through the starts and goals with high throughput. We tackle this problem by formulating a new variant of MAPP called periodic MAPP in which the timing of agent appearances is periodic. The objective with periodic MAPP is to find a periodic plan, a set of collision-free trajectories that the agent streams can use repeatedly over periods, with periods that are as small as possible. To meet this objective, we propose a solution method that is based on constraint relaxation and optimization. We show that the periodic plans once found can be used for a more practical case in which agents in a stream can appear at random times. We confirm the effectiveness of our method compared with baseline methods in terms of throughput in several scenarios that abstract autonomous intersection management tasks.

Multi-agent planning and coordination for automated aircraft ground handling

Inspired by the vision of fully autonomous airside operations at Schiphol airport, this study aims to contribute to the short-term goal of automated aircraft ground handling. In this research, we design and evaluate a multi-agent system for planning of automated ground handling. There are two main components in the system: task allocation optimization and multi-agent path planning. To allocate tasks to ground support equipment (GSE) vehicles, an auction mechanism inspired by temporal sequential single item (TeSSI) auction is proposed. Ground handling tasks scheduling for GSE vehicles is modeled as several single-vehicle pickup and delivery optimization problems (SPDP), and the values of the objective functions are used to generate bids for GSE vehicle agents in the auction. Prioritized safe interval path planning for large agents (LA-SIPP) is used to plan collision-free paths for GSE vehicle agents in the model to execute tasks. The aim is to increase the success rates of allocating tasks and finding collision free paths without causing flight delays, given the limited resources such as a small number of available GSE vehicles, time windows constraints and conflicting interests of different agents. Due to the results, even for the instances with frequent flights and the most limited resources, the success rates of allocation and path planning were higher than 81% and 98%, respectively. Furthermore, periodic task allocation and path planning of the ground handling tasks for flights in three aircraft stands during a planning time window of the day, as well as replanning in case of disruptions were performed in a short CPU time. There is a lack of research dealing with the complete process of ground handling, since existing studies concerning the automation of ground handling operations involve fleet assignment or task scheduling models without an integration of detailed path planning. Our main contribution is to present a framework that combines task allocation and path planning for automation of ground handling operations and provides solutions using a multi-agent perspective.

Dual deep Q-learning network guiding a multiagent path planning approach for virtual fire emergency scenarios

Dual deep Q-learning network guiding a multiagent path planning approach for virtual fire emergency scenarios

Collaborative optimization of task scheduling and multi-agent path planning in automated warehouses

AbstractTask scheduling (TS) and multi-agent-path-finding (MAPF) are two cruxes of pickup-and-delivery in automated warehouses. In this paper, the two cruxes are optimized simultaneously. Firstly, the system model, task model, and path model are established, respectively. Then, a task scheduling algorithm based on enhanced HEFT, a heuristic MAPF algorithm and a TS- MAPF algorithm are proposed to solve this combinatorial optimization problem. In EHEFT, a novel rank priority rule is used to determine task sequencing and task allocation. In MAPF algorithm, a CBS algorithm with priority rules is designed for path search. Subsequently, the TS-MAPF algorithm which combines EHEFT and MAPF is proposed. Finally, the proposed algorithms are tested separately against relevant typical algorithms at different scales. The experimental results indicate that the proposed algorithms exhibited good performance.

Multi-agent path planning based on improved double DQN

Multi-agent path planning based on improved double DQN

Multi-Agent Dynamic Scheduling With a Posteriori Path Tracking and Collision Avoidance Using Model Predictive Control

This research work investigates a coordinated multi-agent path planning and tracking method. The solution of a pre-processed dynamic scheduling problem performs target assignment and provides optimal starting times and paths for each agent. Afterwards, a linear model predictive controller ensures robust and fast path tracking while preventing agents from collisions. This task is formulated as a discretized quadratic programming (QP) problem and is solved using an in-house developed semi-smooth Newton method. Numerical experiments have demonstrated the efficiency of the approach.

A Load-Balanced and Energy-Efficient Navigation Scheme for UAV-Mounted Mobile Edge Computing

With the development of Unmanned Aerial Vehicles (UAVs) in recent years, UAV-mounted Mobile Edge Computing (MEC) systems are widely used for broadband connectivity and emergency communications in areas without internet. However, the onboard energy of UAVs is limited, and how to provide long-term computing services for ground User Equipments (UEs) still faces significant challenges. In this paper, UAVs are used as mobile base stations to offer MEC services for UEs. Unlike existing solutions that mainly solve UAV path planning from convex optimization methods, we propose a Multi-Agent Path planning (MAP) scheme based on deep reinforcement learning. We aim to minimize the total energy consumption of UAVs while completing the offloading tasks of UEs. At the same time, fairness is taken into account to ensure UE offloading balance and UAV load balance. To this end, we establish the optimization problem and design the state space, action space, and reward function for modeling each UAV through Deep Neural Networks (DNNs). Extensive simulation experiments show that the proposed scheme outperforms other benchmark schemes.

Synthesizing Decentralized Controllers With Graph Neural Networks and Imitation Learning

Dynamical systems consisting of a set of autonomous agents face the challenge of having to accomplish a global task, relying only on local information. While centralized controllers are readily available, they face limitations in terms of scalability and implementation, as they do not respect the distributed information structure imposed by the network system of agents. Given the difficulties in finding optimal decentralized controllers, we propose a novel framework using graph neural networks (GNNs) to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">learn</i> these controllers. GNNs are well-suited for the task since they are naturally distributed architectures and exhibit good scalability and transferability properties. We show that GNNs learn appropriate decentralized controllers by means of imitation learning, leverage their permutation invariance properties to successfully scale to larger teams and transfer to unseen scenarios at deployment time. The problems of flocking and multi-agent path planning are explored to illustrate the potential of GNNs in learning decentralized controllers.

Message-Aware Graph Attention Networks for Large-Scale Multi-Robot Path Planning

The domains of transport and logistics are increasingly relying on autonomous mobile robots for the handling and distribution of passengers or resources. At large system scales, finding decentralized path planning and coordination solutions is key to efficient system performance. Recently, Graph Neural Networks (GNNs) have become popular due to their ability to learn communication policies in decentralized multi-agent systems. Yet, vanilla GNNs rely on simplistic message aggregation mechanisms that prevent agents from prioritizing important information. To tackle this challenge, in this letter, we extend our previous work that utilizes GNNs in multi-agent path planning by incorporating a novel mechanism to allow for message-dependent attention. Our Message-Aware Graph Attention neTwork (MAGAT) is based on a key-query-like mechanism that determines the relative importance of features in the messages received from various neighboring robots. We show that MAGAT is able to achieve a performance close to that of a coupled centralized expert algorithm. Further, ablation studies and comparisons to several benchmark models show that our attention mechanism is very effective across different robot densities and performs stably in different constraints in communication bandwidth. Experiments demonstrate that our model is able to generalize well in previously unseen problem instances, and that it achieves a 47% improvement over the benchmark success rate, even in very large-scale instances that are ×100 larger than the training instances.

Improving the On-Vehicle Experience of Passengers Through SC-M*: A Scalable Multi-Passenger Multi-Criteria Mobility Planner

The rapid growth in urban population poses significant challenges to moving city dwellers in a fast and convenient manner. This paper contributes to solving the challenges from the viewpoint of passengers by improving their on-vehicle experience. Specifically, we focus on the problem: Given an urban public transit network and a number of passengers, with some of them controllable and the rest uncontrollable, how can we plan for the controllable passengers to improve their experience in terms of their service preference? We formalize this problem as a multi-agent path planning (MAPP) problem with soft collisions, where multiple controllable passengers are allowed to share on-vehicle service resources with one another under certain constraints. We then propose a customized version of the SC-M* algorithm to efficiently solve the MAPP task for bus transit system in complex urban environments, where we have a large passenger size and multiple types of passengers requesting various types of service resources. We demonstrate the use of SC-M* in a case study of the bus transit system in Porto, Portugal. In the case study, we implement a data-driven on-vehicle experience simulator for the bus transit system, which simulates the passenger behaviors and on-vehicle resource dynamics, and evaluate the SC-M* on it. The experimental results show the advantages of the SC-M* in terms of path cost, collision-free constraint, and the scalability in run time and success rate.

SC-M*: A Multi-Agent Path Planning Algorithm with Soft-Collision Constraint on Allocation of Common Resources

Multi-agent path planning (MAPP) is increasingly being used to address resource allocation problems in highly dynamic, distributed environments that involve autonomous agents. Example domains include surveillance automation, traffic control and others. Most MAPP approaches assume hard collisions, e.g., agents cannot share resources, or co-exist at the same node or edge. This assumption unnecessarily restricts the solution space and does not apply to many real-world scenarios. To mitigate this limitation, this paper introduces a more general class of MAPP problems—MAPP in a soft-collision context. In soft-collision MAPP problems, agents can share resources or co-exist in the same location at the expense of reducing the quality of the solution. Hard constraints can still be modeled by imposing a very high cost for sharing. This paper motivates and defines the soft-collision MAPP problem, and generalizes the widely-used M* MAPP algorithm to support the concept of soft-collisions. Soft-collision M* (SC-M*) extends M* by changing the definition of a collision, so paths with collisions that have a quality penalty below a given threshold are acceptable. For each candidate path, SC-M* keeps track of the reduction in satisfaction level of each agent using a collision score, and it places agents whose collision scores exceed its threshold into a soft-collision set for reducing the score. Our evaluation shows that SC-M* is more flexible and more scalable than M*. It can also handle complex environments that include agents requesting different types of resources. Furthermore, we show the benefits of SC-M* compared with several baseline algorithms in terms of path cost, success rate and run time.