Multi-robot Coordination Research Articles

Toward a Generic Framework for Mission Planning and Execution with a Heterogeneous Multi-Robot System

This paper presents a comprehensive framework for mission planning and execution with a heterogeneous multi-robot system, specifically designed to coordinate unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) in dynamic and unstructured environments. The proposed architecture evaluates the mission requirements, allocates tasks, and optimizes resource usage based on the capabilities of the available robots. It then executes the mission utilizing a decentralized control strategy that enables the robots to adapt to environmental changes and maintain formation stability in both 2D and 3D spaces. The framework’s architecture supports loose coupling between its components, enhancing system scalability and maintainability. Key features include a robust task allocation algorithm, and a dynamic formation control mechanism, using a ROS 2 communication protocol that ensures reliable information exchange among robots. The effectiveness of this framework is demonstrated through a case study involving coordinated exploration and data collection tasks, showcasing its ability to manage missions while optimizing robot collaboration. This work advances the field of heterogeneous robotic systems by providing a scalable and adaptable solution for multi-robot coordination in challenging environments.

An improved whale optimization algorithm for multi-robot path planning

Multi-robot path planning is challenging and has increasingly attracted attention with its widespread applications. This article proposes an improved Whale Optimization Algorithm (WOA) with Refracted opposition-based learning and Quantum behaviour (RQWOA). The algorithm is able to plan smooth and collisionless paths for robots combining cubic spline interpolation and multi-robot coordination. A quantum behavioural mechanism is used to coordinate the evolution of the whale population during the variable phase to increase the population quality and balance the exploration and exploitation capabilities of the WOA. Simultaneously, refracted opposition-based learning is introduced to improve the algorithm's optimization accuracy and convergence speed. The RQWOA was compared with seven efficient algorithms in experiments on classical test functions and multi-robot path planning cases. The results of these methods were tested statistically. The experimental results indicate that the RQWOA has superior solution accuracy. The RQWOA is highly competitive in terms of pathlength and stability in solving multi-robot path planning problems.

Multi-Robot Coordination and Layout Design for Automated Warehousing (Extended Abstract)

With the rapid progress in Multi-Agent Path Finding (MAPF), researchers have studied how MAPF algorithms can be deployed to coordinate hundreds of robots in large automated warehouses. While most works try to improve the throughput of such warehouses by developing better MAPF algorithms, we focus on improving the throughput by optimizing the warehouse layout. We show that, even with state-of-the-art MAPF algorithms, commonly used human-designed layouts can lead to congestion for warehouses with large numbers of robots and thus have limited scalability. We extend existing automatic scenario generation methods to optimize warehouse layouts. Results show that our optimized warehouse layouts (1) reduce traffic congestion and thus improve throughput, (2) improve the scalability of the automated warehouses by doubling the number of robots in some cases, and (3) are capable of generating layouts with user-specified diversity measures. We include the source code at: https://github.com/lunjohnzhang/warehouse_env_gen_public

L-NORM: Learning and Network Orchestration at the Edge for Robot Connectivity and Mobility in Factory Floor Environments

Robotic factory floors will revolutionize the future of manufacturing and the service industry by automating tasks. However, to fully supplement human effort, these robots will need low-latency, reliable connectivity throughout the work zone through links established by wireless access points (APs). This will allow the robot to assuredly respond to programming directives that rely on the real-time relaying of robot-generated sensor data to the Mobile Edge Computing (MEC) server. In this paper, we propose L-NORM, a multi-AP and multi-robot coordination framework, as a multi-tiered solution for such autonomous edge networks. First, multi-robot motion planning through reinforcement learning occurs at the MEC, using as input multi-modal robot sensor data. Second, multi-AP resource orchestration is performed using another reinforcement learning-based method that maps a subset of available APs to each robot toward meeting their sensor data delivery requirements. Furthermore, we suggest diversity combination of uplink channels with the 802.11ax scheduled access mode that will (i) support high reliability of multi-robot uplink sensor packets and (ii) enable multi-AP coordination, for optimized resource utilization. Through extensive simulation studies, we show that the probability of robot deviation to remain within 0.5 m from its optimal path, is 19% more in L-NORM compared to classical 802.11ax based edge network solution, considering <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 1 MB of sensor data per robot.

Ordering-flexible multi-robot coordination for moving target convoying using long-term task execution

In this paper, we propose a cooperative long-term task execution (LTTE) algorithm for protecting a moving target into the interior of an ordering-flexible convex hull by a team of robots resiliently in the changing environments. Particularly, by designing target-approaching and sensing-neighbor collision-free subtasks, and incorporating these subtasks into the constraints rather than the traditional cost function in an online constraint-based optimization framework, the proposed LTTE can systematically guarantee long-term target convoying under changing environments in the n-dimensional Euclidean space. Then, the introduction of slack variables allows for the constraint violation of different subtasks; i.e., the attraction from target-approaching constraints and the repulsion from time-varying collision-avoidance constraints, which results in the desired formation with arbitrary spatial ordering sequences. Rigorous analysis is provided to guarantee asymptotical convergence with challenging nonlinear couplings induced by time-varying collision-free constraints. Finally, 2D experiments using three autonomous mobile robots (AMRs) are conducted to validate the effectiveness of the proposed algorithm, and 3D simulations tackling changing environmental elements, such as different initial positions, some robots suddenly breakdown and static obstacles are presented to demonstrate the multi-dimensional adaptability, robustness and the ability of obstacle avoidance of the proposed method.

A Survey of Machine Learning Approaches for Mobile Robot Control

Machine learning (ML) is a branch of artificial intelligence that has been developing at a dynamic pace in recent years. ML is also linked with Big Data, which are huge datasets that need special tools and approaches to process them. ML algorithms make use of data to learn how to perform specific tasks or make appropriate decisions. This paper presents a comprehensive survey of recent ML approaches that have been applied to the task of mobile robot control, and they are divided into the following: supervised learning, unsupervised learning, and reinforcement learning. The distinction of ML methods applied to wheeled mobile robots and to walking robots is also presented in the paper. The strengths and weaknesses of the compared methods are formulated, and future prospects are proposed. The results of the carried out literature review enable one to state the ML methods that have been applied to different tasks, such as the following: position estimation, environment mapping, SLAM, terrain classification, obstacle avoidance, path following, learning to walk, and multirobot coordination. The survey allowed us to associate the most commonly used ML algorithms with mobile robotic tasks. There still exist many open questions and challenges such as the following: complex ML algorithms and limited computational resources on board a mobile robot; decision making and motion control in real time; the adaptability of the algorithms to changing environments; the acquisition of large volumes of valuable data; and the assurance of safety and reliability of a robot’s operation. The development of ML algorithms for nature-inspired walking robots also seems to be a challenging research issue as there exists a very limited amount of such solutions in the recent literature.

Graph Soft Actor-Critic Reinforcement Learning for Large-Scale Distributed Multirobot Coordination.

Learning distributed cooperative policies for large-scale multirobot systems remains a challenging task in the multiagent reinforcement learning (MARL) context. In this work, we model the interactions among the robots as a graph and propose a novel off-policy actor-critic MARL algorithm to train distributed coordination policies on the graph by leveraging the ability of information extraction of graph neural networks (GNNs). First, a new type of Gaussian policy parameterized by the GNNs is designed for distributed decision-making in continuous action spaces. Second, a scalable centralized value function network is designed based on a novel GNN-based value function decomposition technique. Then, based on the designed actor and the critic networks, a GNN-based MARL algorithm named graph soft actor-critic (G-SAC) is proposed and utilized to train the distributed policies in an effective and centralized fashion. Finally, two custom multirobot coordination environments are built, under which the simulation results are performed to empirically demonstrate both the sample efficiency and the scalability of G-SAC as well as the strong zero-shot generalization ability of the trained policy in large-scale multirobot coordination problems.

FSOC: Flexible Subgrouping and Ordering of Multi-Robot Coordination for Convoying Multiple Scattered Targets

FSOC: Flexible Subgrouping and Ordering of Multi-Robot Coordination for Convoying Multiple Scattered Targets

Heterogeneous multi-agent task allocation based on graph neural network ant colony optimization algorithms

Heterogeneous multi-agent task allocation is a key optimization problem widely used in fields such as drone swarms and multi-robot coordination. This paper proposes a new paradigm that innovatively combines graph neural networks and ant colony optimization algorithms to solve the assignment problem of heterogeneous multi-agents. The paper introduces an innovative Graph-based Heterogeneous Neural Network Ant Colony Optimization (GHNN-ACO) algorithm for heterogeneous multi-agent scenarios. The multi-agent system is composed of unmanned aerial vehicles, unmanned ships, and unmanned vehicles that work together to effectively respond to emergencies. This method uses graph neural networks to learn the relationship between tasks and agents, forming a graph representation, which is then integrated into ant colony optimization algorithms to guide the search process of ants. Firstly, the algorithm in this paper constructs heterogeneous graph data containing different types of agents and their relationships and uses the algorithm to classify and predict linkages for agent nodes. Secondly, the GHNN-ACO algorithm performs effectively in heterogeneous multi-agent scenarios, providing an effective solution for node classification and link prediction tasks in intelligent agent systems. Thirdly, the algorithm achieves an accuracy rate of 95.31% in assigning multiple tasks to multiple agents. It holds potential application prospects in emergency response and provides a new idea for multi-agent system cooperation.

Autonomous Navigation System for Multi-Quadrotor Coordination and Human Detection in Search and Rescue

There are many methodologies assisting in the detection and tracking of trapped victims in the context of disaster management. Disaster management in the aftermath of such sudden occurrences requires preparedness in terms of technology, availability, accessibility, perception, training, evaluation, and deployability. This can be achieved through intensive test, evaluation and comparison of different techniques that are alternative to each other, eventually covering each module of the technology used for the search and rescue operation. Intensive research and development by academia and industry have led to an increased robustness of deep learning techniques such as the use of convolutional neural networks, which has resulted in increased reliance of first responders on the unmanned aerial vehicle (UAV) technology equipped with state-of-the-art computers to process real-time sensory information from cameras and other sensors in quest of possibility of life. In this paper, we propose a method to implement simulated detection of life in the sudden onset of disasters with the help of a deep learning model, and simultaneously implement multi-robot coordination between the vehicles with the use of a suitable region-partitioning technique to further expedite the operation. A simulated test platform was developed with parameters resembling real-life disaster environments using the same sensors.

Gradient Monitored Reinforcement Learning.

This article presents a novel neural network training approach for faster convergence and better generalization abilities in deep reinforcement learning (RL). Particularly, we focus on the enhancement of training and evaluation performance in RL algorithms by systematically reducing gradient's variance and, thereby, providing a more targeted learning process. The proposed method, which we term gradient monitoring (GM), is a method to steer the learning in the weight parameters of a neural network based on the dynamic development and feedback from the training process itself. We propose different variants of the GM method that we prove to increase the underlying performance of the model. One of the proposed variants, momentum with GM (M-WGM), allows for a continuous adjustment of the quantum of backpropagated gradients in the network based on certain learning parameters. We further enhance the method with the adaptive M-WGM (AM-WGM) method, which allows for automatic adjustment between focused learning of certain weights versus more dispersed learning depending on the feedback from the rewards collected. As a by-product, it also allows for automatic derivation of the required deep network sizes during training as the method automatically freezes trained weights. The method is applied to two discrete (real-world multirobot coordination problems and Atari games) and one continuous control task (MuJoCo) using advantage actor-critic (A2C) and proximal policy optimization (PPO), respectively. The results obtained particularly underline the applicability and performance improvements of the methods in terms of generalization capability.

Intelligent Planning for Large-Scale Multi-Robot Coordination

Robots will play a crucial role in the future and need to work as a team in increasingly more complex applications. Advances in robotics have laid the hardware foundations for building large-scale multi-robot systems. But how to coordinate robots intelligently is a difficult problem. We believe that graph-search-based planning can systematically exploit the combinatorial structure of multi-robot coordination problems and efficiently generate solutions with rigorous guarantees on correctness, completeness, and solution quality. We started with one problem that is central to many multi-robot applications. Multi-Agent Path Finding (MAPF) is an NP-hard problem of planning collision-free paths for a team of agents while minimizing their travel times. We addressed the MAPF problem from both (1) a theoretical perspective by developing efficient algorithms to solve large MAPF instances with completeness and optimality guarantees via a variety of AI and optimization technologies, such as constraint reasoning, heuristic search, stochastic local search, and machine learning, and (2) an applicational perspective by developing algorithmic techniques for integrating MAPF with task planning and execution for various multi-robot systems, such as mobile robot coordination, traffic management, drone swarm control, multi-arm assembly, and character control in video games. This paper is part of the AAAI-23 New Faculty Highlights.

LaCAM: Search-Based Algorithm for Quick Multi-Agent Pathfinding

We propose a novel complete algorithm for multi-agent pathfinding (MAPF) called lazy constraints addition search for MAPF (LaCAM). MAPF is a problem of finding collision-free paths for multiple agents on graphs and is the foundation of multi-robot coordination. LaCAM uses a two-level search to find solutions quickly, even with hundreds of agents or more. At the low-level, it searches constraints about agents' locations. At the high-level, it searches a sequence of all agents' locations, following the constraints specified by the low-level. Our exhaustive experiments reveal that LaCAM is comparable to or outperforms state-of-the-art sub-optimal MAPF algorithms in a variety of scenarios, regarding success rate, planning time, and solution quality of sum-of-costs.

A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers

Active simultaneous localization and mapping (SLAM) is the problem of planning and controlling the motion of a robot to build the most accurate and complete model of the surrounding environment. Since the first foundational work in active perception appeared, more than three decades ago, this field has received increasing attention across different scientific communities. This has brought about many different approaches and formulations, and makes a review of the current trends necessary and extremely valuable for both new and experienced researchers. In this article, we survey the state of the art in active SLAM and take an in-depth look at the open challenges that still require attention to meet the needs of modern applications. After providing a historical perspective, we present a unified problem formulation and review the well-established modular solution scheme, which decouples the problem into three stages that identify, select, and execute potential navigation actions. We then analyze alternative approaches, including belief-space planning and deep reinforcement learning techniques, and review related work on multirobot coordination. This article concludes with a discussion of new research directions, addressing reproducible research, active spatial perception, and practical applications, among other topics.

Quantum Multiagent Actor–Critic Neural Networks for Internet-Connected Multirobot Coordination in Smart Factory Management

As one of the latest fields of interest in both academia and industry, quantum computing has garnered significant attention. Among various topics in quantum computing, variational quantum circuits (VQC) have been noticed for their ability to carry out quantum deep reinforcement learning (QRL). This paper verifies the potential of QRL, which will be further realized by implementing quantum multi-agent reinforcement learning (QMARL) from QRL, especially for Internet-connected autonomous multi-robot control and coordination in smart factory applications. However, the extension is not straightforward due to the non-stationarity of classical MARL. To cope with this, the centralized training and decentralized execution (CTDE) QMARL framework is proposed under the Internet connection. A smart factory environment with the Internet of Things (IoT)-based multiple agents is used to show the efficacy of the proposed algorithm. The simulation corroborates that the proposed QMARL-based autonomous multi-robot control and coordination performs better than the other frameworks.

Distributed Online Aggregative Optimization for Dynamic Multirobot Coordination

This paper focuses on an online version of the emerging distributed constrained aggregative optimization framework, which is particularly suited for applications arising in cooperative robotics. Agents in a network want to minimize the sum of local cost functions, each one depending both on a local optimization variable, subject to a local constraint, and on an aggregated version of all the variables (e.g., the mean). We focus on a challenging online scenario in which the cost, the aggregation functions and the constraints can all change over time, thus enlarging the class of captured applications. Inspired by an existing scheme, we propose a distributed algorithm with constant step size, named Projected Aggregative Tracking, to solve the online optimization problem. We prove that the dynamic regret is bounded by a constant term and a term related to time variations. Moreover, in the static case (i.e., with constant cost and constraints), the solution estimates are proved to converge with a linear rate to the optimal solution. Finally, numerical examples show the efficacy of the proposed approach on a robotic surveillance scenario.

Navigation of a compartmentalized robot system

After making progression in developing the fundamental problems related to single-robot control, many researchers swerved and diverged their focus to studying multi-robot coordination. This research aims to take the motion planning and control (MPC) problem of a multi-robot system into a new space by considering a compartmentalized robot. An efficient variant of globally rigid formation, in which multiple car-like units are adjoint and move in parallel without collisions. The motion is governed by one of the sub-units acting as a leader, while other units maintain the fixed distance amongst each other and the leader in a rigid formation. The minimum distance technique is an important input to facilitate collision avoidance, robot decision making, and robot navigation. In this study a novel method to analytically compute the minimum distance between the closest point on the line segments of rectangular protective region and the obstacle is presented. Utilizing the Lyapunov-based Control Scheme a set of autonomous controllers are designed. Computer simulations of the proposed Lyapunov-based controllers for the compartmentalized robot are presented in interesting scenarios to show the efficacy of the unique set of controllers. In these simulations, the compartmentalized robot shows strict maintenance of a rigid formation with efficient collision and obstacle avoidance. The results open up research in the design and implementation of controllers by considering multiple compartmentalized robots into swarm models, splitting and re-joining units, and applying rotational leadership ideas.

Online Submodular Coordination With Bounded Tracking Regret: Theory, Algorithm, and Applications to Multi-Robot Coordination

We enable efficient and effective coordination in unpredictable environments, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , in environments whose future evolution is unknown a priori and even adversarial. We are motivated by the future of autonomy that involves multiple robots coordinating in dynamic, unstructured, and adversarial environments to complete complex tasks such as target tracking, environmental mapping, and area monitoring. Such tasks are often modeled as submodular maximization coordination problems. We introduce the first submodular coordination algorithm with bounded <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tracking regret</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , with bounded suboptimality with respect to optimal time-varying actions that know the future a priori. The bound gracefully degrades with the environments' capacity to change adversarially. It also quantifies how often the robots must re-select actions to “learn” to coordinate as if they knew the future a priori. The algorithm requires the robots to select actions sequentially based on the actions selected by the previous robots in the sequence. Particularly, the algorithm generalizes the seminal Sequential Greedy algorithm by Fisher et al. to unpredictable environments, leveraging submodularity and algorithms for the problem of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tracking the best expert</i> . We validate our algorithm in simulated scenarios of target tracking.

Distributed Nonlinear Trajectory Optimization for Multi-Robot Motion Planning

This work presents a method for multi-robot coordination based on a novel distributed nonlinear model predictive control (NMPC) formulation for trajectory optimization and its modified version to mitigate the effects of packet losses and delays in the communication among the robots. Our algorithms consider that each robot is equipped with an onboard computation unit to solve a local control problem and communicate with neighboring autonomous robots via a wireless network. The difference between the two proposed methods is in the way the robots exchange information to coordinate. The information exchange can occur in a following: 1) synchronous or 2) asynchronous fashion. By relying on the theory of the nonconvex alternating direction method of multipliers (ADMM), we show that the proposed solutions converge to a (local) solution of the centralized problem. For both algorithms, the communication exchange preserves the safety of the robots; that is, collisions with neighboring autonomous robots are prevented. The proposed approaches can be applied to various multi-robot scenarios and robot models. In this work, we assess our methods, both in simulation and with experiments, for the coordination of a team of autonomous vehicles in the following: 1) an unsupervised intersection crossing and 2) the platooning scenarios.

Distributed Framework Matching

This article studies the problem of distributed framework matching (FM), which originates from the assignment task in multirobot coordination and the matching task in pattern recognition. The objective of distributed FM is to distributively seek a correspondence which minimizes some metrics describing the disagreement between two frameworks (i.e., graphs and their embeddings). In view of the type of the underlying graph in the framework, two formulations, undirected framework matching (UFM) and directed framework matching (DFM), and their convex relaxations, relaxed UFM (RUFM), and relaxed DFM (RDFM), are presented. UFM is converted into a graph matching (GM) problem with the adjacency matrix being replaced by a matrix constructed from the undirected framework under certain graphical conditions, and can be solved distributively. Sufficient conditions for the equivalence between UFM and RUFM, and the perturbation admitting exact recovery of correspondence are established. On the other hand, DFM embeds the configuration of the directed framework via another type of matrix, whose computation is distributed, and can deal with the case of two frameworks with different sizes of node sets. A distributed optimization algorithm for solving RDFM is proposed and its convergence results are established which allows DFM to be solved in a fully distributed manner. Simulation examples on both synthetic data and real world datasets demonstrate the applicability and efficacy of our theoretical results in formation control and object matching problems.