Large-scale Multi-robot Systems Research Articles

Graph Soft Actor-Critic Reinforcement Learning for Large-Scale Distributed Multirobot Coordination.

Learning distributed cooperative policies for large-scale multirobot systems remains a challenging task in the multiagent reinforcement learning (MARL) context. In this work, we model the interactions among the robots as a graph and propose a novel off-policy actor-critic MARL algorithm to train distributed coordination policies on the graph by leveraging the ability of information extraction of graph neural networks (GNNs). First, a new type of Gaussian policy parameterized by the GNNs is designed for distributed decision-making in continuous action spaces. Second, a scalable centralized value function network is designed based on a novel GNN-based value function decomposition technique. Then, based on the designed actor and the critic networks, a GNN-based MARL algorithm named graph soft actor-critic (G-SAC) is proposed and utilized to train the distributed policies in an effective and centralized fashion. Finally, two custom multirobot coordination environments are built, under which the simulation results are performed to empirically demonstrate both the sample efficiency and the scalability of G-SAC as well as the strong zero-shot generalization ability of the trained policy in large-scale multirobot coordination problems.

A novel cooperative path planning method based on UCR-FCE and behavior regulation for large-scale multi-robot system

A novel cooperative path planning method based on UCR-FCE and behavior regulation for large-scale multi-robot system

Distributed control for geometric pattern formation of large-scale multirobot systems.

Introduction: Geometric pattern formation is crucial in many tasks involving large-scale multi-agent systems. Examples include mobile agents performing surveillance, swarms of drones or robots, and smart transportation systems. Currently, most control strategies proposed to achieve pattern formation in network systems either show good performance but require expensive sensors and communication devices, or have lesser sensor requirements but behave more poorly. Methods and result: In this paper, we provide a distributed displacement-based control law that allows large groups of agents to achieve triangular and square lattices, with low sensor requirements and without needing communication between the agents. Also, a simple, yet powerful, adaptation law is proposed to automatically tune the control gains in order to reduce the design effort, while improving robustness and flexibility. Results: We show the validity and robustness of our approach via numerical simulations and experiments, comparing it, where possible, with other approaches from the existing literature.

Intelligent Planning for Large-Scale Multi-Robot Coordination

Robots will play a crucial role in the future and need to work as a team in increasingly more complex applications. Advances in robotics have laid the hardware foundations for building large-scale multi-robot systems. But how to coordinate robots intelligently is a difficult problem. We believe that graph-search-based planning can systematically exploit the combinatorial structure of multi-robot coordination problems and efficiently generate solutions with rigorous guarantees on correctness, completeness, and solution quality. We started with one problem that is central to many multi-robot applications. Multi-Agent Path Finding (MAPF) is an NP-hard problem of planning collision-free paths for a team of agents while minimizing their travel times. We addressed the MAPF problem from both (1) a theoretical perspective by developing efficient algorithms to solve large MAPF instances with completeness and optimality guarantees via a variety of AI and optimization technologies, such as constraint reasoning, heuristic search, stochastic local search, and machine learning, and (2) an applicational perspective by developing algorithmic techniques for integrating MAPF with task planning and execution for various multi-robot systems, such as mobile robot coordination, traffic management, drone swarm control, multi-arm assembly, and character control in video games. This paper is part of the AAAI-23 New Faculty Highlights.

Research on Multi-Objective Multi-Robot Task Allocation by Lin–Kernighan–Helsgaun Guided Evolutionary Algorithms

Multi-robot task allocation (MRTA) and route planning are crucial for a large-scale multi-robot system. In this paper, the problem is formulated to minimize the total energy consumption and overall task completion time simultaneously, with some constraints taken into consideration. To represent a solution, a novel one-chromosome representation technique is proposed, which eases the consequent genetic operations and the construction of the cost matrix. Lin–Kernighan–Helsgaun (LKH), a highly efficient sub-tour planner, is employed to generate prophet generation beforehand as well as guide the evolutionary direction during the proceeding of multi-objective evolutionary algorithms, aiming to promote convergence of the Pareto front. Numerical experiments on the benchmark show the LKH guidance mechanism is effective for two famous multi-objective evolutionary algorithms, namely multi-objective evolutionary algorithm based on decomposition (MOEA/D) and non-dominated sorting genetic algorithm (NSGA), of which LKH-guided NSGA exhibits the best performance on three predefined indicators, namely C-metric, HV, and Spacing, respectively. The generalization experiment on a multiple depots MRTA problem with constraints further demonstrates the effectiveness of the proposed approach for practical decision making.

Frontier-led swarming: Robust multi-robot coverage of unknown environments

This paper proposes a novel swarm-based control algorithm for exploration and coverage of unknown environments, while maintaining a formation that permits short-range communication. The algorithm combines two elements: swarm rules for maintaining a close-knit formation and frontier search for driving exploration and coverage. Inspired by natural systems in which large numbers of simple agents (e.g., schooling fish, flocking birds, swarming insects) perform complicated collective behaviors for efficiency and safety, the first element uses three simple rules to maintain a swarm formation. The second element provides a means to select promising regions to explore (and cover) by minimising a cost function involving robots’ relative distance to frontier cells and the frontier’s size. We tested the performance of our approach on heterogeneous and homogeneous groups of mobile robots in different environments. We measure both coverage performance and swarm formation statistics as indicators of the robots’ ability to explore effectively while maintaining a formation conducive to short-range communication. Through a series of comparison experiments, we demonstrate that our proposed strategy has superior performance to recently presented map coverage methodologies and conventional swarming methods. • A novel, distributed algorithm leveraging the strengths of frontier-based area coverage and swarm behaviors is proposed for large-scale multi-robot systems. • Each robot can directly exchange the individual coverage and obstacle information with a set of neighbors without passing through a central station. • An improved artificial potential field obstacle avoidance method is designed based on Lidar information. • System robustness and fault tolerance of the proposed strategy is validated and analysed through a series of real experiments. • This paper demonstrates our space coverage controller’s relative merits with respect to the frontierbased map coverage strategy without swarming, and the three different map coverage control laws through a rigorous comparative study. • Algorithms are examined on both homogeneous and heterogeneous robot teams within a shortranged communication. Hence, the proposed system can normally work in unknown and uncertain environments.

Graph-Based Multi-Robot Path Finding and Planning

Purpose of Review Planning collision-free paths for multiple robots is important for real-world multi-robot systems and has been studied as an optimization problem on graphs, called Multi-Agent Path Finding (MAPF). This review surveys different categories of classic and state-of-the-art MAPF algorithms and different research attempts to tackle the challenges of generalizing MAPF techniques to real-world scenarios. Recent Findings Solving MAPF problems optimally is computationally challenging. Recent advances have resulted in MAPF algorithms that can compute collision-free paths for hundreds of robots and thousands of navigation tasks in seconds of runtime. Many variants of MAPF have been formalized to adapt MAPF techniques to different real-world requirements, such as considerations of robot kinematics, online optimization for real-time systems, and the integration of task assignment and path planning. Summary Algorithmic techniques for MAPF problems have addressed important aspects of several multi-robot applications, including automated warehouse fulfillment and sortation, automated train scheduling, and navigation of non-holonomic robots and quadcopters. This showcases their potential for real-world applications of large-scale multi-robot systems.

An intern-sufficient cloud for large-scale multi-robot systems and its application in multitarget navigation

Due to generally limited computing capability of an individual robot, cloud-based robotic systems are increasingly used. However, applications in large-scale multi-robot systems will be hindered by communication congestion and consequent lack of computing resources. In this study, an intern-sufficient cloud is investigated to alleviate the burden of communication and thus support more robots. At the same time, it enables heterogeneously idle computing resources of robots inside the system to be shared on demand, instead of relying on cloud servers and communication infrastructures, to make the scope of application wider. To this end, a hierarchical communication mechanism and a resource schedule algorithm are proposed. In the mechanism, the transmission power, signal-to-noise ratio, available bandwidth, and other relevant features are taken into account to estimate link quality for data transmission. Then, the constrained communication conditions and heterogeneous computing resources are balanced by the resource scheduling algorithm, so that the most appropriate computing resources of the robots are contributed to the mobile cloud. Furthermore, a multitarget navigation task is applied on the cloud to validate the work. Thereby, simulations and experiments are performed. The results show that the proposed intern-sufficient cloud can provide stable resources of communication and computation for a multi-robot system with 20 physical robots while achieving more effective multitarget navigation.

Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios

Developing a safe and efficient collision-avoidance policy for multiple robots is challenging in the decentralized scenarios where each robot generates its paths with limited observation of other robots’ states and intentions. Prior distributed multi-robot collision-avoidance systems often require frequent inter-robot communication or agent-level features to plan a local collision-free action, which is not robust and computationally prohibitive. In addition, the performance of these methods is not comparable with their centralized counterparts in practice. In this article, we present a decentralized sensor-level collision-avoidance policy for multi-robot systems, which shows promising results in practical applications. In particular, our policy directly maps raw sensor measurements to an agent’s steering commands in terms of the movement velocity. As a first step toward reducing the performance gap between decentralized and centralized methods, we present a multi-scenario multi-stage training framework to learn an optimal policy. The policy is trained over a large number of robots in rich, complex environments simultaneously using a policy-gradient-based reinforcement-learning algorithm. The learning algorithm is also integrated into a hybrid control framework to further improve the policy’s robustness and effectiveness. We validate the learned sensor-level collision-3avoidance policy in a variety of simulated and real-world scenarios with thorough performance evaluations for large-scale multi-robot systems. The generalization of the learned policy is verified in a set of unseen scenarios including the navigation of a group of heterogeneous robots and a large-scale scenario with 100 robots. Although the policy is trained using simulation data only, we have successfully deployed it on physical robots with shapes and dynamics characteristics that are different from the simulated agents, in order to demonstrate the controller’s robustness against the simulation-to-real modeling error. Finally, we show that the collision-avoidance policy learned from multi-robot navigation tasks provides an excellent solution for safe and effective autonomous navigation for a single robot working in a dense real human crowd. Our learned policy enables a robot to make effective progress in a crowd without getting stuck. More importantly, the policy has been successfully deployed on different types of physical robot platforms without tedious parameter tuning. Videos are available at https://sites.google.com/view/hybridmrca.

STyLuS*: A Temporal Logic Optimal Control Synthesis Algorithm for Large-Scale Multi-Robot Systems

This article proposes a new highly scalable and asymptotically optimal control synthesis algorithm from linear temporal logic specifications, called for large-Scale optimal Temporal Logic Synthesis, that is designed to solve complex temporal planning problems in large-scale multi-robot systems. Existing planning approaches with temporal logic specifications rely on graph search techniques applied to a product automaton constructed among the robots. In our previous work, we have proposed a more tractable sampling-based algorithm that builds incrementally trees that approximate the state space and transitions of the synchronous product automaton and does not require sophisticated graph search techniques. Here, we extend our previous work by introducing bias in the sampling process that is guided by transitions in the Büchi automaton that belong to the shortest path to the accepting states. This allows us to synthesize optimal motion plans from product automata with hundreds of orders of magnitude more states than those that existing optimal control synthesis methods or off-the-shelf model checkers can manipulate. We show that is probabilistically complete and asymptotically optimal and has exponential convergence rate. This is the first time that convergence rate results are provided for sampling-based optimal control synthesis methods. We provide simulation results that show that can synthesize optimal motion plans for very large multi-robot systems, which is impossible using state-of-the-art methods.

Generic, scalable and decentralized fault detection for robot swarms.

Robot swarms are large-scale multirobot systems with decentralized control which means that each robot acts based only on local perception and on local coordination with neighboring robots. The decentralized approach to control confers number of potential benefits. In particular, inherent scalability and robustness are often highlighted as key distinguishing features of robot swarms compared with systems that rely on traditional approaches to multirobot coordination. It has, however, been shown that swarm robotics systems are not always fault tolerant. To realize the robustness potential of robot swarms, it is thus essential to give systems the capacity to actively detect and accommodate faults. In this paper, we present a generic fault-detection system for robot swarms. We show how robots with limited and imperfect sensing capabilities are able to observe and classify the behavior of one another. In order to achieve this, the underlying classifier is an immune system-inspired algorithm that learns to distinguish between normal behavior and abnormal behavior online. Through a series of experiments, we systematically assess the performance of our approach in a detailed simulation environment. In particular, we analyze our system’s capacity to correctly detect robots with faults, false positive rates, performance in a foraging task in which each robot exhibits a composite behavior, and performance under perturbations of the task environment. Results show that our generic fault-detection system is robust, that it is able to detect faults in a timely manner, and that it achieves a low false positive rate. The developed fault-detection system has the potential to enable long-term autonomy for robust multirobot systems, thus increasing the usefulness of robots for a diverse repertoire of upcoming applications in the area of distributed intelligent automation.

Multi-robot Task Allocation Based on Ant Colony Algorithm

With the development of information technology, the capability and application fields of robots become wider. In order to complete a complex task, the cooperation and coordination of robots are needed to be adopted. As the main problem of the multi-robot systems, multi-robot task allocation (MRTA) reflects the organization form and operation mechanism of the robots system. The cooperation and allocation for large-scale multi-robot system in loosely environment is the hot issue. As a popular bionic intelligence method, ant colony algorithm is powerful for solving MRTA. By analyzing the existing algorithms, this paper proposed a new solution for MRTA based on ant colony algorithm, built up the model of the algorithm and described the robots coalition, high-level task allocation process in details. Finally, we realized the simulation of ant colony algorithm based on MATLAB, and then compared the robustness and the best incomes of the four algorithms. The simulation results show that, ant colony algorithm is a high degree of ability and stability for solving MRTA.

Software Agent Systems for Improving Performance of Multi-Robot Groups

Agent paradigm is becoming more and more popular in design of complex multi-robot systems. Similarities between features attributed to robots and agents lead to defining an agent as a single robot control program or even calling a robot an “embodied agents”. This approach can cause serious issues concerning scalability, extensibility and performance of large-scale multi-robot systems. In this paper a different approach to design and implementation of software systems managing mobile robots is presented. It is based on the logical separation of a hardware robot and a software agent. The paper focuses on improvements in system performance achieved by using the approach. Three exemplary applications are described and experimental results are provided.

Experimental Testbed for Large Multirobot Teams

Experimental validation is particularly important in multi-robot systems research. The differences between models and real-world conditions that may not be apparent in single robot experiments are amplified because of the large number of robots, interactions between robots, and the effects of asynchronous and distributed control, sensing, and actuation. Over the last two years, we have developed an experimental testbed to support research in multirobot systems with the goal of making it easy for users to model, design, benchmark, and validate algorithms. In this article, we describe our approach to the design of a large-scale multirobot system for the experimental verification and validation of a variety of distributed robotic applications in an indoor environment.