Heterogeneous Multi-robot System Research Articles

An Algorithm for Task Allocation and Planning for a Heterogeneous Multi-Robot System to Minimize the Last Task Completion Time.

This paper proposes an algorithm that provides operational strategies for multiple heterogeneous mobile robot systems utilized in many real-world applications, such as deliveries, surveillance, search and rescue, monitoring, and transportation. Specifically, the authors focus on developing an algorithm that solves a min–max multiple depot heterogeneous asymmetric traveling salesperson problem (MDHATSP). The algorithm is designed based on a primal–dual technique to operate given multiple heterogeneous robots located at distinctive depots by finding a tour for each robot such that all the given targets are visited by at least one robot while minimizing the last task completion time. Building on existing work, the newly developed algorithm can solve more generalized problems, including asymmetric cost problems with a min–max objective. Though producing optimal solutions requires high computational loads, the authors aim to find reasonable sub-optimal solutions within a short computation time. The algorithm was repeatedly tested in a simulation with varying problem sizes to verify its effectiveness. The computational results show that the algorithm can produce reliable solutions to apply in real-time operations within a reasonable time.

Complete coverage problem of multiple robots with different velocities

Complete coverage, which is integral to many robotic applications, aims to cover an area as quickly as possible. In such tasks, employing multiple robots can reduce the overall coverage time by appropriate task allocation. Several multi-robot coverage approaches divide the environment into balanced subareas and minimize the maximum subarea of all robots. However, balanced coverage in many situations, such as in the cases of robots with different velocities and heterogeneous multi-robot systems, may have inefficient results. This study addresses the unbalanced complete coverage problem of multiple robots with different velocities for a known environment. First, we propose a novel credit model to transform the unbalanced coverage problem into a set of single-objective optimization problems, which can find a combinational optimal solution by optimizing each separate objective function of the single-objective optimization problem to alleviate the computational complexity. Then, we propose a credit-based algorithm composed of a cyclic region growth algorithm and a region fine-tuning algorithm. The cyclic region growth algorithm finds an initial solution to the single-objective optimization problems set by a regional growth strategy with multiple restricts, whereas the region fine-tuning algorithm reallocates the tasks of the partitions with too many tasks to the partitions with too few tasks by constructing a search tree, thereby converging the initial solution to the optimal solution. Simulation results indicate that compared with conventional multi-robot complete coverage problem algorithms, the credit-based algorithm can obtain the optimal solution with the increased number of robots and enlarged size of the mission environment.

A Resilient and Energy-Aware Task Allocation Framework for Heterogeneous Multirobot Systems

In the context of heterogeneous multirobot teams deployed for executing multiple tasks, this article develops an energy-aware framework for allocating tasks to robots in an online fashion. With a primary focus on long-duration autonomy applications, we opt for a survivability-focused approach. Toward this end, the task prioritization and execution—through which the allocation of tasks to robots is effectively realized—are encoded as constraints within an optimization problem aimed at minimizing the energy consumed by the robots at each point in time. In this context, an allocation is interpreted as a prioritization of a task over all others by each of the robots. Furthermore, we present a novel framework to represent the heterogeneous capabilities of the robots, by distinguishing between the features available on the robots and the capabilities enabled by these features. By embedding these descriptions within the optimization problem, we make the framework resilient to situations, where environmental conditions make certain features unsuitable to support a capability and when component failures on the robots occur. We demonstrate the efficacy and resilience of the proposed approach in a variety of use-case scenarios, consisting of simulations and real robot experiments.

Coordination and path planning of a heterogeneous multi-robot system for sheet metal drilling

This paper presents the details of a sub-system developed to address coordination between a serial manipulator robot (machining) and SwarmItFIX robot (fixturing) for a sheet metal drilling process. A heterogeneous multi-robot coordination methodology that has already been demonstrated to be successful in a milling process has been further enhanced here to make it suitable for a drilling process. For the convergence of joint angles in the trajectory planning of the serial manipulator robot, an optimisation-based approach is proposed. The velocity of the tool center point (TCP) is considered to be constant throughout, as it improves the quality of the machining. The SwarmItFIX robot abides by a revised five-step locomotion strategy to traverse between any two support locations. A new time plan that ensures multi-robot coordination has also been proposed in this work. The proposed method has been tested with three different drilling patterns, and the results show that the proposed method computes the trajectory of the serial manipulator, support locations of the SwarmItFIX and locomotion sequence of the base agent accurately.

GRSTAPS: Graphically Recursive Simultaneous Task Allocation, Planning, and Scheduling

Effective deployment of multi-robot teams requires solving several interdependent problems at varying levels of abstraction. Specifically, heterogeneous multi-robot systems must answer four important questions: what (task planning), how (motion planning), who (task allocation), and when (scheduling). Although there are rich bodies of work dedicated to various combinations of these questions, a fully integrated treatment of all four questions lies beyond the scope of the current literature, which lacks even a formal description of the complete problem. In this article, we address this absence, first by formalizing this class of multi-robot problems under the banner Simultaneous Task Allocation and Planning with Spatiotemporal Constraints (STAP-STC), and then by proposing a solution that we call Graphically Recursive Simultaneous Task Allocation, Planning, and Scheduling (GRSTAPS). GRSTAPS interleaves task planning, task allocation, scheduling, and motion planning, performing a multi-layer search while effectively sharing information among system modules. In addition to providing a unified solution to STAP-STC problems, GRSTAPS includes individual innovations both in task planning and task allocation. At the task planning level, our interleaved approach allows the planner to abstract away which agents will perform a task using an approach that we refer to as agent-agnostic planning. At the task allocation level, we contribute a search-based algorithm that can simultaneously satisfy planning constraints and task requirements while optimizing the associated schedule. We demonstrate the efficacy of GRSTAPS using detailed ablative and comparative experiments in a simulated emergency-response domain. Results of these experiments conclusively demonstrate that GRSTAPS outperforms both ablative baselines and state-of-the-art temporal planners in terms of computation time, solution quality, and problem coverage.

Gestures-teleoperation of a heterogeneous multi-robot system

Gestures-teleoperation of a heterogeneous multi-robot system

Capabilities in Heterogeneous Multi-Robot Systems

Groups of robots are often able to to accomplish missions that no single robot can achieve by themselves. Teamwork is a very important factor in complex, dynamic domains. In heterogeneous teams, robustness and flexibility are increased by the diversity of the robots, each contributing different capabilities. In such heterogeneous Multi-Robot Systems it is reasonable to explicitly take the robots' capabilities into account when determining which one is best suited for a task. In this paper I present a framework that formalizes robots' capabilities and provides a means to estimate their suitability for a task. In highly unpredictable domains, accurate predictions of the outcomes of a robot's actions are virtually impossible. Approximate models and algorithms are required which help to estimate the outcome with highest possible confidence. The proposed architecture can provide estimates of task solution qualities at three levels of confidence: the lowest level only taking the mere existence of capabilities into account, the middle level considering task-specific details with approximate parameters of the capabilities, and the highest confidence level considering more elaborate planning algorithms.

Control of a multirobot bricklaying system

AbstractResearch is increasingly shifting from the design and control of single robots toward collaborative methods for multirobot systems. Particularly interesting are heterogeneous systems since the combination of different capabilities can tackle tasks that are difficult, if not impossible, to perform with a single robot. In this article we propose a scheme for the control of a novel heterogeneous multirobot system designed for bricklaying where the use of a single robot is not feasible due to the weight of the payloads to be managed. The proposed solution is based on a crane which cooperates with a lightweight rigid robot. The correct cooperation between the two robotic subunits poses a series of control challenges that must be studied in the context of cooperative manipulation of an object. In this article, we first derive the mathematical model of this novel robotic system during the positioning of the block. Then a control law is proposed. The proposed law is based on an ad hoc inverse dynamic control which takes advantage of the overactuation of the resulting connected system to unload the robotic arm from the weight of the block. A physical CAD‐based simulator of the system is used to show the effectiveness of the proposed control law and to show that the proposed architecture allows to position precisely large and heavy blocks while using a relatively small rigid robotic manipulator.

Priority-Based Distributed Coordination for Heterogeneous Multi-Robot Systems With Realistic Assumptions

A standing challenge in current intralogistics is to reliably, effectively, yet safely coordinate large-scale, heterogeneous multi-robot fleets without posing constraints on the infrastructure or unrealistic assumptions on robots. A centralized approach, proposed by some of the authors in prior work, allows to overcome these limitations with medium-scale fleets (i.e., tens of robots). With the aim of scaling to hundreds of robots, in this article we explore a decentralized variant of the same approach. The proposed framework maintains the key features of the original approach, namely, ensuring safety despite uncertainties on robot motions, and generality with respect to robot platforms, motion planners and controllers. We include considerations on liveness and report solutions to prevent or recover from deadlocks in specific situations. We validate the approach empirically in simulation with large, heterogeneous multi-robot fleets (with up to 100 robots) operating in both benchmark and realistic environments.

Resilient Task Allocation in Heterogeneous Multi-Robot Systems

This letter presents a resilient mechanism to allocate heterogeneous robots to tasks under difficult environmental conditions such as weather events or adversarial attacks. Our primary objective is to ensure that each task is assigned the requisite level of resources, measured as the aggregated capabilities of the robots allocated to the task. By keeping track of task performance deviations under external perturbations, our framework quantifies the extent to which robot capabilities (e.g., visual sensing or aerial mobility) are affected by environmental conditions. This enables an optimization-based framework to flexibly reallocate robots to tasks based on the most degraded capabilities within each task. In the face of resource limitations and adverse environmental conditions, our algorithm relaxes the resource constraints corresponding to some tasks, thus exhibiting a graceful degradation of performance. Simulated experiments in a multi-robot coverage and target tracking scenario demonstrate the efficacy of the proposed approach.

Distributed Adaptive Cooperative Control With Fault Compensation Mechanism for Heterogeneous Multi-Robot System

In this paper, a distributed adaptive consensus law with fault compensation for a heterogeneous multi-robot system (MRS) is proposed. The design paradigm adopted in this work involves a leader-following cooperative algorithm featuring two distinct adaptive coupling gains to compensate for multiple additive time-varying faults. Exacerbating the situation, the follower robots commissioned in the leader-following mission are non-identical due to their dynamic characteristic as normally exist in a physical setup. The capability of the proposed scheme is investigated and compared with the other two recent works in two facets; one is to gauge how the algorithm is able to mitigate faults of varying nature in the presence of heterogeneous robot(s) while maintaining the platoon formation during the leader-following task; two is the ability to cope with subsequent topology reconfiguration. The stability and the robustness of the proposed scheme against bounded time-varying faults are proven using rigorous Lyapunov analysis. The proposed control strategy exempts the use of an observer or estimator, thereby simplifies the synthesis and implementation on mobile robots. The simulation results of the proposed adaptive consensus law demonstrate the best performance as compared to the other two recent works in the presence of multiple faulty robots.

Optimization for cooperative task planning of heterogeneous multi-robot systems in an order picking warehouse

This article investigates the cooperative task planning problem of heterogeneous multi-robot systems in an order picking warehouse, with challenging characteristics such as heterogeneous agents, interrelated utilities and complex intertask dependencies. These characteristics bring great challenges for modelling and optimization of the problem. This study establishes a novel mapping mechanism to describe the problem as open shop scheduling with sequence-dependent set-up and transportation times, and develops an efficient mixed-integer linear programming model to obtain the optimal task planning scheme for small-size problems. To increase the size of solvable problems and improve the performance of the solution, an effective hybrid artificial bee colony algorithm is proposed, which introduces some concepts from the tabu search algorithm and the artificial immune algorithm. The effectiveness of the proposed method is confirmed by simulation experiments.

Ground and Aerial Collaborative Mapping in Urban Environments

A heterogeneous multi-robot system consisting of Unmanned Ground Vehicles (UGVs) and Unmanned Aerial Vehicles (UAVs) have advantages over a single-robot system in efficiency and flexibility, enabling them to perform a larger range of tasks. To allow heterogeneous platforms to work together in GPS-denied scenarios, it is crucial to build a complete 3D map of the environment. In this letter, a novel method is presented to perform ground and aerial collaborative mapping leveraging visual and range data collected by cameras and 3D LiDAR sensors. In the proposed system, a visual-LiDAR ego-motion estimation module that considers point, line and planar constraints can provide robust odometry information. Thumbnail images representing obstacle outlines are generated and descriptors are extracted using a neural network to help perform data association between separate runs. Map segments and the robot poses are organized together and are updated during a pose graph optimization procedure. The proposed ground-aerial collaborative mapping approach is evaluated on both synthetic and real-world datasets comparing with other methods. Experiment results demonstrate that our method can achieve outstanding mapping results.

Spatial winding: cooperative heterogeneous multi-robot system for fibrous structures

This research presents a cooperative heterogeneous multi-robot fabrication system for the spatial winding of filament materials. The system is based on the cooperation of a six-axis robotic arm and a customized 2 + 2 axis CNC gantry system. Heterogeneous multi-robot cooperation allows to deploy the strategy of Spatial Winding: a new method of sequential spatial fiber arrangement, based on directly interlocking filament-filament connections, achieved through wrapping one filament around another. This strategy allows to create lightweight non-regular fibrous space frame structures. The new material system was explored through physical models and digital simulations prior to deployment with the proposed robotic fabrication process. An adaptable frame setup was developed which allows the fabrication of a variety of geometries within the same frame. By introducing a multi-step curing process that integrates with the adaptable frame, the iterative production of continuous large-scale spatial frame structures is possible. This makes the structure’s scale agnostic of robotic reach and reduces the necessary formwork to the bare minimum. Through leveraging the capacities of two cooperating machines, the system allows to counteract some of their limitations. A flexible, dynamic and collaborative fabrication system is presented as a strategy to tailor the fiber in space and expand the design possibilities of lightweight fiber structures. The artifact of the proposed fabrication process is a direct expression of the material tectonics and the robotic fabrication system.

A Routing Framework for Heterogeneous Multi-Robot Teams in Exploration Tasks

This letter proposes a routing framework for heterogeneous multi-robot teams in exploration tasks. The proposed framework deals with a combinatorial optimization problem, and provides a new solving algorithm, for Generalized Team Orienteering Problem (GTOP). In this letter, a route optimization problem is formulated for a heterogeneous multi-robot system. A novel problem solver is also proposed based on self-organizing map. The proposed framework has a strong advantage in its scalability because the processing time is independent from the number of robots, and the heterogeneity of the team. The validity of the proposed framework is evaluated in the exploration, and mapping tasks by heterogeneous robot team with overlapping abilities. The simulation results show the effectiveness of the proposed framework, and how it outperforms the conventional greedy exploration scheme.

A Decentralised Strategy for Heterogeneous AUV Missions via Goal Distribution and Temporal Planning

Heterogeneous multi-robot systems offer the potential to support complex missions, such as those needed for persistent autonomy in underwater domains. Such systems enable each robot to be optimised for specific tasks to better manage dynamic situations. In this context, temporal planning can generate plans to support the execution of multi-robot missions. However, the task distribution quality in the generated plans is often poor due to the strategies that existing planners employ to search for suitable actions, which do not tend to optimise task allocation. In this paper, we propose a new algorithm called the Decentralised Heterogeneous Robot Task Allocator (DHRTA) which enhances goal distribution by considering task spatial distribution, execution time, and the capabilities of the available robots. DHRTA is the first phase of our decentralised planning strategy which supports individual robot plan generation using temporal planners. Experiments illustrate the robustness of the approach and indicate improvements in plan quality by reducing the planning time, mission time and the rate of mission failures.

Coupled task scheduling for heterogeneous multi-robot system of two robot types performing complex-schedule order fulfillment tasks

This paper addresses multi-robot task scheduling for two robot types arising from heterogeneous robotic order fulfillment systems. The heterogeneous multi-robot system comprises two types of robots with specialized and complementary capabilities to achieve long-cycle and multi-station order fulfillment tasks on a logistic network. This problem is extremely challenging because of innate complex-schedule constraints of tasks and coupled temporal–spatial relations between all robots. After set-theoretic and mixed integer linear programming problem formulations, we use coupled approach, instead of decoupled approaches to explore the synergy between heterogeneous robots, which is different from most existing similar works. To model the structural (complex-schedule) and quantitative (temporal–spatial) coupledness of robots’ time-extended task schedules, an edge-weighted and vertex-weighted block sequence graph is introduced. Based on this model, time-extended task scheduling is achieved using rank-minimal heuristic and genetic algorithm metaheuristic. Theoretically, this model is complete and non-redundant. Empirically, compared with decoupled approach, optimality and efficiency of the proposed methods are evaluated on designed instances. The results demonstrate that coupled methods can achieve near-optimal solutions with higher performance ratio than decoupled methods in moderate time. At the same time, coupled methods can leverage spatial and temporal properties of miscellaneous tasks, and balance instantaneous and time-extended decisions to achieve tight collective synergy in the long run.

Development of a task-oriented, auction-based task allocation framework for a heterogeneous multirobot system

A multirobot system has cooperative team of robots designed to enhance efficiency of its operations. One of the critically investigated problems of multirobot system is the multirobot task allocation (MRTA) issue. The main objective of MRTA is to assign tasks to the most suitable robot based on its functions and capability as well as availability. In this paper, a task-oriented, auction-based task allocation framework is presented and tested through simulations and real-world experiments. The developed framework consists of a novel heuristic-based task allocation algorithm and communication module. It is implemented in a multirobot system, allowing tasks to be dynamically assigned to the robots as they achieve given tasks. The implemented framework shows robustness in its flexibility to the task and environment requirements such as resource and energy requirements and size of the environment. The framework involved a task allocation algorithm, which consists of bid generation and bid selection process, and a TCP/IP-based client-server communication module. The results from both simulations and real-world experiments matched, producing optimum results in task allocation.

Engineering Challenges Ahead for Robot Teamwork in Dynamic Environments

The increasing number of robots around us creates a demand for connecting these robots in order to achieve goal-driven teamwork in heterogeneous multi-robot systems. In this paper, we focus on robot teamwork specifically in dynamic environments. While the conceptual modeling of multi-agent teamwork was studied extensively during the last two decades and commercial multi-agent applications were built based on the theoretical foundations, the steadily increasing use of autonomous robots in many application domains gave the topic new significance and shifted the focus more toward engineering concerns for multi-robot systems. From a distributed systems perspective, we discuss general engineering challenges that apply to robot teamwork in dynamic application domains and review state-of-the-art solution approaches for these challenges. This leads us to open research questions that need to be tackled in future work.

UWB-Based Localization for Multi-UAV Systems and Collaborative Heterogeneous Multi-Robot Systems

Ultra-wideband technology has emerged in recent years as a robust solution for localization in GNSS denied environments. In particular, its high accuracy when compared to other wireless localization solutions is enabling a wider range of collaborative and multi-robot application scenarios, being able to replace more complex and expensive motion-capture areas for use cases where accuracy in the order of tens of centimeters is sufficient. We present the first survey of UWB-based localization focused on multi-UAV systems and heterogeneous multi-robot systems. We have found that previous literature reviews do not consider in-depth the challenges in both aerial navigation and navigation with multiple robots, but also in terms of heterogeneous multi-robot systems. In particular, this is, to the best of our knowledge, the first survey to review recent advances in UWB-based (i) methods that enable ad-hoc and dynamic deployments; (ii) collaborative localization techniques; and (iii) cooperative sensing and cooperative maneuvers such as UAV docking on mobile platforms. Finally, we also review existing datasets and discuss the potential of this technology for both localization in GNSS-denied environments and collaboration in multi-robot systems.