Multi-robot Collaboration Research Articles

Edge Intelligence Enabled Heterogeneous Multi-Robot Networks: Hybrid Framework, Communication Issues, and Potential Solutions

Heterogeneous multi-robot systems have been increasingly employed to collaboratively execute tasks in harsh and unknown environments. High-quality collaboration requires robots to efficiently communicate with each other and have ample intelligence to deal with uncertainties and emergencies. However, these demands are still difficult to be satisfied under existing single-mode networking architectures and onboard computing paradigms. Therefore, this article first develops a hybrid edge-aided framework for multi-robot collaboration, where the centralized and decentralized modes are elegantly combined to support a robust and efficient network, while an edge intelligence platform with powerful servers and base stations is employed to provide abundant intelligence. Under this framework, we discuss three critical communication issues that keep robots coordinated and guarantee efficient information sharing. The issues include network construction among heterogenous robots, bandwidth-efficient communication protocol design, and link transmission adaption. For each issue, potential solutions are provided and the challenges that need to be further explored are also discussed.

DeepBrain

With the recent advancements of electroencephalograph (EEG) techniques, some brain-computer interface (BCI) solutions have been explored to assist individuals performing various tasks with their minds. One promising application is to combine BCI with robotic systems so that the mobility-impaired people can control robots to take care of themselves. Towards this ultimate goal to design BCIs for mobility-impaired, we firstly conducted an online survey with 54 mobility-impaired participants who barely had previous experience with BCI to identify the challenges they face in life for the purpose of designing a personalized BCI system in need. The results revealed these challenges including small daily tasks (such as feeding and cleaning), which weigh on the financial burdens of hiring a caregiver. Meanwhile, the off-the-shelf high-fidelity BCIs are often expensive, whereas the cheaper devices only collect coarse-grained signals, preventing practical application in care aids due to lack of temporal resolution and accuracy. Based on the survey findings, we then designed DeepBrain, a human-centered learning augmented BCI system, that requires only coarse-grained brain signals with low-cost BCI equipment, but supports fine-grained brain-robot interaction and scalable multi-robot collaboration for domestic multi-task operations. A follow-up system comparison with other approaches show that the proposed human-centered solution is a promising step towards the ultimate goal, as it achieves satisfactory accuracy with less low computation resources. Also the practical brain to multi-robot interaction system validates the feasibility of our framework and model used in DeepBrain.

A Reconfigurable Multirobot Cooperation Workcell for Personalized Manufacturing

Most robotic systems designed for mass manufacturing are optimized for a specific type of product. They generally lack the ability to adapt to low-volume customized products. In this article, we present a system based on a modular design for manufacturing personalized medical stent graft implants. The concept is based on learning-by-demonstration by integrating real-time 3-D vision, multirobot collaboration, and personalization to guide the robots to learn and execute tasks continuously with adaptation to different implant geometry. The system is optimized to generate customized and collision-free paths for efficient object manipulation and task completion. We show that the system is generalizable to different stent graft designs and the proposed multirobot system can seamlessly work together with high efficiency without collisions. The results have also suggested its usability for other manipulation tasks, especially for flexible production of customized products where bimanual or multirobot cooperation is required. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this article is the problem of automatic sewing of personalized stent grafts (a tailor-made artificial vessel). Existing personalized stent grafts are mostly hand sewn, which is time consuming and often undersupplied. Automating such process can significantly improve the production and this requires a sewing system that can handle different designs. This article suggests a new schema to design a robotic system to handle personalized designs. The first methodology is modularized design to separate the task into a repetitive part and a personalized part, each handled by a module. The second methodology is to find the best relative pose between the modules such that the robots can complete their task within their working space and with minimum motion. This ensures that a stent graft can be sewn feasibly and with the lowest cost. Computational results show this approach can find optimal solution for different personalized stent grafts and preliminary on robot experiment verifies that this approach is feasible. Please note that this approach is not limited to sewing personalized stent graft. The schema can be applied to solve similar problem of customized product motion planning and system design.

Formation optimization of blockchain-assisted swarm robotics systems against failures based on energy balance

The blockchain-assisted swarm robotics system, as an emerging distributed multi-robot system, has been widely discussed and studied for its outstanding advantages in secure communication, data logging and consensus achievement. Due to faults, formation changes, and obstructions, the network topology and communication rate change in the process of multi-robot collaboration, which inevitably affect the stability and operation efficiency of the swarm robotics system. However, few studies have quantitatively assessed the degradation of system stability performance after failure and how to respond to failure of team members and optimize the system for failure. In this paper, the concept of “energy” is introduced into the quantitative evaluation of system failure resistance, and an evaluation model based on communication distance, communication quality and communication volume is proposed, which can evaluate the stability of swarm robotics systems at different moments and any member in real-time collaboration. As the stability improves, the “energy” increases, and vice versa. Additionally, the “energy” balance is applied to the optimization process of multi-robot formation, and the optimization equation based on the efficiency of swarm robots is proposed. The optimal formation efficiency equation of the system at a certain moment is obtained by jointly solving the energy optimization equation and the system. The simulation results show that the proposed formation optimization method can improve the stability of the swarm robotics system at any time and in all directions, and maximize the operational efficiency of the queue.

Multi robot navigation formation simulation based on ROS

Multi-robot collaboration is an important development direction of current robotics technology. This paper proposes a navigator and follower model formation control framework that integrates path planning and trajectory tracking, and conducts experiments in the gazebo simulation environment to realize the simulation of navigation and formation led by the navigator in the simulation environment indoors.

Simultaneous Calibration of the Hand-Eye, Flange-Tool and Robot-Robot Relationship in Dual-Robot Collaboration Systems.

A multi-robot collaboration system can complete more complex tasks than a single robot system. Ensuring the calibration accuracy between robots in the system is a prerequisite for the effective inter-robot cooperation. This paper presents a dual-robot system for orthopedic surgeries, where the relationships between hand-eye, flange-tool, and robot-robot need to be calibrated. This calibration problem can be summarized to the solution of the matrix equation of AXB=YCZ. A combined solution is proposed to solve the unknown parameters in the equation of AXB=YCZ, which consists of the dual quaternion closed-form method and the iterative method based on Levenberg–Marquardt (LM) algorithm. The closed-form method is used to quickly obtain the initial value for the iterative method so as to increase the convergence speed and calibration accuracy of the iterative method. Simulation and experimental analyses are carried out to verify the accuracy and effectiveness of the proposed method.

Progress and Development Trend of Space Intelligent Robot Technology

Since space intelligent robots are not restricted by physiological conditions, it is an attractive choice for the development of automation technology to use them for space exploration and utilization. It is currently the key development direction of the major space powers over the world. This paper first investigates the robotic manipulators and humanoid robot systems for space station applications and reviews theories and methods for robots to achieve large-range stable motion and intelligent dexterous manipulation. Then, the intelligent robot systems for on-orbit satellite maintenance are reviewed, and the related technologies of multirobot collaboration are analyzed. Finally, we investigate the intelligent robot systems for on-orbit assembly of large-scale spatial structures and summarize the technologies of modular assembly and on-orbit manufacture. Overall, this paper reviews the technological progress and development trends of space robots, which provides a good reference for further technical research in this field.

Enabling impedance-based physical human–multi–robot collaboration: Experiments with four torque-controlled manipulators

Robotics research into multi-robot systems so far has concentrated on implementing intelligent swarm behavior and contact-less human interaction. Studies of haptic or physical human-robot interaction, by contrast, have primarily focused on the assistance offered by a single robot. Consequently, our understanding of the physical interaction and the implicit communication through contact forces between a human and a team of multiple collaborative robots is limited. We here introduce the term Physical Human Multi-Robot Collaboration (PHMRC) to describe this more complex situation, which we consider highly relevant in future service robotics. The scenario discussed in this article covers multiple manipulators in close proximity and coupled through physical contacts. We represent this set of robots as fingers of an up-scaled agile robot hand. This perspective enables us to employ model-based grasping theory to deal with multi-contact situations. Our torque-control approach integrates dexterous multi-manipulator grasping skills, optimization of contact forces, compensation of object dynamics, and advanced impedance regulation into a coherent compliant control scheme. For this to achieve, we contribute fundamental theoretical improvements. Finally, experiments with up to four collaborative KUKA LWR IV+ manipulators performed both in simulation and real world validate the model-based control approach. As a side effect, we notice that our multi-manipulator control framework applies identically to multi-legged systems, and we execute it also on the quadruped ANYmal subject to non-coplanar contacts and human interaction.

Service Robots in Wuhan Cabin Hospitals

medical service robot, human-robot interaction, COVID-19, sensor fusion The cabin hospitals used for COVID-19 victims have an infectious environment and thus require highly autonomous systems to provide various services. This article presents a multirobot architecture designed for cabin hospitals with special focus on the sensors used by service robots. The robots' functions include autonomous disinfection, delivery, cleaning, temperature measurement, physical interaction, and conversation assistance. The robots collaborate with each other by using multiple sensors in large and complicated scenes and evolve together. The robots are controlled by cloud controllers considering a human-centered multi-robot collaboration mechanism. This article presents the details of these robots, including the hardware and software architecture of the robots with sensors, the communication modules, and the cloudbased central controllers. It also presents several applications of the robots and discusses open problems. © 2021 M Y U Scientific Publishing Division. All rights reserved.

Adaptive heterogeneous multi-robot collaboration from formal task specifications

Efficiently coordinating different types of robots is an important enabler for many commercial and industrial automation tasks. Here, we present a distributed framework that enables a team of heterogeneous robots to dynamically generate actions from a common, user-defined goal specification. In particular, we discuss the integration of various robotic capabilities into a common task allocation and planning formalism, as well as the specification of expressive, temporally-extended goals by non-expert users. Models for task allocation and execution both consider non-deterministic outcomes of actions and thus, are suitable for a wide range of real-world tasks including formally specified reactions to online observations. One main focus of our paper is to evaluate the framework and its integration of software modules through a number of experiments. These experiments comprise industry-inspired scenarios as motivated by future real-world applications. Finally, we discuss the results and learnings for motivating practically relevant, future research questions.

DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM

The assumption of scene rigidity is common in visual SLAM algorithms. However, it limits their applicability in populated real-world environments. Furthermore, most scenarios including autonomous driving, multi-robot collaboration and augmented/virtual reality, require explicit motion information of the surroundings to help with decision making and scene understanding. We present in this paper DynaSLAM II, a visual SLAM system for stereo and RGB-D camera configurations that tightly integrates the multi-object tracking capability. DynaSLAM II makes use of instance semantic segmentation and ORB features to track dynamic objects. The structures of the static scene and the dynamic objects are optimized jointly with the trajectories of both the camera and the moving agents within a novel bundle adjustment proposal. The 3D bounding boxes of the objects are also estimated and loosely optimized within a fixed temporal window. We demonstrate that tracking dynamic objects does not only provide rich clues for scene understanding but can be also beneficial for camera tracking.

Hallway exploration-inspired guidance: applications in autonomous material transportation in construction sites

Continued attempts have been conducted to improve construction productivity with automatic material transportation robots, yet neither of them has been popularized. The reason is that navigating in highly unstructured and dynamic indoor environments remains a great challenge. Therefore, we present a hallway exploration-inspired guidance approach (HEIGA), which guides an agent to its destination based on a series of directional guidance without the need for self-localization in a global frame. A series of experiments were conducted which prove that fully autonomous material transportation can be realized without being influenced by the complexity or dynamics of real construction sites. We expect this new approach to effectively: (i)free workers from heavy and repetitive material transportation tasks; and (ii) improve construction productivity. In the future, the presented approach will be applied in construction sites considering multi-robot collaboration, aiming at a higher material transportation efficiency.

Multi-Robot Collaborative Source Searching Strategy in Large-Scale Chemical Clusters

In hazardous gas leakage accidents in chemical clusters, quickly and accurately locating the diffusion source via mobile sensors (searcher) can effectively speed up emergency response. While state-of-the-art researches designed kinds of autonomous sensing and searching techniques, e.g. cognitive-based strategies to complete the localization of the leakage source; the current studies cannot meet chemical cluster’s requirements with complex road network constraints and spatial scales. To address this problem, we firstly combine a cognitive search strategy with great theoretical performance (Entrotaxis algorithm) with an intermittent search strategy and propose a fresh searching algorithm, namely Entrotaxis-Turn (ET). The hybrid algorithm can make use of the triggered turn motion to bypass obstacles to avoid falling into a loop, thereby improving the success rate and saving the searching time. Secondly, we extend the ET algorithm to the multi-robot collaboration pattern (Multi-ET algorithm), in which each mobile sensor shares the information collected along its route with each other. To reveal the performance of the proposed algorithms, a simulation scene is built in a typical chemical cluster scene and diffused gas is generated based on the advection-diffusion equation. Through Monte Carlo simulations, the optimal parameter combinations in the ET and Multi-ET algorithm are obtained. Finally, a verification experiment is carried out based on the simulation scene generated by the computational fluid dynamics (CFD) model. Results show that compared with the Entrotaxis algorithm, the efficiency and success rate of the ET algorithm are dramatically improved. Moreover, the Multi-ET algorithm obtains the optimal source searching performance.

Decentralized Ability-Aware Adaptive Control for Multi-Robot Collaborative Manipulation

Multi-robot teams can achieve more dexterous, complex and heavier payload tasks than a single robot, yet effective collaboration is required. Multi-robot collaboration is extremely challenging due to the different kinematic and dynamics capabilities of the robots, the limited communication between them, and the uncertainty of the system parameters. In this letter, a Decentralized Ability-Aware Adaptive Control (DA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> C) is proposed to address these challenges based on two key features. Firstly, the common manipulation task is represented by the proposed nominal task ellipsoid, which is used to maximize each robot's force capability online via optimizing its configuration. Secondly, a decentralized adaptive controller is designed to be Lyapunov stable in spite of heterogeneous actuation constraints of the robots and uncertain physical parameters of the object and environment. In the proposed framework, decentralized coordination and load distribution between the robots is achieved without communication, while only the control deficiency is broadcast if any of the robots reaches its force limits. In this case, the object's reference trajectory is modified in a decentralized manner to guarantee stable interaction. Finally, we perform several numerical and physical simulations to analyse and verify the proposed method with heterogeneous multi-robot teams in collaborative manipulation tasks.

Design and Collaborative Operation of Multimobile Inspection Robots in Smart Microgrids

This paper investigates the substation inspection problems of multimobile robots for large power stations in smart microgrids. Most multirobot inspection robots generally face the challenge of path planning, while the current widely used biological excitation neural network (BENN) methods often have the defect of the neuronal active field near boundaries and obstacles. To end this, we propose an improved biological excitation neural network (IBENN) method for path planning based on a detailed architecture and ontology framework, through which the single-point inspection, multipoint inspection, and full-area inspection tasks of substations in smart microgrids can be well completed. Simulation results show that the designed IBENN-based multirobot collaboration inspection (MRCI) system can effectively shorten the path as well as the number of turns and then show better performance than most existing results when implementing various substation inspection tasks.

Multirobot Cooperative Localization Based on Visible Light Positioning and Odometer

With the development of robotics, multirobot collaboration system (MRCS) has become a popular focus of research in recent years. As the key technology of MRCS, multirobot cooperative localization (MRCL) is becoming more and more popular due to its higher accuracy and robustness by exchanging and sharing information. In this article, we propose an MRCL system based on visible light positioning (VLP) technology and odometer (VO-MRCL). Using rolling shutter camera and odometer, the system relaxes the quantity required for observable-light emitting diode (LED) lamps of each robot to one and improves the robustness of the VLP-based robot positioning methods under LED shortage. The accuracy of the proposed VO-MRCL system is verified by real-world experiments, where a two-robot system is proposed as an example specifically. The results of experiment show that our proposed system is feasible and can achieve the positioning accuracy with an average error of 4.31 cm.

Blockchain-Empowered Multi-Robot Collaboration to Fight COVID-19 and Future Pandemics.

This conceptual paper overviews how blockchain technology is involving the operation of multi-robot collaboration for combating COVID-19 and future pandemics. Robots are a promising technology for providing many tasks such as spraying, disinfection, cleaning, treating, detecting high body temperature/mask absence, and delivering goods and medical supplies experiencing an epidemic COVID-19. For combating COVID-19, many heterogeneous and homogenous robots are required to perform different tasks for supporting different purposes in the quarantine area. Managmnt and decentralizing multi-robot play a vital role in combating COVID-19 by reducing human interaction, monitoring, delivering goods. Blockchain technology can manage multi-robot collaboration in a decentralized fashion, improve the interaction among them to exchange information, share representation, share goals, and trust. We highlight the challenges and provide the tactical solutions enabled by integrating blockchain and multi-robot collaboration to combat the COVID-19 pandemic. The proposed conceptual framework can increase the intelligence, decentralization, and autonomous operations of connected multi-robot collaboration in the blockchain network. We overview blockchain potential benefits to defining a framework of multi-robot collaboration applications to combat COVID-19 epidemics such as monitoring and outdoor and hospital End to End (E2E) delivery systems. Furthermore, we discuss the challenges and opportunities of integrated blockchain, multi-robot collaboration, and the Internet of Things (IoT) for combating COVID-19 and future pandemics.

Role and task allocation framework for Multirobot collaboration with latent knowledge estimation

SummaryIn this work a novel framework for modeling role and task allocation in Cooperative Heterogeneous Multi‐Robot Systems (CHMRSs) is presented. This framework encodes a CHMRS as a set of multidimensional relational structures (MDRSs). This set of structure defines collaborative tasks through both temporal and spatial relations between processes of heterogeneous robots. These relations are enriched with tensors which allow for geometrical reasoning about collaborative tasks. A learning schema is also proposed in order to derive the components of each MDRS. According to this schema, the components are learnt from data reporting the situated history of the processes executed by the team of robots. Data are organized as a multirobot collaboration treebank (MRCT) in order to support learning. Moreover, a generative approach, based on a probabilistic model, is combined together with nonnegative tensor decomposition (NTD) for both building the tensors and estimating latent knowledge. Preliminary evaluation of the performance of this framework is performed in simulation with three heterogeneous robots, namely, two Unmanned Ground Vehicles (UGVs) and one Unmanned Aerial Vehicle (UAV).

Physically distributed multi-robot coordination and collaboration in construction

This paper presents strategies for the scalable fabrication of long-span composite structures made possible through physically distributed heterogeneous multi-robot collaboration. An interactive and collaborative fiber laying process between industrial robots and an unmanned aerial vehicle (UAV) is described as a case study, investigating the challenges of multi-robot distributed fabrication, but also the design and system potentials of such an approach. A proof-of-concept interactive and collaborative process based on hardware exchange was developed for the process of long-span fiber composite filament winding. Components of the fabrication workcell, including two high pay-load industrial six-axis robots, a custom developed UAV drone, pneumatic winding end-effectors with sensor integration, and a signal integrated mechanism for tension control are described. This paper covers particularly the hardware and software components and strategies that coordinated a physically distributed collaborative process between the diverse system of devices, robots, and sensor integrated machines. These aspects include: (1) a platform-agnostic communication infrastructure based on ROS (2) compilation of coordinated and sequence dependent fabrication instructions from the design environment into a custom task list for execution in a web user interface (3) an adaptive and flexible strategy for enabling the flexible execution of industrial robot control code (4) hardware strategies and sensors for handling process tolerances emerging from physically distributed collaboration. The paper concludes with an outlook on important considerations and technical components for increasing the scale and complexity of fabrication through digital coordination and physically distributed collaboration.

MRCDRL: Multi-robot coordination with deep reinforcement learning

This paper proposes a multi-robot cooperative algorithm based on deep reinforcement learning (MRCDRL). We use end-to-end methods to train directly from each robot-centered, relative perspective-generated image, and each robot’s reward as the input. During training, it is not necessary to specify the target position and movement path of each robot. MRCDRL learns the actions of each robot by training the neural network. MRCDRL uses the neural network structure that was modified from the Duel neural network structure. In the Duel network structure, there are two streams that each represents the state value function and the state-dependent action advantage function, and the results of the two streams are merged. The proposed method can solve the resource competition problem on the one hand and can solve the static and dynamic obstacle avoidance problems between multi-robot in real time on the other hand. Our new MRCDRL algorithm has higher accuracy and robustness than DQN and DDQN and can be effectively applied to multi-robot collaboration.