Heterogeneous Robotic Systems Research Articles

Digital Battle: A Three-Layer Distributed Simulation Architecture for Heterogeneous Robot System Collaboration

In this paper, we propose a three-layer distributed simulation network architecture, which consists of a distributed virtual simulation network, a perception and control subnetwork, and a cooperative communication service network. The simulation architecture runs on a distributed platform, which can provide unique virtual scenarios and multiple simulation services for the verification of basic perception, control, and planning algorithms of a single-robot system and can verify the distributed collaboration algorithms of heterogeneous multirobot systems. Further, we design simulation experimental scenarios for classic heterogeneous robotic systems such as unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs). Through the analysis of experimental measurement data, we draw several important conclusions: firstly, the replication time characteristics and update frequency characteristics of entity synchronization in our system indicate that the replication time of entity synchronization in our system is relatively short, and the update frequency can meet the needs of multirobot collaboration and ensure the real-time use and accuracy of the system; secondly, we analyze the bandwidth usage of data frames in the whole session and observe that the server side occupies almost half of the data throughput during the whole session, which indicates that the allocation and utilization of data transmission in our system is reasonable; and finally, we construct a bandwidth estimation surface model to estimate the bandwidth requirements of the current model when scaling the server-side scale and synchronization-state scale, which provides an important reference for better planning and optimizing of the resource allocation and performance of the system. Based on this distributed simulation framework, future research will improve the key technical details, including further refining the coupling object dynamic model update method to support the simulation theory of the coupling relationship between system objects, studying the impact of spatiotemporal consistency of distributed systems on multirobot control and decision making, and in-depth research on the impact of collaborative frameworks combined with multirobot systems for specific tasks.

Optimization-based trajectory planning for transport collaboration of heterogeneous systems

Abstract This paper describes an optimization-based trajectory planning scheme for handing over an object between a quadrotor and a wheeled robot in a transportation scenario. Concretely, a quadrotor should pick up an object from a moving ground mobile robot and deliver it to its destination. An optimization framework based on discrete mechanics and complementarity constraints is utilized here to jointly ensure dynamic feasibility and determine the position, timing, and coordination of the handover autonomously. Cooperative trajectories of the heterogeneous robot system can be generated simultaneously to satisfy different requirements by adjusting the objective function and constraints. The proposed planning scheme provides a novel paradigm combining trajectory planning and handover decision-making within an optimal control problem.

Retracted: Collaboration and Management of Heterogeneous Robotic Systems for Road Network Construction, Management, and Maintenance under the Vision of “BIM + GIS” Technology

Retracted: Collaboration and Management of Heterogeneous Robotic Systems for Road Network Construction, Management, and Maintenance under the Vision of “BIM + GIS” Technology

Landmark Topology Descriptor-Based Place Recognition and Localization under Large View-Point Changes.

Accurate localization between cameras is a prerequisite for a vision-based heterogeneous robot systems task. The core issue is how to accurately perform place recognition from different view-points. Traditional appearance-based methods have a high probability of failure in place recognition and localization under large view-point changes. In recent years, semantic graph matching-based place recognition methods have been proposed to solve the above problem. However, these methods rely on high-precision semantic segmentation results and have a high time complexity in node extraction or graph matching. In addition, methods only utilize the semantic labels of the landmarks themselves to construct graphs and descriptors, making such approaches fail in some challenging scenarios (e.g., scene repetition). In this paper, we propose a graph-matching method based on a novel landmark topology descriptor, which is robust to view-point changes. According to the experiment on real-world data, our algorithm can run in real-time and is approximately four times and three times faster than state-of-the-art algorithms in the graph extraction and matching phases, respectively. In terms of place recognition performance, our algorithm achieves the best place recognition precision at a recall of 0-70% compared with classic appearance-based algorithms and an advanced graph-based algorithm in the scene of significant view-point changes. In terms of positioning accuracy, compared to the traditional appearance-based DBoW2 and NetVLAD algorithms, our method outperforms by 95%, on average, in terms of the mean translation error and 95% in terms of the mean RMSE. Compared to the state-of-the-art SHM algorithm, our method outperforms by 30%, on average, in terms of the mean translation error and 29% in terms of the mean RMSE. In addition, our method outperforms the current state-of-the-art algorithm, even in challenging scenarios where the benchmark algorithms fail.

Output scaled consensus for heterogeneous robotic systems with aperiodically intermittent communication and transmission delays

Output scaled consensus for heterogeneous robotic systems with aperiodically intermittent communication and transmission delays

Precision farming technologies for crop protection: A meta-analysis

Precision farming technologies (PFTs) can have a significant impact on reducing the dependency on plant protection products (PPP) for crop protection. PFTs for crop protection comprises an extensive suite of digital solutions that can be used to predict, detect, and control pests. In this paper, we report the results of a systematic literature meta-analysis to map PFTs in relation to crop protection and identify trends and gaps in the use of these technologies in order for them to be easily adopted by farmers. In total, 239 different research articles were assessed in terms of sensor, platform, crop, pest, pest management stage, and impact type. The majority of research articles focused on arable crops and on weed management. The identified PFTs can achieve up to 97 % savings in herbicides, reduce the area that needs insecticide application by up to 70 %, and reduce weed densities by 89 %. In the future, high resolutions images and proximal sensing with heterogeneous robotic systems is expected to make pest detection and control more efficient.

Self-Organized Patchy Target Searching and Collecting with Heterogeneous Swarm Robots Based on Density Interactions

The issue of searching and collecting targets with patchy distribution in an unknown environment is a challenging task for multiple or swarm robots because the targets are unevenly dispersed in space, which makes the traditional solutions based on the idea of path planning and full spatial coverage very inefficient and time consuming. In this paper, by employing a novel framework of spatial-density-field-based interactions, a collective searching and collecting algorithm for heterogeneous swarm robots is proposed to solve the challenging issue in a self-organized manner. In our robotic system, two types of swarm robots, i.e., the searching robots and the collecting robots, are included. To start with, the searching robots conduct an environment exploration by means of formation movement with Levy flights; when the targets are detected by the searching robots, they spontaneously form a ring-shaped envelope to estimate the spatial distribution of targets. Then, a single robot is selected from the group to enter the patch and locates at the patch’s center to act as a guiding beacon. Subsequently, the collecting robots are recruited by the guiding beacon to gather the patch targets; they first form a ring-shaped envelope around the target patch and then push the scattered targets inward by using a spiral shrinking strategy; in this way, all targets eventually are stacked near the center of the target patch. With the cooperation of the searching robots and the collecting robots, our heterogeneous robotic system can operate autonomously as a coordinated group to complete the task of collecting targets in an unknown environment. Numerical simulations and real swarm robot experiments (up to 20 robots are used) show that the proposed algorithm is feasible and effective, and it can be extended to search and collect different types of targets with patchy distribution.

Collaboration and Management of Heterogeneous Robotic Systems for Road Network Construction, Management, and Maintenance under the Vision of “BIM + GIS” Technology

In modern society, intelligent spatial analysis services have been playing an important role in building smart cities. Especially, people’s spatial information needs have changed with the technological progress of times, from traditional two-dimensional appearance to three-dimensional interior. In this case, the construction of street networks is a key part of spatial information analysis research. Establishing an integrated indoor-outdoor street network data model with accurate information and complete attributes can provide strong technical support for indoor-outdoor navigation, emergency evacuation, and other application services. However, most of the current studies on internal and external road networks are independent of each other. This means that indoor road networks are mainly based on floor plans, which contain little information and do not represent the spatial relationships between floors. Indoor and outdoor links often suffer from poor connectivity and data loss. Heterogeneous robotics is a high degree of integration of intelligent control and platform control technologies. In a sense, it is a technology platform for connecting heterogeneous robotic populations. By developing effective collaboration and control strategies, tasks can be performed with the highest success rate and the lowest risk while respecting spatial and temporal constraints. As a result, the study of the coordination and management of heterogeneous robotic systems for road network construction, management, and maintenance is a topic of a great theoretical value and significance. Therefore, this paper proposes an intelligent road network construction method that incorporates BIM and GIS technologies. This study uses the building information model to enrich the information of indoor building components and combines the strong spatial analysis capability of outdoor areas with GIS road network technology to build an indoor road network and design a connection scheme for indoor and outdoor areas. After that, the construction of an integrated indoor-outdoor street network is completed, and the interaction between indoor as well as outdoor spatial data is very smooth.

Intelligent Communications and Heterogeneous Systems in 5G and 6G: Sensing, Security, Space, Thermodynamics, MultipathTCP, Reflection, and Cell-Free Access

In this issue, we are pleased to present two timely feature topics: a collection of six articles on “Intelligent Ultra-Reliable and Low-Latency Communications in 6G,” organized by guest editors C. She, T.Q. Duong, T.Q.S. Quek, H. Viswanathan, and D. Lopez-Perez; and a collection of five articles on ‘Heterogeneous Robotic Systems in 5G and Beyond Networks,” organized by guest editors, H. Lu, B. Khalfi, P.-H. Ho, and C. Verikoukis. Both guest editorial teams did an excellent job in organizing these feature topics via a rigorous selection process. Please refer to the guest editorials and individual articles for more details.

Multi-Robot Distributed Communication in Heterogeneous Robotic Systems on 5G Networking

This work aims at new opportunities for the development of the robot communication network. Besides, a robot operation and decision control method driven by Digital Twins (DT) is designed to utilize lasers in a virtual environment. The multi-agent is integrated with the Proximal Policy Optimization (PPO) algorithm, forming the Muti-PPO algorithm for robot path planning. In addition, a consistent hashing algorithm is introduced and optimized as a Weighted Consistent Hash of Multiple Mappings (WCHMM) algorithm with multiple mappings. The experimental results indicate that the average task execution time of the WCHMM algorithm in the case of three bus nodes is 8.09s, which is 32.47 percent and 21.38 percent lower than the task execution time in the case of a single bus node and two bus nodes, respectively. The multi-robot distributed communication system based on Service Oriented Architecture (SOA) gives full play to the various inherent advantages of the multi-robot system.

Guest Editorial: Introduction to Heterogeneous Robotic Systems in 5G and Beyond Networks

Remote intelligence in the application of robotics and the autonomous system relies heavily on seamless wireless connections. The 5G mobile network technology meets traditional manufacturing enterprises' application requirements for wireless networks based on robot transformation and upgrading, robot interconnection, and remote interactive applications in production. However, there exists many challenging 5G communication issues, such as different communication protocols in the system varies with different robots and no dis-ruptive changes to the physical layer.

Implementation of a Heterogeneous Multi-Robot System for a Construction Task

Cooperative robotic systems are becoming increasingly important, as cooperation between two or more robots makes it possible to tackle tasks that are difficult, if not impossible, to perform with a single robot. Among the various cooperative robotic systems, in this paper we focus on a heterogeneous robotic system to perform a building activity where large and heavy blocks are used. The proposed robotic system is based on a lifting mechanism and a robotic arm. This system exploits the characteristics of the lifting cable mechanism to hold most of the weight of the block, while the use of the rigid robot allows to obtain the desired precision during the fine placement. In this paper, we first derive the dynamic model of the system under consideration. Then, the kinematics analysis of the multi-robot system is investigated. Based on these two previous results, a control scheme is designed to ensure safe and correct cooperation between the two robotic systems. Experimental results with an in-scale prototype of the proposed robotic solution show the efficiency of our approach.

On Tracking Control of Robot Manipulator with Flexible Joints in Operational Space

In this paper, an adaptive control framework and a command-filtered control framework are proposed for flexible joint robot manipulators to track a desired trajectory in the workspace. First, we present an adaptive control algorithm which ensures that the output (the end-effector) of the uncertain manipulator with flexible joints tracks a desired trajectory in the operational space. Subsequently, a control algorithm is synthesized with the command-filtered technique to eliminate the requirements of the joint-jerk feedback and the third-order derivative of system models. The advantages of the proposed control frameworks are guaranteeing the tracking performance in the workspace and scaling down the complexity of the controller. Therefore, the proposed approaches can be applied to heterogeneous robotic systems and are easier to implement. The stability and convergence of the closed-loop systems are analyzed based on Lyapunov technique. Simulation results are provided to show the efficiency of the proposed control frameworks.

Scheduling of Missions with Constrained Tasks for Heterogeneous Robot Systems

We present a formal tasK AllocatioN and scheduling apprOAch for multi-robot missions (KANOA). KANOA supports two important types of task constraints: task ordering, which requires the execution of several tasks in a specified order; and joint tasks, which indicates tasks that must be performed by more than one robot. To mitigate the complexity of robotic mission planning, KANOA handles the allocation of the mission tasks to robots, and the scheduling of the allocated tasks separately. To that end, the task allocation problem is formalised in first-order logic and resolved using the Alloy model analyzer, and the task scheduling problem is encoded as a Markov decision process and resolved using the PRISM probabilistic model checker. We illustrate the application of KANOA through a case study in which a heterogeneous robotic team is assigned a hospital maintenance mission.

Energy-Efficient Ground Traversability Mapping Based on UAV-UGV Collaborative System

With the development of science and technology, robots have been widely used in smart cities. The traversability mapping of environment perception is the prerequisite for robots to perform tasks. To save the energy consumption of traversability mapping for unmanned ground vehicle (UGV), we fusion a wide range of aerial images and a small amount of ground images to provide vision for UGV. Current map fusion methods are usually constrained by homogeneous model of robotic systems and lack of diverse sensors. As a result, they cannot work well in heterogeneous collaborative robotic systems that consist of aerial and ground robots. In this paper, we use heterogeneous robot systems, including UGV and unmanned aerial vehicles (UAV) to build an occupancy grid map that can be used for navigation. To fuse sensor data of different types, we propose a Collaborative Map Fusion algorithm based on Multi-task Gaussian Process Classification (MTGPC) using heterogeneous robotic systems. Besides, probabilistic model is exploited in traversability mapping, so the active perception can be used to build the map efficiently. Our system is tested in real scenes and can achieve an accuracy of more than 70%. The map fusion using active perception is better than map fusion using random strategy in terms of speed and accuracy. To our knowledge, this is the first work that can build the occupancy grid map using sparse data points sampled from aerial images and ground lidar map.

Estimation of the Energy Consumption of an All-Terrain Mobile Manipulator for Operations in Steep Vineyards

A heterogeneous robotic system that can perform various tasks in the steep vineyards of the Mediterranean region was developed and tested as part of the HEKTOR—Heterogeneous Autonomous Robotic System in Viticulture and Mariculture—project. This article describes the design of hardware and an easy-to-use method for evaluating the energy consumption of the system, as well as, indirectly, its deployment readiness level. The heterogeneous robotic system itself consisted of a flying robot—a light autonomous aerial robot (LAAR)—and a ground robot—an all-terrain mobile manipulator (ATMM), composed of an all-terrain mobile robot (ATMR) platform and a seven-degree-of-freedom (DoF) torque-controlled robotic arm. A formal approach to describe the topology and parameters of selected vineyards is presented. It is shown how Google Earth data can be used to make an initial estimation of energy consumption for a selected vineyard. On this basis, estimates of energy consumption were made for the tasks of protective spraying and bud rubbing. The experiments were conducted in two different vineyards, one with a moderate slope and the other with a much steeper slope, to evaluate the proposed estimation method.

Publish and Subscribe-Based Formation and Containment Control of Heterogeneous Robotic System with Actuator Time Delay

This paper addresses the problem of controlling a heterogeneous system composed of multiple Unmanned Aerial Vehicles (UAVs) and Autonomous Underwater Vehicles (AUVs) for formation and containment maintenance. The proposed approach considers actuator time delay and, in addition to formation and containment, considers obstacle avoidance, and offers a robust navigation algorithm and uses a reliable middleware for data transmission and exchange. The methodology followed uses both flocking technique and modified L1 adaptive control to ensure the proper navigation and coordination while avoiding obstacles. The data exchange between all the agents is provided through the data distribution services (DDS) middleware, which solves the interoperability issue when dealing with heterogeneous multiagent systems. The modified L1 controller is a local controller for stabilizing the dynamic model of each UAV and AUV, and the flocking approach is used to coordinate the followers around the leader or within the space delimited by their leaders. Potential Field (PF) allows obstacle avoidance during the agents’ movement. The performance of the proposed approach under the considerations mentioned above are verified and demonstrated using simulations.

Output Multiformation Tracking of Networked Heterogeneous Robotic Systems via Finite-Time Hierarchical Control.

This article investigates the finite-time output multiformation tracking (OMFT) problem of networked heterogeneous robotic systems (NHRSs), where each robot model involves external disturbances, parametric uncertainties, and possible kinematic redundancy. Besides, the interactions among robotic systems are described as a directed graph with an acyclic partition. Then, several novel practical finite-time hierarchical control (FTHC) algorithms are designed. The convergence analysis of the closed-loop dynamics is extremely difficult due to the lack of effective analysis methods. Based on the mathematics induction and reductio ad absurdum, a new nonsmooth Lyapunov function is proposed to derive the sufficient conditions and settling time functions. Finally, numerical simulations are performed on the NHRS to verify the main results.

Delivery path planning of heterogeneous robot system under road network constraints

This paper focuses on the path-planning problem of a heterogeneous robot system applied to parcel delivery in urban environments. The system consists of an unmanned ground vehicle (UGV) and an unmanned aerial vehicle (UAV). The UGV is restricted to drive in road network. The UAV unloads parcels from UGV and delivers them to customers. This study considers on the route schemes of UGV and UAV together on the basis of an air-ground cooperating. A two-step strategy combining ant colony optimization (ACO) and genetic algorithm (GA) is proposed to decouple their routes. In the first step, the ACO is used to search for UGV’s routes. In the second step, the GA is employed to solve the routes of UAV once the routes of UGV are pre-decided in the first step. Simulation results show that the method can effectively solve the heterogeneous delivering problem and obtain an optimal route of UAV.

An automated heterogeneous robotic system for radiation surveys: Design and field testing

AbstractDuring missions involving radiation exposure, unmanned robotic platforms may embody a valuable tool, especially thanks to their capability of replacing human operators in certain tasks to eliminate the health risks associated with such an environment. Moreover, rapid development of the technology allows us to increase the automation rate, making the human operator generally less important within the entire process. This article presents a multirobotic system designed for highly automated radiation mapping and source localization. Our approach includes a three‐phase procedure comprising sequential deployment of two diverse platforms, namely, an unmanned aircraft system (UAS) and an unmanned ground vehicle (UGV), to perform aerial photogrammetry, aerial radiation mapping, and terrestrial radiation mapping. The central idea is to produce a sparse dose rate map of the entire study site via the UAS and, subsequently, to perform detailed UGV‐based mapping in limited radiation‐contaminated regions. To accomplish these tasks, we designed numerous methods and data processing algorithms to facilitate, for example, digital elevation model‐based terrain following for the UAS, automatic selection of the regions of interest, obstacle map‐based UGV trajectory planning, and source localization. The overall usability of the multirobotic system was demonstrated by means of a 1‐day, authentic experiment, namely, a fictitious car accident including the loss of several radiation sources. The ability of the system to localize radiation hotspots and individual sources has been verified.