Robotics Research Community Research Articles

A deep residual reinforcement learning algorithm based on Soft Actor-Critic for autonomous navigation

The problem of autonomous navigation has attracted significant attention from robotics research community in the last few decades. In this paper, we address the problem of low data utilization due to the large amount of episode experience value data. A maximum entropy algorithm based on prioritized experience replay (Learning Good Experience based on Soft Actor-Criti, LGE-SAC) is proposed to quickly reproduce past good experience episodes. As the deep reinforcement learning method is susceptible to failure to plan ahead and explore the target position in a long sequence environment, a deep Residual Soft Actor-Critic (RSAC) is proposed to alleviate this problem. The reinforcement learning policy is fused with the Artificial Potential Field method to improve the generalization ability of the proposed algorithm, thus improving robot adaptation in new test environments. In order to validate the effectiveness of the proposed algorithm, we conducted simulation experiments in Gazebo simulator environment and real experiments on a Turtlebot3 robot equipped with LiDAR sensor. Simulation and experiment results show that the proposed algorithm effectively avoids obstacles and succeeds in reaching the goal compared to other obstacle avoidance algorithms. In comparison with the Artificial Potential Field method, the planning success rate of the proposed RSAC algorithm in the test environment is increased by 30%, and at the same time, the number of planning steps is reduced by half, and the generalization ability is improved.

DIG-SLAM: an accurate RGB-D SLAM based on instance segmentation and geometric clustering for dynamic indoor scenes

Simultaneous localization and mapping (SLAM) has emerged as a critical technology enabling robots to navigate in unknown environments, drawing extensive attention within the robotics research community. However, traditional visual SLAM ignores the presence of dynamic objects in indoor scenes, and dynamic point features of dynamic objects can lead to incorrect data correlation, making the traditional visual SLAM is difficult to accurately estimate the camera’s pose when the objects in the scenes are moving. Using only point features cannot fully extract geometric information in dynamic indoor scenes, reducing the system’s robustness. To solve this problem, we develop a RGB-D SLAM system called DIG-SLAM. Firstly, the objects’ contour regions are extracted using the YOLOv7 instance segmentation method, serving as a prerequisite for determining dynamic objects and constructing a semantic information map. Meanwhile, the line features are extracted using the line segment detector algorithm, and the redundant line features are optimized via K-means clustering. Secondly, moving consistency checks combined with instance partitioning determine dynamic regions, and the point and line features of the dynamic regions are removed. Finally, the combination of static line features and point features optimizes the camera pose. Meanwhile, a static semantic octree map is created to provide richer and higher-level scene understanding and perception capabilities for robots or autonomous systems. The experimental results on the Technische Universität München dataset show that the average absolute trajectory error of the developed DIG-SLAM is reduced by 28.68% compared with the dynamic semantic SLAM. Compared with other dynamic SLAM methods, the proposed system shows better camera pose estimation accuracy and system’s robustness in dynamic indoor environments and better map building in real indoor scenes.

If blockchain is the solution, robot security is the problem

Robotics systems of all types are revolutionizing a wide variety of industries—transportation, manufacturing, and even healthcare—and yet, many essential ingredients for robotics systems in the real world are not technologically ready for deployment. Currently, robots lack the protocols and standards required to be safe and secure outside factories. In an attempt to close this gap, recent research has demonstrated the security benefits of combining robotics systems with blockchain-based and related technologies (e.g., smart contracts, zero-knowledge proofs, Merkle trees). In this perspective article, I argue that blockchain-based robotics is starting to provide innovative solutions (e.g., secure data sharing, consensus mechanisms, and new interaction methods) to urgent problems of robot security. I list the most important takeaways so far from this emerging field of research that I helped establish together with a growing community. I close the article by discussing the implications of the security challenges that the robotics research community is facing, and possible ways for us to move forward.

A benchmark analysis of data‐driven and geometric approaches for robot ego‐motion estimation

AbstractIn the last decades, ego‐motion estimation or visual odometry (VO) has received a considerable amount of attention from the robotic research community, mainly due to its central importance in achieving robust localization and, as a consequence, autonomy. Different solutions have been explored, leading to a wide variety of approaches, mostly grounded on geometric methodologies and, more recently, on data‐driven paradigms. To guide researchers and practitioners in choosing the best VO method, different benchmark studies have been published. However, the majority of them compare only a small subset of the most popular approaches and, usually, on specific data sets or configurations. In contrast, in this work, we aim to provide a complete and thorough study of the most popular and best‐performing geometric and data‐driven solutions for VO. In our investigation, we considered several scenarios and environments, comparing the estimation accuracies and the role of the hyper‐parameters of the approaches selected, and analyzing the computational resources they require. Experiments and tests are performed on different data sets (both publicly available and self‐collected) and two different computational boards. The experimental results show pros and cons of the tested approaches under different perspectives. The geometric simultaneous localization and mapping methods are confirmed to be the best performing, while data‐driven approaches show robustness with respect to nonideal conditions present in more challenging scenarios.

Simulation of Reinforcement Learning Algorithm for Motion Control of an Autonomous Humanoid

Autonomy is an issue in robotics systems. Currently the robotics research community is focusing on autonomy in the decision. The pre-programmed humanoid robots perform the operation in a known scenario. The presence of an obscure environment, a lack of awareness of the environment, humanoid robot fails to perform the random task. In such cases, the preprogrammed robot needs to be reprogrammed to enable it to perform in the changing environment. There is very limited success achieved in the autonomy of the decision-making process. This research work considered the problem of an autonomy while the deicion making in th real time interaction. The reinforcement algorithm helps to do the task in such type of unstructured and unknown environment. Reinforcement learning problems are categorized into partial Markov Decision Processes (MDP). The goal of RL agent is to minimize its immediate and expected costs. When the system interacts with the Markov Decision Process, RL agent passes through an intermediate sequence of states that depends on another by transition probabilities. The agents action takes, and the agents experience a sequence of immediate costs incurred. Reinforcement Learning and teaching approach like Queue Learning (Q-Learning) is implemented for humanoid robot for navigation and exploration. The Q-learning expresses the expected costs to go of a state action pair defined, which is meant to express the expected costs arising after having taken action in the state following policy. Based on the optimal policy of the reinforcement algorithm, a reinforcement controller was implemented. The transition probabilities of the controller depend on the randomness of the controller. The random values of the controller decide the action. Simulations were carried out for the different positions of the proposed model, and an interesting result were was observed while the transition from the sitting position to the goal position.

Embodied Intelligence & Morphological Computation in Soft Robotics Community: Collaborations, Coordination, and Perspective

The agile nature of physical interactions in animal and plant species has inspired many recent advances in robotics and their control frameworks. However, they still face challenges in interaction with ever-changing unconstructured world that we live in. Intelligence is one of nature’s survival solutions for biological creatures to adapt to and reshape their surroundings. Our robots are no different in these remits. An important key to their survival and effectiveness in the natural world is the concept of Artificial Intelligence (AI). In this chapter we provide a brief overview of the rapidly emerging Soft Robotics research community around AI relevant concepts such as Embodied intelligence and Morphological Computation. More specifically, we focused on the importance of setting such “community goals” to create a diverse interdisciplinary research environment, an enormously important element to keep up with our rapidly progressing world. To this end, we focused on the collaborations withing and between the communities, impacts of the recently established IEEE Soft Robotics Technical Committee on coordinating these efforts, and most important of all, the key ideas and perspectives based on 200 interviews with researchers across different fields.

Framework and Evaluation Methodology for Autonomous Drone Racing

In recent years, autonomous drone races have become increasingly popular in the aerial robotics research community due to challenges in perception, localization, navigation, and control at high speeds, pushing forward the state-of-the-art every year. However, autonomous racing drones are still far from reaching human pilot performance, and a lot of research has to be done to accomplish that. In this work, a complete architecture system and an evaluation method for autonomous drone racing research are proposed, based on the Aerostack 5.0 open-source framework. To evaluate the performance of the whole system and of each algorithm used separately, this framework is validated not only with simulated flights, but also through real flights in an indoor drone race circuit using different configurations.

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

Robotic-assisted surgery is now well established in clinical practice and has become the gold-standard clinical treatment option for several clinical indications. The field of robotic-assisted surgery is expected to grow substantially in the next decade, with a range of new robotic devices emerging to address unmet clinical needs across different specialties. A vibrant surgical robotics research community is pivotal for conceptualizing such new systems as well as for developing and training the engineers and scientists to translate them into practice. The da Vinci Research Kit (dVRK), an academic and industry collaborative effort to repurpose decommissioned da Vinci surgical systems [Intuitive Surgical Inc. (ISI), California, USA] as a research platform for surgical robotics research, has been a key initiative for addressing a barrier to entry for new research groups in surgical robotics. In this article, we present an extensive review of the publications that have been facilitated by the dVRK over the past decade. We classify research efforts into different categories and outline some of the major challenges and needs for the robotics community to maintain and build upon this initiative.

On Differential Mechanisms for Underactuated, Lightweight, Adaptive Prosthetic Hands.

Over the last decade underactuated, adaptive robot grippers and hands have received an increased interest from the robotics research community. This class of robotic end-effectors can be used in many different fields and scenarios with a very promising application being the development of prosthetic devices. Their suitability for the development of such devices is attributed to the utilization of underactuation that provides increased functionality and dexterity with reduced weight, cost, and control complexity. The most critical components of underactuated, adaptive hands that allow them to perform a broad set of grasp poses are appropriate differential mechanisms that facilitate the actuation of multiple degrees of freedom using a single motor. In this work, we focus on the design, analysis, and experimental validation of a four output geared differential, a series elastic differential, and a whiffletree differential that can incorporate a series of manual and automated locking mechanisms. The locking mechanisms have been developed so as to enhance the control of the differential outputs, allowing for efficient grasp selection with a minimal set of actuators. The differential mechanisms are applied to prosthetic hands, comparing them and describing the benefits and the disadvantages of each.

Deep Learning Point Cloud Odometry

Achieving persistent and reliable autonomy for mobile robots in challenging field mission scenarios is a long-time quest for the Robotics research community. Deep learning-based LIDAR odometry is attracting increasing research interest as a technological solution for the robot navigation problem and showing great potential for the task.In this work, an examination of the benefits of leveraging learning-based encoding representations of real-world data is provided. In addition, a broad perspective of emergent Deep Learning robust techniques to track motion and estimate scene structure for real-world applications is the focus of a deeper analysis and comprehensive comparison.Furthermore, existing Deep Learning approaches and techniques for point cloud odometry tasks are explored, and the main technological solutions are compared and discussed.Open challenges are also laid out for the reader, hopefully offering guidance to future researchers in their quest to apply deep learning to complex 3D non-matrix data to tackle localization and robot navigation problems.

Enhancing continuous control of mobile robots for end-to-end visual active tracking

In the last decades, visual target tracking has been one of the primary research interests of the Robotics research community. The recent advances in Deep Learning technologies have made the exploitation of visual tracking approaches effective and possible in a wide variety of applications, ranging from automotive to surveillance and human assistance. However, the majority of the existing works focus exclusively on passive visual tracking, i.e., tracking elements in sequences of images by assuming that no actions can be taken to adapt the camera position to the motion of the tracked entity. On the contrary, in this work, we address visual active tracking, in which the tracker has to actively search for and track a specified target. Current State-of-the-Art approaches use Deep Reinforcement Learning (DRL) techniques to address the problem in an end-to-end manner. However, two main problems arise: (i) most of the contributions focus only on discrete action spaces, and the ones that consider continuous control do not achieve the same level of performance; and (ii) if not properly tuned, DRL models can be challenging to train, resulting in considerably slow learning progress and poor final performance. To address these challenges, we propose a novel DRL-based visual active tracking system that provides continuous action policies. To accelerate training and improve the overall performance, we introduce additional objective functions and a Heuristic Trajectory Generator (HTG) to facilitate learning. Through extensive experimentation, we show that our method can reach and surpass other State-of-the-Art approaches performances, and demonstrate that, even if trained exclusively in simulation, it can successfully perform visual active tracking even in real scenarios.

Bootstrapping MDE development from ROS manual code: Part 2\u2014Model generation and leveraging models at runtime

Model-driven engineering (MDE) addresses central aspects of robotics software development. MDE could enable domain experts to leverage the expressiveness of models, while implementation details on different hardware platforms would be handled by automatic code generation. Today, despite strong MDE efforts in the robotics research community, most evidence points to manual code development being the norm. A possible reason for MDE not being accepted by robot software developers could be the wide range of applications and target platforms, which make all-encompassing MDE IDEs hard to develop and maintain. Therefore, we chose to leverage a large corpus of open-source software widely adopted by the robotics community to extract common structures and gain insight on how and where MDE can support the developers to work more efficiently. We pursue modeling as a complement, rather than imposing MDE as separate solution. Our previous work introduced metamodels to describe components, their interactions, and their resulting composition. In this paper, we present two methods based on metamodels for automated generation of models from manually written artifacts: (1) through static code analysis and (2) by monitoring the execution of a running system. For both methods, we present tools that leverage the potentials of our contributions, with a special focus on their application at runtime to observe and diagnose a real system during its execution. A comprehensive example is provided as a walk-through for robotics software practitioners.

Dynamic Balance of the Head in a Flexible Legged Robot for Efficient Biped Locomotion

In the biped robotics domain, head oscillations may be extremely harmful, especially if the robot is teleoperated, since vibrations strongly reduce the operator’s spatial awareness. In particular, undesired head oscillations occur in under-actuated robots, where springs and passive mechanisms are used to achieve a human-like motion. This paper proposes an approach to reduce the vibrations of a biped robot’s head; the proposed solution does not affect the dynamic locomotion properties, on which specific control logic could have been already tuned. The approach is tested on Rollo, a flexible-biped-wheeled robot, whose head vibrates throughout the robot locomotion. The two requirements, i.e., head vibration reduction and unchanged Rollo locomotion properties, are traduced in constraints to the robot possible modifications. Based on a 1D finite element model of the robot, tuned on experimental modal analysis, the undesired vibration causes are detected, and a solution for their reduction is proposed. Rollo’s head vibration amplitude is attenuated using a tuned vibration absorber, which achieves impressive performance in the robot. An archetype of the proposed vibration absorber is tailored designed on Rollo, without invasive changes to the robot structure. The proposed approach solves a significant problem in the biped robotic research community. The approach used to reduce the Rollo head oscillations may be utilized in other biped robot machines with or without flexible legs.

Scaling Up Soft Robotics: A Meter-Scale, Modular, and Reconfigurable Soft Robotic System.

Today's use of large-scale industrial robots is enabling extraordinary achievement on the assembly line, but these robots remain isolated from the humans on the factory floor because they are very powerful, and thus dangerous to be around. In contrast, the soft robotics research community has proposed soft robots that are safe for human environments. The current state of the art enables the creation of small-scale soft robotic devices. In this article we address the gap between small-scale soft robots and the need for human-sized safe robots by introducing a new soft robotic module and multiple human-scale robot configurations based on this module. We tackle large-scale soft robots by presenting a modular and reconfigurable soft robotic platform that can be used to build fully functional and untethered meter-scale soft robots. These findings indicate that a new wave of human-scale soft robots can be an alternative to classic rigid-bodied robots in tasks and environments where humans and machines can work side by side with capabilities that include, but are not limited to, autonomous legged locomotion and grasping.

TMTDyn: A Matlab package for modeling and control of hybrid rigid–continuum robots based on discretized lumped systems and reduced-order models

A reliable, accurate, and yet simple dynamic model is important to analyzing, designing, and controlling hybrid rigid–continuum robots. Such models should be fast, as simple as possible, and user-friendly to be widely accepted by the ever-growing robotics research community. In this study, we introduce two new modeling methods for continuum manipulators: a general reduced-order model (ROM) and a discretized model with absolute states and Euler–Bernoulli beam segments (EBA). In addition, a new formulation is presented for a recently introduced discretized model based on Euler–Bernoulli beam segments and relative states (EBR). We implement these models in a Matlab software package, named TMTDyn, to develop a modeling tool for hybrid rigid–continuum systems. The package features a new high-level language (HLL) text-based interface, a CAD-file import module, automatic formation of the system equation of motion (EOM) for different modeling and control tasks, implementing Matlab C-mex functionality for improved performance, and modules for static and linear modal analysis of a hybrid system. The underlying theory and software package are validated for modeling experimental results for (i) dynamics of a continuum appendage, and (ii) general deformation of a fabric sleeve worn by a rigid link pendulum. A comparison shows higher simulation accuracy (8–14% normalized error) and numerical robustness of the ROM model for a system with a small number of states, and computational efficiency of the EBA model with near real-time performances that makes it suitable for large systems. The challenges and necessary modules to further automate the design and analysis of hybrid systems with a large number of states are briefly discussed.

Benchmarking Cluttered Robot Pick-and-Place Manipulation With the Box and Blocks Test

In this work, we propose a pick-and-place benchmark to assess the manipulation capabilities of a robotic system. The benchmark is based on the Box and Blocks Test (BBT), a task utilized for decades by the rehabilitation community to assess unilateral gross manual dexterity in humans. We propose three robot benchmarking protocols in this work that hold true to the spirit of the original clinical tests—the Modified-BBT, the Targeted-BBT, and the Standard-BBT. These protocols can be implemented by the greater robotics research community, as the physical BBT setup has been widely distributed with the Yale-CMU-Berkeley (YCB) Object and Model Set. Difficulty of the three protocols increase sequentially, adding a new performance component at each level, and therefore aiming to assess various aspects of the system separately. Clinical task-time norms are summarized for able-bodied human participants. We provide baselines for all three protocols with off-the-shelf planning and perception algorithms on a Barrett WAM and a Franka Emika Panda manipulator, and compare results with human performance.

Guided Policy Search for Sequential Multitask Learning

Policy search in reinforcement learning (RL) is a practical approach to interact directly with environments in parameter spaces, that often deal with dilemmas of local optima and real-time sample collection. A promising algorithm, known as guided policy search (GPS), is capable of handling the challenge of training samples using trajectory-centric methods. It can also provide asymptotic local convergence guarantees. However, in its current form, the GPS algorithm cannot operate in sequential multitask learning scenarios. This is due to its batch-style training requirement, where all training samples are collectively provided at the start of the learning process. The algorithm’s adaptation is thus hindered for real-time applications, where training samples or tasks can arrive randomly. In this paper, the GPS approach is reformulated, by adapting a recently proposed, lifelong-learning method, and elastic weight consolidation. Specifically, Fisher information is incorporated to impart knowledge from previously learned tasks. The proposed algorithm, termed sequential multitask learning-GPS, is able to operate in sequential multitask learning settings and ensuring continuous policy learning, without catastrophic forgetting. Pendulum and robotic manipulation experiments demonstrate the new algorithms efficacy to learn control policies for handling sequentially arriving training samples, delivering comparable performance to the traditional, and batch-based GPS algorithm. In conclusion, the proposed algorithm is posited as a new benchmark for the real-time RL and robotics research community.

Maplab: An Open Framework for Research in Visual-Inertial Mapping and Localization

Robust and accurate visual-inertial estimation is crucial to many of today's challenges in robotics. Being able to localize against a prior map and obtain accurate and driftfree pose estimates can push the applicability of such systems even further. Most of the currently available solutions, however, either focus on a single session use-case, lack localization capabilities or an end-to-end pipeline. We believe that only a complete system, combining state-of-the-art algorithms, scalable multi-session mapping tools, and a flexible user interface, can become an efficient research platform. We therefore present maplab, an open, research-oriented visual-inertial mapping framework for processing and manipulating multi-session maps, written in C++. On the one hand, maplab can be seen as a ready-to-use visual-inertial mapping and localization system. On the other hand, maplab provides the research community with a collection of multisession mapping tools that include map merging, visual-inertial batch optimization, and loop closure. Furthermore, it includes an online frontend that can create visual-inertial maps and also track a global drift-free pose within a localization map. In this paper, we present the system architecture, five use-cases, and evaluations of the system on public datasets. The source code of maplab is freely available for the benefit of the robotics research community.

LightDenseYOLO: A Fast and Accurate Marker Tracker for Autonomous UAV Landing by Visible Light Camera Sensor on Drone.

Autonomous landing of an unmanned aerial vehicle or a drone is a challenging problem for the robotics research community. Previous researchers have attempted to solve this problem by combining multiple sensors such as global positioning system (GPS) receivers, inertial measurement unit, and multiple camera systems. Although these approaches successfully estimate an unmanned aerial vehicle location during landing, many calibration processes are required to achieve good detection accuracy. In addition, cases where drones operate in heterogeneous areas with no GPS signal should be considered. To overcome these problems, we determined how to safely land a drone in a GPS-denied environment using our remote-marker-based tracking algorithm based on a single visible-light-camera sensor. Instead of using hand-crafted features, our algorithm includes a convolutional neural network named lightDenseYOLO to extract trained features from an input image to predict a marker’s location by visible light camera sensor on drone. Experimental results show that our method significantly outperforms state-of-the-art object trackers both using and not using convolutional neural network in terms of both accuracy and processing time.

On Fault Detection and Diagnosis in Robotic Systems

The use of robots in our daily lives is increasing. Different types of robots perform different tasks that are too dangerous or too dull to be done by humans. These sophisticated machines are susceptible to different types of faults. These faults have to be detected and diagnosed in time to allow recovery and continuous operation. The field of Fault Detection and Diagnosis (FDD) has been studied for many years. This research has given birth to many approaches and techniques that are applicable to different types of physical machines. Yet the domain of robotics poses unique requirements that are very challenging for traditional FDD approaches. The study of FDD for robotics is relatively new, and only few surveys were presented. These surveys have focused on traditional FDD approaches and how these approaches may broadly apply to a generic type of robot. Yet robotic systems can be identified by fundamental characteristics, which pose different constraints and requirements from FDD. In this article, we aim to provide the reader with useful insights regarding the use of FDD approaches that best suit the different characteristics of robotic systems. We elaborate on the advantages these approaches have and the challenges they must face. To meet this aim, we use two perspectives: (1) we elaborate on FDD from the perspective of the different characteristics a robotic system may have and give examples of successful FDD approaches, and (2) we elaborate on FDD from the perspective of the different FDD approaches and analyze the advantages and disadvantages of each approach with respect to robotic systems. Finally, we describe research opportunities for robotic systems’ FDD. With these three contributions, readers from the FDD research communities are introduced to FDD for robotic systems, and the robotics research community is introduced to the field of FDD.