Autonomous Robot Operating Research Articles

FEDERATED LEARNING FOR VISION-BASED OBSTACLE AVOIDANCE IN MOBILE ROBOTS

Federated learning (FL) is a machine learning approach that allows multiple devices or systems to train a model collaboratively, without exchanging their data. This is particularly useful for autonomous mobile robots, as it allows them to train models customized to their specific environment and tasks, while keeping the data they collect private. Research Objective to train a model to recognize and classify different types of objects, or to navigate around obstacles in its environment. Materials and me¬thods we used FL to train models for a variety of tasks, such as object recognition, obstacle avoidance, localization, and path planning by an autonomous mobile robot operating in a warehouse FL. We equipped the robot with sensors and a processor to collect data and perform machine learning tasks. The robot must communicate with a central server or cloud platform that coordinates the training process and collects model updates from different devices. We trained a neural network (CNN) and used a PID algorithm to generate a control signal that adjusts the position or other variable of the system based on the difference between the desired and actual values, using the relative, integrative and derivative terms to achieve the desired performance. Results through careful design and execution, there are several challenges to implementing FL in autonomous mobile robots, including the need to ensure data privacy and security, and the need to manage communications and the computational resources needed to train the model. Conclusion. We conclude that FL enables autonomous mobile robots to continuously improve their performance and adapt to changing environments and potentially improve the performance of vision-based obstacle avoidance strategies and enable them to learn and adapt more quickly and effectively, leading to more robust and autonomous systems.

Reconstructing Complete Human Bodies Using Limited Scans Obtained via Viewpoint Auto-Changing Oriented Toward Rescue Operations

When disasters such as earthquakes and collapses occur, rescuers must go deep into dangerous regions and rush to transport the wounded. Long-term rescue operations in dangerous environments challenge the rescuers’ physical strength and threaten their lives. We propose a human-body reconstruction method for autonomous rescue operations with a wounded rescue robot. This method constructs a human body model that can support a robot’s autonomous rescue operations given limited scans of the injured. Most casualties lie on the ground in supine positions, so this scanning limit is typical. We propose a scanning viewpoint planning method using deep reinforcement learning and change the viewpoint with the help of a 6-degree-of-freedom (DOF) robotic arm to obtain a complete scan of the visible surface. For successful autonomous casualty rescue, a robot must have access to a watertight human body mesh with precise surfaces and associated semantics. To this end, we develop a neural network that predicts a completely clothed body mesh from visible point clouds. The results of experiments show that our method can reconstruct a watertight body mesh with accurate surfaces and human semantics to satisfy the needs of autonomous robotic casualty rescue operations under scanning limitations.

Advancements in autonomous mobility of planetary wheeled mobile robots: A review

Mobility analysis is crucial to fast, safe, and autonomous operation of planetary Wheeled Mobile Robots (WMRs). This paper reviews implemented odometry techniques on currently designed planetary WMRs and surveys methods for improving their mobility and traversability. The methods are categorized based on the employed approaches ranging from signal-based and model-based estimation to terramechanics-based, machine learning, and global sensing techniques. They aim to detect vehicle motion parameters (kinematic states and forces/torques), terrain hazards (slip and sinkage) and terrain parameters (soil cohesion and friction). The limitations of these methods and recommendations for future missions are stated.

Robust Structure Identification and Room Segmentation of Cluttered Indoor Environments From Occupancy Grid Maps

Identifying the environment’s structure, through detecting core components such as rooms and walls, can facilitate several tasks fundamental for the successful operation of indoor autonomous mobile robots, including semantic environment understanding. These robots often rely on 2D occupancy maps for core tasks such as localisation and motion and task planning. However, reliable identification of structure and room segmentation from 2D occupancy maps is still an open problem due to clutter (e.g., furniture and movable objects), occlusions, and partial coverage. We propose a method for the RObust StructurE identification and ROom SEgmentation (ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> ) of 2D occupancy maps that may be cluttered and incomplete. ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> identifies the main directions of walls and is resilient to clutter and partial observations, allowing to extract a clean, abstract geometrical floor-plan-like description of the environment, which is used to segment, i.e., to identify rooms in, the original occupancy grid map. ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> is tested in several real-world publicly available cluttered maps obtained in different conditions. The results show that it can robustly identify the environment structure in 2D occupancy maps suffering from clutter and partial observations, while significantly improving room segmentation accuracy. Thanks to the combination of clutter removal and robust room segmentation, ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> consistently achieves higher performance than the state-of-the-art methods, against which it is compared.

ENERGY GENERATOR PROTOTYPES DEVELOPMENT AND THESE RESEARCH INTEGRATION INTO THE EDUCATIONAL PROCESS

The paper presents the preconditions for improving the quality of technical training of engineers by integrating experimental research based on patented technology into the power supply circuits of autonomous robotic systems to improve power consumption and regeneration rate, ensuring longer autonomous robot operation. Not only the topics of arousal of curiosity in researching technical nuances are discussed, but also the promotion of motivation to delve into the study subjects envisaged in the curriculum and their integral understanding by examining the interrelationships. After-school activities are discussed: participation in the research, analysis of the research methodology, preparation of the layout for research and measurements; and the impact of these activities on the overall educational process. Suggestions for the integration of research results into the practical educational process of students are presented.

Leaf-Like Origami with Bistability for Self-Adaptive Grasping Motions.

The leaf-like origami structure is inspired by geometric patterns found in nature, exhibiting unique transitions between open and closed shapes. With a bistable energy landscape, leaf-like origami is able to replicate the autonomous grasping of objects observed in biological systems such as the Venus flytrap. We show uniform grasping motions of the leaf-like origami, as well as various nonuniform grasping motions that arise from its multitransformable nature. Grasping motions can be triggered with high tunability due to the structure's bistable energy landscape. We demonstrate the self-adaptive grasping motion by dropping a target object onto our paper prototype, which does not require an external power source to retain the capture of the object. We also explore the nonuniform grasping motions of the leaf-like structure by selectively controlling the creases, which reveals various unique grasping configurations that can be exploited for versatile, autonomous, and self-adaptive robotic operations.

A robotic wheel locally transforming its diameters and the reinforcement learning for robust locomotion

The implementation of the neural network has been paid attention in the autonomous operation of robots. In particular, it is efficient for a robot itself to learn the locomoting method to get over different obstacles on rough terrains. We are developing a robotic wheel that can locomote stably even on rough terrain, and introduce the reinforcement learning for the ability to robustly get over an obstacle. Our robot is able to locomote by utilising the extension and returning of the diameters by moving its centre of gravity. We study its mobility through four experiments, which are the testing of the locomotion on flat ground, the climbing over a step, controlling the robotic wheel by IMU, and the braking performance. After the learning, we verify the performance of getting over a step of 10 cm and 20 cm, which are equivalent to 25% and 50% of the wheel diameter, respectively.

NASA Space Robotics Challenge 2 Qualification Round: An Approach to Autonomous Lunar Rover Operations

Plans for establishing a long-term human presence on the Moon will require substantial increases in robot autonomy and multirobot coordination to support establishing a lunar outpost. To achieve these objectives, algorithm design choices for the software developments need to be tested and validated for expected scenarios such as autonomous in situ resource utilization, localization in challenging environments, and multirobot coordination. However, real-world experiments are extremely challenging and limited for extraterrestrial environment. Also, realistic simulation demonstrations in these environments are still rare and demanded for initial algorithm testing capabilities. To help some of these needs, the NASA Centennial Challenges program established the Space Robotics Challenge Phase 2 (SRC2), which consist of virtual robotic systems in a realistic lunar simulation environment, where a group of mobile robots were tasked with reporting volatile locations within a global map, excavating and transporting these resources, and detecting and localizing a target of interest. The main goal of this article is to share our team's experiences on the design tradeoffs to perform autonomous robotic operations in a virtual lunar environment and to share strategies to complete the mission requirements posed by NASA SRC2 competition during the qualification round. Of the 114 teams that registered for participation in the NASA SRC2, team Mountaineers finished as one of only six teams to receive the top qualification round prize.

Grasp synthesis of continuum robots

Despite the importance of grasp synthesis for autonomous robotic operations, its formulation for continuum robots (CRs) has yet to be investigated. This paper presents a grasp taxonomy and a synthesis approach for CRs. The proposed synthesis relies on an analytical model for grasping using CRs. The method is extended for the application of cooperative continuum robots (CCRs). To present a comprehensive formulation of the problem, both constant-curvature and Cosserat-rod models are adopted in the proposed grasp synthesis. A set of grasp quality measures are formulated for this study, in which a new quantitative grasp quality measure is introduced to reflect the limited workspace of CRs. Finally, two experimental grasp quality measures are introduced (i.e., path-following error and grasp success rate) to compare and assess the grasps using statistical tests. The effectiveness of the proposed methodology is shown through extensive simulations and experiments using single-segment tendon-driven catheters.

Human Diver-Inspired Visual Navigation: Towards Coverage Path Planning of Shipwrecks

Abstract This work proposes vision-only navigation strategies for an autonomous underwater robot. This approach is a step towards solving the coverage path planning problem in a 3-D environment for surveying underwater structures. Given the challenging conditions of the underwater domain, it is very complicated to obtain accurate state estimates reliably. Consequently, it is a great challenge to extend known path planning or coverage techniques developed for aerial or ground robot controls. In this work, we are investigating a navigation strategy utilizing only vision to assist in covering a complex underwater structure. We propose to use a navigation strategy akin to what a human diver will execute when circumnavigating around a region of interest, in particular when collecting data from a shipwreck. The focus of this article is a step towards enabling the autonomous operation of lightweight robots near underwater wrecks in order to collect data for creating photo-realistic maps and volumetric 3-D models while at the same time avoiding collisions. The proposed method uses convolutional neural networks to learn the control commands based on the visual input. We have demonstrated the feasibility of using a system based only on vision to learn specific strategies of navigation with 80% accuracy on the prediction of control command changes. Experimental results and a detailed overview of the proposed method are discussed.

Pitch‐Black Nanostructured Copper Oxide as an Alternative to Carbon Black for Autonomous Environments

Light detection and ranging (LiDAR) systems are becoming crucial for measuring the distance and creating a point cloud of the local environment, critical data for artificial intelligence to enable collision‐avoidance mechanisms. However, LiDAR utilizes radiation in the near‐infrared (NIR) region of the electromagnetic spectrum, which is prone to complete absorption by typical dark (such as painted by carbon black) colored objects, leading to loss of timely data points. Till date a very limited number of solutions have been put forward to address this. Herein, nanocrystallites of copper (II) oxide with specific prevalence of crystal facets that create nearly perfect black material at visible wavelengths are proposed. The sharp transition of absorbance near 700 nm wavelength light is attributed to the near‐unity ratio of (−111)/(111) the crystal facets and a crystal size of around 100 Å for the (−111) plane. Although indistinguishable from carbon black and with the same degree of measured blackness (My value 135.5), the nanocrystalline CuO shows 1500% better detectability by LiDAR. The study paves the way for the unconstrained use of dark objects in future society and infrastructure, moving a step closer toward fully autonomous operation of vehicles and robots.

Human-Aware Reinforcement Learning for Fault Recovery Using Contextual Gaussian Processes

This work addresses the iterated nonstationary assistant selection problem, in which over the course of repeated interactions on a mission, an autonomous robot experiencing a fault must select a single human from among a group of assistants to restore it to operation. The assistants in our problem have a level of performance that changes as a function of their experience solving the problem. Our approach uses reinforcement learning via a multi-arm bandit formulation to learn about the capabilities of each potential human assistant and decide which human to task. This study, which is built on our past work, evaluates the potential for a Gaussian-process-based machine learning method to effectively model the complex dynamics associated with human learning and forgetting. Application of our method in simulation shows that our method is capable of tracking performance of human-like dynamics for learning and forgetting. Using a novel selection policy called the proficiency window, it is shown that our technique can outperform baseline selection strategies while providing guarantees on human use. Our work offers an effective potential alternative to dedicated human supervisors, with application to any human–robot system where a set of humans is responsible for overseeing autonomous robot operations.

Modest-vocabulary loop-closure detection with incremental bag of tracked words

A key feature in the context of simultaneous localization and mapping is loop-closure detection, a process determining whether the current robot’s environment perception coincides with previous observation. However, in long-term operations, both computational efficiency and memory requirements involved in an autonomous robot operation in uncontrolled environments, are of particular importance. The majority of approaches scale linearly with the environment’s size in terms of storage and query time. The article at hand presents an efficient appearance-based loop-closure detection pipeline, which encodes the traversed trajectory by a low amount of unique visual words generated on-line through feature tracking. The incrementally constructed visual vocabulary is referred to as the “Bag of Tracked Words.” A nearest-neighbor voting scheme is utilized to query the database and assign probabilistic scores to all visited locations. Exploiting the inherent temporal coherency in the loop-closure task, the produced scores are processed through a Bayesian filter to estimate the belief state about the robot’s location on the map. Also, a geometrical verification step ensures consistency between image matches. Management is also applied to the resulting vocabulary to reduce its growth rate and constraint the system’s computational complexity while improving its voting distinctiveness. The proposed approach’s performance is experimentally evaluated on several publicly available and challenging datasets, including hand-held, car-mounted, aerial, and ground trajectories. Results demonstrate the method’s adaptability, which retains high operational frequency in environments of up to 13 km and high recall rates for perfect precision, outperforming other state-of-the-art techniques. The system’s effectiveness is owed to the reduced vocabulary size, which is at least one order of magnitude smaller than other contemporary approaches. An open research-oriented source code has been made publicly available, which is dubbed as “BoTW-LCD.”

Contrast Signal to Noise Ratio

The detection and recognition of objects is essential for the operation of autonomous vehicles and robots. Designing and predicting the performance of camera systems intended to supply information to neural networks and vision algorithms is nontrivial. Optimization has to occur across many parameters, such as focal length, f-number, pixel and sensor size, exposure regime and transmission schemes. As such numerous metrics are being explored to assist with these design choices. Detectability index (SNRI) is derived from signal detection theory as applied to imaging systems and is used to estimate the ability of a system to statistically distinguish objects [1], most notably in the medical imaging and defense fields [2]. A new metric is proposed, Contrast Signal to Noise Ratio (CSNR), which is calculated simply as mean contrast divided by the standard deviation of the contrast. This is distinct from contrast to noise ratio which uses the noise of the image as the denominator [3,4]. It is shown mathematically that the metric is proportional to the idealized observer for a cobblestone target and a constant may be calculated to estimate SNRI from CSNR, accounting for target size. Results are further compared to Contrast Detection Probability (CDP), which is a relatively new objective image quality metric proposed within IEEE P2020 to rank the performance of camera systems intended for use in autonomous vehicles [5]. CSNR is shown to generate information in illumination and contrast conditions where CDP saturates and further can be modified to provide CDP-like results.

Mobile Robot Simulation and Navigation in ROS and Gazebo

mobile robots are entering our daily lives as wellas in the industry. Their task is usually associated with carryingout transportation. This leads to the need to performautonomous movement of mobile robots. On the other hand,modern practice is that the planning of most processes is donethrough simulations. Thus, various future production problemscan be anticipated and remedied or improved. The articledescribes the creation of a mobile robot model in the Gazebosimulation environment. Specific settings and features forrunning a mobile robot in autonomous navigation mode underthe robot operating system are presented. The steps for creatinga map, localization and navigation are presented. Experimentshave been conducted to optimize and tune the parameters ofboth the robot model itself and the simulation controlparameters.

FeatFlow: Learning geometric features for 3D motion estimation

3D motion estimation is an important prerequisite for the autonomous operation of vehicles and robots in dynamic environments. This work presents FeatFlow, a novel neural network architecture to estimate 3D motions from unstructured point clouds. Specifically, we learn deep geometric features to estimate the dense scene flow and the ego-motion of the platform. We build a scene flow estimation pipeline by an encoder-decoder architecture which comprises three novel modules: feature extractor, motion embedder, and flow decoder. By using a point-score layer to assign scores to the extracted features in a learning procedure, the feature extractor effectively extracts keypoints and features that are most significant for estimating the relative transformation between two consecutive point clouds. The whole model adaptively learns the required robust descriptors to represent a variety of point motions at the object or scene level. We evaluated our approach on synthetic data from FlyingThings3D, and real-world LiDAR scans from KITTI and Oxford RobotCar. Our network successfully generalizes to datasets with different patterns, outperforming various baselines and achieving state-of-the-art performance.

Legal Aspects of Production and Operation of Autonomous Combat Robots

Mobile robots are increasingly replacing people in particularly dangerous situations and those at risk of losing their lives or health. One of such applications are military and preventive activities, where mobile robots are not only used for reconnaissance and patrol operations and the development trend is focused on the increasing use of mobile robots on the battlefield. Therefore, this type of equipment must meet the requirements and standards guaranteeing safety for the user and the environment. The paper presents a set of requirements for the production and use of autonomous combat vehicles put up by organizations such as the Defence Department, the European Union and NATO.

Everybody Needs Somebody Sometimes: Validation of Adaptive Recovery in Robotic Space Operations

This letter assesses an adaptive approach to fault recovery in autonomous robotic space operations, which uses indicators of opportunity, such as physiological state measurements and observations of past human assistant performance, to inform future selections. We validated our reinforcement learning approach using data we collected from humans executing simulated mission scenarios. We present a method of structuring human-factors experiments that permits collection of relevant indicator of opportunity and assigned assistance task performance data, as well as evaluation of our adaptive approach, without requiring large numbers of test subjects. Application of our reinforcement learning algorithm to our experimental data shows that our adaptive assistant selection approach can achieve lower cumulative regret compared to existing nonadaptive baseline approaches when using real human data. Our work has applications beyond space robotics to any application where autonomy failures may occur that require external intervention.

Recognition and Pose Estimation of Auto Parts for an Autonomous Spray Painting Robot

The autonomous operation of industrial robots with minimal human supervision has always been in high demand. To prepare the autonomous operation of a car part spray painting robot, novel object detection, and pose estimation algorithms have been developed in this paper. The object detection part used principal components analysis (PCA) to reduce the dimension of three-dimensional (3-D) point cloud to 2-D binary image. Distance measure between the auto and cross correlation of the binary features was established to find out the similarity between them. Resultantly, the type of auto part was successfully obtained. Furthermore, iterative closest point (ICP) algorithm was used to estimate the pose difference of the auto part with respect to the camera reference frame, which was mounted on the robot. An issue with ICP's lack of robustness to local minimum was solved by the combination of ICP and genetic algorithm (GA). This allowed the optimization of pose error and addressed the problem of local minimum entrapment in ICP. For experimental validation: the proposed object recognition pipeline was implemented in both serial and parallel programming paradigms. The results were obtained for the acquired point clouds of side body car parts and compared with the major 3-D object detection systems in terms of computational cost. Pose estimation error was calculated with both ICP and the modified point set registration schemes, and it was shown to be decreasing in the case of later. All shown results supported the research claims.

Real-Time Path-Generation and Path-Following Using an Interoperable Multi-Agent Framework

Autonomous unmanned vehicles are preferable in patrolling, surveillance and, search and rescue missions. Multi-agent architectures are commonly used for autonomous control of unmanned vehicles. Existing multi-robot architectures for unmanned aerial and ground robots are generally mission and platform oriented. Collision avoidance, path-planning and tracking are some of the fundamental requirements for autonomous operation of unmanned robots. Though aerial and ground vehicles operate differently, the algorithms for obstacle avoidance, path-planning and path-tracking can be generalized. Service Oriented Interoperable Framework for Robot Autonomy (SOIFRA) proposed in this work is an interoperable multi-agent framework focused on generalizing platform independent algorithms for unmanned aerial and ground vehicles. SOIFRA is behavior-based, modular and interoperable across unmanned aerial and ground vehicles. SOIFRA provides collision avoidance, and, path-planning and tracking behaviors for unmanned aerial and ground vehicles. Vector Directed Path-Generation and Tracking (VDPGT), a vector-based algorithm for real-time path-generation and tracking, is proposed in this work. VDPGT dynamically adopts the shortest path to the destination while minimizing the tracking error. Collision avoidance is performed utilizing Hough transform, Canny contour, Lucas–Kanade sparse optical flow algorithm and expansion of object-based time-to-contact estimation. Simulation and experimental results from Turtlebot and AR Drone show that VDPGT can dynamically generate and track paths, and SOIFRA is interoperable across multiple robotic platforms.