Real-time Appearance-based Mapping Research Articles

Real-Time 3D Reconstruction of UAV Acquisition System for the Urban Pipe Based on RTAB-Map

In urban underground projects, such as urban drainage systems, the real-time acquisition and generation of 3D models of pipes can provide an important foundation for pipe safety inspection and maintenance. The simultaneous localization and mapping (SLAM) technique, compared to the traditional structure from motion (SfM) reconstruction technique, offers high real-time performance and improves the efficiency of 3D object reconstruction. Underground pipes are situated in complex environments with unattended individuals and often lack natural lighting. To address this, this paper presents a real-time and cost-effective 3D perception and reconstruction system that utilizes an unmanned aerial vehicle (UAV) equipped with Intel RealSense D435 depth cameras and an artificial light-supplementation device. This system carries out real-time 3D reconstruction of underground pipes using the RTAB-Map (real-time appearance-based mapping) method. RTAB-Map is a graph-based visual SLAM method that combines closed-loop detection and graph optimization algorithms. The unique memory management mechanism of RTAB-Map enables synchronous mapping for multiple sessions during UAV flight. Experimental results demonstrate that the proposed system, based on RTAB-Map, exhibits the robustness, textures, and feasibility for 3D reconstruction of underground pipes.

DOPESLAM: High-Precision ROS-Based Semantic 3D SLAM in a Dynamic Environment.

Recent advancements in deep learning techniques have accelerated the growth of robotic vision systems. One way this technology can be applied is to use a mobile robot to automatically generate a 3D map and identify objects within it. This paper addresses the important challenge of labeling objects and generating 3D maps in a dynamic environment. It explores a solution to this problem by combining Deep Object Pose Estimation (DOPE) with Real-Time Appearance-Based Mapping (RTAB-Map) through means of loose-coupled parallel fusion. DOPE's abilities are enhanced by leveraging its belief map system to filter uncertain key points, which increases precision to ensure that only the best object labels end up on the map. Additionally, DOPE's pipeline is modified to enable shape-based object recognition using depth maps, allowing it to identify objects in complete darkness. Three experiments are performed to find the ideal training dataset, quantify the increased precision, and evaluate the overall performance of the system. The results show that the proposed solution outperforms existing methods in most intended scenarios, such as in unilluminated scenes. The proposed key point filtering technique has demonstrated an improvement in the average inference speed, achieving a speedup of 2.6× and improving the average distance to the ground truth compared to the original DOPE algorithm.

Voice Interaction Recognition Design in Real-Life Scenario Mobile Robot Applications

This paper designed a voice interactive robot system that can conveniently execute assigned service tasks in real-life scenarios. It is equipped without a microphone where users can control the robot with spoken commands; the voice commands are then recognized by a well-trained deep neural network model of automatic speech recognition (ASR), which enables the robot to execute and complete the command based on the navigation of a real-time simultaneous localization and mapping (SLAM) algorithm. The voice interaction recognition model is divided into two parts: (1) speaker separation and (2) ASR. The speaker separation is applied by a deep-learning system consisting of eight convolution layers, one LSTM layer, and two fully connected (FC) layers to separate the speaker’s voice. This model recognizes the speaker’s voice as a referrer that separates and holds the required voiceprint and removes noises from other people’s voiceprints. Its automatic speech recognition uses the novel sandwich-type conformer model with a stack of three layers, and combines convolution and self-attention to capture short-term and long-term interactions. Specifically, it contains a multi-head self-attention module to directly convert the voice data into text for command realization. The RGB-D vision-based camera uses a real-time appearance-based mapping algorithm to create the environment map and replace the localization with a visional odometer to allow the robot to navigate itself. Finally, the proposed ASR model was tested to check if the desired results will be obtained. Performance analysis was applied to determine the robot’s environment isolation and voice recognition abilities. The results showed that the practical robot system successfully completed the interactive service tasks in a real environment. This experiment demonstrates the outstanding performance with other ASR methods and voice control mobile robot systems. It also verified that the designed voice interaction recognition system enables the mobile robot to execute tasks in real-time, showing that it is a convenient way to complete the assigned service applications.

Research on Multi-Sensor Fusion SLAM Algorithm Based on Improved Gmapping

Simultaneous Localization and Mapping (SLAM) is the core technology of the intelligent robot system, and it is also the basis for its autonomous movement. In recent years, it has been found that SLAM using a single sensor has certain limitations, such as Inertial Measurement Unit (IMU) noise and serious drift, and 2D radar can only detect environmental information on the same horizontal plane. In this regard, this paper constructs a multi-sensor back-end fusion SLAM algorithm that combines vision, laser, encoder and IMU information. Experiments have proved that compared with using a single sensor, the application of a multi-sensor fusion system makes the edges of the constructed map clearer and the noise reduced. Aiming at the problem of increased calculation caused by particle degradation and too many particles, this paper improves the Gmappping algorithm, and uses the combination of selective resampling and Kullback-Leibler Distance (KLD) sampling to complete resampling. It has been proved by experiments that compared with the original algorithm of Gmapping, the application of the improved algorithm increases the particle convergence speed by 39.85% in the process of indoor mapping. Aiming at the problems that the traditional loop detection algorithm is easily affected by environmental factors, resulting in low detection accuracy, and the loop detection algorithm based on deep convolutional neural network has a large amount of calculation and takes a long time to detect. The main research of this paper is to apply a deep learning-based loop detection algorithm on the multi-sensor fusion framework, and use the combination of high-dimensional and low-dimensional features of the image for loop detection. This paper uses different algorithms to conduct comparative experiments on the dataset CityCentre. The experimental results show that compared with the traditional algorithms Bag of Words (BoW), AlexNet algorithm, VGG19 algorithm, and ResNet32 algorithm, the accuracy of the algorithm proposed in this paper has increased by 31.26%, 14.21%, 3.05%, and 1.56%, respectively. In addition, the comparison experiment results of SLAM mapping with the original Real-Time Appearance-Based Mapping (RTAB-MAP) algorithm prove that the loop closure detection algorithm based on deep learning proposed in this paper can enable the system to better build a globally consistent map, including more environmental information.

Real-Time Appearance Based Mapping using Visual Sensor for Unknown Environment

Underwater environment is one of the most challenging areas when considering scene reconstruction and mapping, feature extraction of environment etc. GPS usage is denied in underwater environment and a waterproof lidar is also not a cost-effective solution to map and localize. Lidar-based simultaneous localization and mapping (SLAM) are also not preferred in budget-constrained applications. So, a visual-slam (V-SLAM) approach called Real-Time Appearance Based Mapping (RTAB-Map) is used to reconstruct the view in underwater scenario, i.e., map and localize the camera in the environment. In this work, Intel Real Sense D455 depth camera with inbuilt inertial measurement unit (IMU) is used for scene reconstruction. In order to get better visualization, it needs high performance embedded computer. Here, Jetson Nano embedded computer with robot operating system is used. Experiment is conducted for 3-D reconstruction map with the help of RTAB mapping technique and inbuilt IMU localized the camera in unknown environment. Then various depth, point clouds, odometry of a camera, visual object information is extracted in the indoor environment.

Autonomous vehicle navigation using high-definition maps through CARLA-ROS simulator bridge

Autonomous vehicles (AV) have garnered significant interest in recent years due to its potential for controlling thousands of accidents that happen yearly due to human error. However, AV bring with it very complex and sophisticated requirements and challenges related to extensive testing of the algorithms and hardware in the physical world. The evolution of automotive simulation tools provides an opportunity to fully test and validate AV architectures without the risk of creating hazardous situations in real world. This research demonstrates the application of HD (High Definition) maps in autonomous vehicle navigation using ROS interface integrated with the CARLA (CAR Learning to Act) simulator. The sensor data includes Light Detection and Ranging (LiDAR), RGB-Depth (RGB-D), and vehicle odometry. HD maps play an important role in robustness of the autonomous systems where due to sensor obstruction or weather conditions the vehicle is unable to perceive the information ahead. It also aids the vehicle to sense its environment even outside from the sensor’s field of view. The research is divided into the three fundamental concepts of Simultaneous Localization and Mapping (SLAM) approach that is, (i) mapping, (ii) localization, and (iii) navigation. Two ROS tools are used for mapping, (a) OctoMap mapping, and (ii) Real-Time Appearance-Based Mapping (RTAB-Map). We demonstrate the effectiveness of localization using RTAB-Map and compare actual path, position and orientation to their estimated equivalents. Our results show acceptable error in XY axes and exemplifies the error accumulated in Z axis.

Subsurface robotic exploration for geomorphology, astrobiology and mining during MINAR6 campaign, Boulby Mine, UK: part II (Results and Discussion)

AbstractGeomorphological studies of the hidden and protected subsurface environments are crucial to obtain a greater insight into the evolution of planetary landforms, hydrology, climate, geology and mineralogy. From an astrobiological point of view subsurface environments are of interest for their potential habitability as they are local environments that are partially or fully shielded from the high levels of space and solar radiation. Furthermore, in the case of Mars, there is an increasing interest in searching for the presence of past or extant life in its subsurface. These applications make it mandatory to investigate equipment and instrumentation that allow for the study of subsurface geomorphology, as well as organic chemical biomarkers, such as biomolecules, carbon, nitrogen and sulphur isotopes, and other biologically significant minerals and gases. Mines on Earth can be used as analogues to investigate the geomorphology of Martian subsurface environments and perform astrobiology studies. With that goal, we have developed a low-cost, robust, remotely operable subsurface rover called KORE (KOmpact Rover for Exploration). This work illustrates the studies of a terrestrial analogue for the exploration of Mars using KORE during the Mine Analogue Research 6 (MINAR 6) campaign with the low-cost 3D mapping technology InXSpace 3D (In situ3D mapping tool eXploration of space 3D). InXSpace 3D utilizes an RGB-D camera that captures depth information in addition to the RGB data of an image, operating based on the structured light principle capable of providing depth information in mm scale resolution at sub 3 m mapping range. InXSpace 3D is used to capture point clouds of natural and artificial features, thereby obtaining information about geologically relevant structures and also to incorporate them in earth mining safety. We tested two of the dense simultaneous localization and mapping (SLAM) algorithms: Kintinuous and Real-Time Appearance-Based Mapping (RTAB-Map) to check the performance of InXSpace 3D in a dark mine environment. Also, the air accumulation of volatiles such as methane and formaldehyde due to thermogenic and mining process was measured with the environmental station payload on the rover platform, which caters to both astrobiological significance and mine safety. The main conclusions of this work are: (1) a comparison made between the RTAB-Map algorithm and Kintinuous algorithm showed the superiority of Kintinuous algorithm in providing better 3D reconstruction; although RTAB-Map algorithm captured more points than the Kintinuous algorithm in the dark mine environment; (2) a comparison of point cloud images captured with and without lighting conditions had a negligible effect on the surface density of the point clouds; (3) close-range imaging of the polygonal features occurring on the halite walls using InXSpace 3D provided mm-scale resolution to enable further characterization; (4) heuristic algorithms to quickly post-process the 3D point cloud data provided encouraging results for preliminary analyses; (5) we successfully demonstrated the application of KORE to mine safety; and (6) the multi-sensors platform on KORE successfully monitored the accumulated volatiles in the mine atmosphere during its operation. The findings obtained during this KORE campaign could be incorporated in designing and planning future subsurface rover explorations to potential planetary bodies such as Mars with synergistic applications to subsurface environments in mines on Earth.

A Benchmark of Popular Indoor 3D Reconstruction Technologies: Comparison of ARCore and RTAB-Map

The fast evolution in computational and sensor technologies brings previously niche solutions to a wider userbase. As such, 3D reconstruction technologies are reaching new use-cases in scientific and everyday areas where they were not present before. Cost-effective and easy-to-use solutions include camera-based 3D scanning techniques, such as photogrammetry. This paper provides an overview of the available solutions and discusses in detail the depth-image based Real-time Appearance-based Mapping (RTAB-Map) technique as well as a smartphone-based solution that utilises ARCore, the Augmented Reality (AR) framework of Google. To qualitatively compare the two 3D reconstruction technologies, a simple length measurement-based method was applied with a purpose-designed reference object. The captured data were then analysed by a processing algorithm. In addition to the experimental results, specific case studies are briefly discussed, evaluating the applicability based on the capabilities of the technologies. As such, the paper presents the use-case of interior surveying in an automated laboratory as well as an example for using the discussed techniques for landmark surveying. The major findings are that point clouds created with these technologies provide a direction- and shape-accurate model, but those contain mesh continuity errors, and the estimated scale factor has a large standard deviation.

Visual Odometry Implementation and Accuracy Evaluation Based on Real-time Appearance-based Mapping

Visual Odometry Implementation and Accuracy Evaluation Based on Real-time Appearance-based Mapping

Research of visual simultaneous localization and mapping-based navigation system for mobile robots

Subject of Research. The paper considers navigation system for mobile robots by building a map of the environment using Simultaneous Localization and Mapping algorithm and converting the 3D map collected by INTEL Realsense Depth camera into a 2D-cost map. Method. Real-Time Appearance-Based Mapping was used for building a virtual 3D map of the environment. A binary map was obtained by projecting the 3D map on a plane. The D* algorithm was applied on the binary map for planning a global path to the goal. The Dynamic-Window Approach was used as a local planner. Main Results. A point cloud of the environment was created and converted to a 2D map. A robot was safely navigated to the desired location. Practical Relevance. The proposed approach is fast and reliable and can be used for indoor navigation (factories and companies). Since the map needs to be designed only once, the calculation can be handled by CPU without any need for graphics processing unit.

RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation

AbstractDistributed as an open‐source library since 2013, real‐time appearance‐based mapping (RTAB‐Map) started as an appearance‐based loop closure detection approach with memory management to deal with large‐scale and long‐term online operation. It then grew to implement simultaneous localization and mapping (SLAM) on various robots and mobile platforms. As each application brings its own set of constraints on sensors, processing capabilities, and locomotion, it raises the question of which SLAM approach is the most appropriate to use in terms of cost, accuracy, computation power, and ease of integration. Since most of SLAM approaches are either visual‐ or lidar‐based, comparison is difficult. Therefore, we decided to extend RTAB‐Map to support both visual and lidar SLAM, providing in one package a tool allowing users to implement and compare a variety of 3D and 2D solutions for a wide range of applications with different robots and sensors. This paper presents this extended version of RTAB‐Map and its use in comparing, both quantitatively and qualitatively, a large selection of popular real‐world datasets (e.g., KITTI, EuRoC, TUM RGB‐D, MIT Stata Center on PR2 robot), outlining strengths, and limitations of visual and lidar SLAM configurations from a practical perspective for autonomous navigation applications.