Feature-based Simultaneous Localization And Mapping Research Articles

Enhancing Real-Time Visual SLAM with Distant Landmarks in Large-Scale Environments

The efficacy of visual Simultaneous Localization and Mapping (SLAM) diminishes in large-scale environments due to challenges in identifying distant landmarks, leading to a limited perception range and trajectory drift. This paper presents a practical method to enhance the accuracy of feature-based real-time visual SLAM for compact unmanned vehicles by constructing distant map points. By tracking consecutive image features across multiple frames, remote map points are generated with sufficient parallax angles, extending the mapping scope to the theoretical maximum range. Observations of these landmarks from preceding keyframes are supplemented accordingly, improving back-end optimization and, consequently, localization accuracy. The effectiveness of this approach is ensured by the introduction of the virtual map point, a proposed data structure that links relational features to an imaginary map point, thereby maintaining the constrained size of local optimization during triangulation. Based on the ORB-SLAM3 code, a SLAM system incorporating the proposed method is implemented and tested. Experimental results on drone and vehicle datasets demonstrate that the proposed method outperforms ORB-SLAM3 in both accuracy and perception range with negligible additional processing time, thus preserving real-time performance. Field tests using a UGV further validate the efficacy of the proposed method.

Lightweight image denoising method for feature-based visual SLAM in γ radiation environments

In the γ radiation environments, high-energy photons impact image sensor pixels and induce speckle noise in the scene image, which influences the accuracy of feature detection and matching in the visual Simultaneous Localization and Mapping (SLAM) algorithms. To mitigate these effects, this paper presents a lightweight image denoising method based on super-neighborhood down-sampling. Firstly, the super-neighborhood down-sampling strategy is used to split the speckle noise in the camera images into isolated point noises in the sub-image sets. Secondly, these isolated flickering point noises are removed by the detection-based median filter. Finally, the denoised scene image is reconstructed by the super-neighborhood up-sampling. Extensive experiments were performed on the radiation environment datasets that were captured by the camera inside the Cobalt-60 hot cell. The results show that the proposed method can effectively filter out the speckle radiation noise induced by γ rays, significantly improve the quality of features and feature matching rate, and improve the performance of ORB-SLAM3, the state-of-the-art feature-based SLAM algorithm, in radiation environments.

An Automatic and Robust Visual SLAM Method for Intra-Abdominal Environment Reconstruction

Three-dimensional (3D) surface reconstruction is used to solve the problem of the narrow field of view in laparoscopy. It can provide surgeons or computer-assisted surgery systems with real-time complete internal abdominal anatomy. However, rapid changes in image depth, less texture, and specular reflection pose a challenge for the reconstruction. It is difficult to stably complete the reconstruction process using feature-based simultaneous localization and mapping (SLAM) method. This paper proposes a robust laparoscopic 3D surface reconstruction method using SLAM, which can automatically select appropriate parameters for stereo matching and robustly find matching point pairs for laparoscope motion estimation. The changing trend of disparity maps is used to predict stereo matching parameters to improve the quality of the disparity map. Feature patch extraction and tracking are selected to replace feature point extraction and matching in motion estimation, which reduces its failure and interruption in feature-based SLAM. The proposed feature patch matching method is suitable for parallel computing, which can improve its computing speed. Evaluation results on public in vivo and ex vivo porcine abdominal video data show the efficiency and robustness of our 3D surface reconstruction approach.

Rein-SLAM: Narrow the Gaps Between the Matching Task and SLAM System

Image matching task and feature-based SLAM (Simultaneous Localization And Mapping) systems both rely on feature points to complete pixel association, which provide an important support and foundation for the perception and integration of unmanned vehicles in the industrial park environments. In this article, we point out there exist associations and gaps between the matching task and SLAM systems. Based on the feature extraction network for matching task, we propose an effective training strategy in reference to the operating mode of SLAM systems, so that the feature extraction network can better adapt to the SLAM system for pose estimation and mapping. The proposed method only needs one sequence in the same scene for training, and the whole training process can be completed in only one epoch which takes no more than 20 minutes on a RTX 3080 GPU. We conduct experiments on HPatches dataset (matching task dataset), two public SLAM datasets: KITTI dataset for outdoor scenes and Euroc dataset for indoor scenes, and our collected dataset which contains several challenging scenes. Our experiments show that our method can not only effectively improve the performance of the extraction network on the original matching task, but also greatly improve the performance of its integrated SLAM system. Furthermore, our method can be easily transplanted to other image matching networks, so that they can be better used in SLAM systems and narrow the gaps between the matching tasks and SLAM systems.

Semantic closed-loop based visual mapping algorithm for automated valet parking

Simultaneous localization and mapping (SLAM) is one of the core technologies to realize automated valet parking (AVP). Currently, advanced visual feature-based SLAM systems suffer from feature extraction failure and tracking loss due to the constraints of textureless scenes, unclear illumination, and dynamic conditions. To address these problems, this paper proposes a visual SLAM algorithm based on a semantic closed-loop detection algorithm using surround-view cameras and inertial measurement units (IMU) as sensors. The algorithm combines semantic features and the idea of inverse index to improve the traditional keyframes selection methods and the closed-loop detection algorithms, effectively avoiding the tedious and complicated feature point matching and improving the computational efficiency of the computer. Experiments show that the algorithm in this paper achieves better results in terms of precision and recall, absolute trajectory error (ATE), and relative pose error (RPE), and can meet the demand for SLAM and subsequent navigation in indoor parking lots.

A SIMULTANEOUS PATH PLANNING AND POSITIONING BASED ON ARTIFICIAL DISTRIBUTION OF LANDMARKS IN A GNSS DENIED ENVIRONMENT

In recent years, exploration operations by autonomous robots are expanding into unknown environments on Earth, under the sea, or even on other planets. This paper proposes the idea of Concurrent Path Planning and Positioning (CPPAP) using artificially distributed landmarks, while no GNSS signal is available. The method encompasses an observability-based direction search algorithm for path planning in parallel with Simultaneous Localization and Mapping (SLAM) for localization. Most of the path planning methods utilize offline algorithms; however, the proposed method determines the robot’s direction of motion in real-time, concurrently with the positioning tasks by the inclusion of the system observability, related to the features’ distribution. Same as in all feature-based SLAMs, features play an important role in determination of the most observable direction, and hence the direction of the robot’s motion. Moreover, the effectiveness of the distribution of the features and their pattern in the proposed method is investigated. To evaluate the efficiency and accuracy of the CPPAP, outcomes are compared with an existing random SLAM.

Semi-Direct Multimap SLAM System for Real-Time Sparse 3-D Map Reconstruction

The lack of a sufficient number of reliable corners in low-textured environments is a big challenge for classical visual simultaneous localization and mapping (SLAM), especially for point feature-based methods. Many other features (i.e., line and plane segments) are often combined with points to restore an environmental structure. However, using such features requires much computational time. This work focuses on the reliable and high-performance real-time operation of a SLAM system in low-textured scenarios. It proposes a semi-direct multi-map monocular SLAM system (SM-SLAM) that combines direct tracking and feature-based map maintenance with point features and line segments. The proposed system tracks non-keyframes based on a sparse image alignment method for fast tracking, extracts and matches point features and line segments in keyframes for high-quality environment structure and motion optimization. We present an extensive evaluation on two widely-used public datasets and some challenging real-world scenarios. Experimental results show that SM-SLAM can well reconstruct a sparse 3D map with geometrical structure information in 30-40 Hz. Notably, SM-SLAM shows an accuracy improvement of more than 20% than the Oriented FAST and Rotated BRIEF feature-based SLAM (ORB-SLAM) on some low-speed, small-range camera motion datasets and performs well in low-texture scenarios.

GMP-SLAM: A real-time RGB-D SLAM in Dynamic Environments using GPU Dynamic Points Detection Method

Simultaneous Localization and Mapping (SLAM) is a fundamental technology for robotics. Vision-based SLAM has been developed for many years, but it is still difficult for SLAM system to handle dynamic environments. In this paper, we present GMP-SLAM, a realtime RGB-D SLAM system for highly dynamic environments with the help of GPU Grid Map Projection— a GPU dynamic points detection method we design. SLAM is a time-sensitive system for robotics, and it is hard to reach real time, especially in dynamic environments because it is necessary to track moving objects and it takes a lot of time. GMP-SLAM is based on ORB-SLAM2, which is one of the best feature-based SLAM frameworks and can reach real time just in CPU. But ORB-SLAM2 cannot handle highly dynamic environments very well, and most work focus on tracking moving objects with a neural network, which cannot reach real time even with the help of GPU. To solve real-time problem, we propose an all-in-parallel dynamic points detection framework for visual simultaneous localization and mapping (VSLAM) in dynamic environments based on 3D occupancy grid maps. Our SLAM system can provide not only higher trajectory accuracy but also a 3D grid map for navigation. We test our SLAM system in our real-world datasets we record and get higher trajectory accuracy than ORB-SLAM2. At the same time, our system can run nearly in 20Hz, which is much better than existing VSLAM framework in dynamic environments.

SP-SLAM: Surfel-Point Simultaneous Localization and Mapping

In this article, a novel method for simultaneous localization and mapping (SLAM) named surface elements (surfel) point SLAM (SP-SLAM) is proposed using an RGB-D camera. The key idea of SP-SLAM is to use not only keypoints but also surfels as features to cope with both high texture and low texture environments. By decomposing a surface into a small number of surfels, the method can represent spacious environments using a relatively small amount of memory. To optimize the poses of points, surfels, and cameras altogether, new objective functions are proposed, and a new bundle adjustment using these objective functions is developed. The proposed SP-SLAM runs in real time on a central processing unit as in other feature-based visual SLAM methods but works better than them not only in high texture but also in low texture environments, overcoming well-known drawbacks of feature-based visual SLAM with degradation in low texture environments. The proposed method is applied to benchmark datasets and its effectiveness is demonstrated by comparing against those of previous methods in terms of localization accuracy.

Autonomous Exploration of Unknown Indoor Environments for High-Quality Mapping Using Feature-Based RGB-D SLAM

Simultaneous localization and mapping (SLAM) system-based indoor mapping using autonomous mobile robots in unknown environments is crucial for many applications, such as rescue scenarios, utility tunnel monitoring, and indoor 3D modeling. Researchers have proposed various strategies to obtain full coverage while minimizing exploration time; however, mapping quality factors have not been considered. In fact, mapping quality plays a pivotal role in 3D modeling, especially when using low-cost sensors in challenging indoor scenarios. This study proposes a novel exploration algorithm to simultaneously optimize exploration time and mapping quality using a low-cost RGB-D camera. Feature-based RGB-D SLAM is utilized due to its various advantages, such as low computational cost and dense real-time reconstruction ability. Subsequently, our novel exploration strategies consider the mapping quality factors of the RGB-D SLAM system. Exploration time optimization factors are also considered to set a new optimum goal. Furthermore, a Voronoi path planner is adopted for reliable, maximal obstacle clearance and fixed paths. According to the texture level, three exploration strategies are evaluated in three real-world environments. We achieve a significant enhancement in mapping quality and exploration time using our proposed exploration strategies compared to the baseline frontier-based exploration, particularly in a low-texture environment.

LiDAR Odometry by Deep Learning-Based Feature Points with Two-Step Pose Estimation

An accurate ego-motion estimation solution is vital for autonomous vehicles. LiDAR is widely adopted in self-driving systems to obtain depth information directly and eliminate the influence of changing illumination in the environment. In LiDAR odometry, the lack of descriptions of feature points as well as the failure of the assumption of uniform motion may cause mismatches or dilution of precision in navigation. In this study, a method to perform LiDAR odometry utilizing a bird’s eye view of LiDAR data combined with a deep learning-based feature point is proposed. Orthographic projection is applied to generate a bird’s eye view image of a 3D point cloud. Thereafter, an R2D2 neural network is employed to extract keypoints and compute their descriptors. Based on those keypoints and descriptors, a two-step matching and pose estimation is designed to keep these feature points tracked over a long distance with a lower mismatch ratio compared to the conventional strategy. In the experiment, the evaluation of the proposed algorithm on the KITTI training dataset demonstrates that the proposed LiDAR odometry can provide more accurate trajectories compared with the handcrafted feature-based SLAM (Simultaneous Localization and Mapping) algorithm. In detail, a comparison of the handcrafted descriptors is demonstrated. The difference between the RANSAC (Random Sample Consensus) algorithm and the two-step pose estimation is also demonstrated experimentally. In addition, the data collected by Velodyne VLP-16 is also evaluated by the proposed solution. The low-drift positioning RMSE (Root Mean Square Error) of 4.70 m from approximately 5 km mileage shown in the result indicates that the proposed algorithm has generalization performance on low-resolution LiDAR.

Evolution of Simultaneous Localization and Mapping Framework for Autonomous Robotics—A Comprehensive Review

Abstract Autonomous robotics plays a pivotal role to simplify human–machine interaction while meeting the current industrial demands. In that process, machine intelligence plays a dominant role during the decision making in the operational state-space. Primarily, this decision making and control mechanism relies on sensing and actuation. Simultaneous localization and mapping (SLAM) is the most advanced technique that facilitates both sensing and actuation to achieve autonomy for robots. This work aims to collate multidimensional aspects of simultaneous localization and mapping techniques primarily in the purview of both deterministic and probabilistic frameworks. This investigation on SLAM classification is further elaborated into different categories such as feature-based SLAM and optimization-based SLAM. In this work, the chronological evolution of the SLAM technique develops a comprehensive understanding among the concerned research community.

CNN-Based Dense Monocular Visual SLAM for Real-Time UAV Exploration in Emergency Conditions

Unmanned Aerial Vehicles (UAVs) for 3D indoor mapping applications are often equipped with bulky and expensive sensors, such as LIDAR (Light Detection and Ranging) or depth cameras. The same task could be also performed by inexpensive RGB cameras installed on light and small platforms that are more agile to move in confined spaces, such as during emergencies. However, this task is still challenging because of the absence of a GNSS (Global Navigation Satellite System) signal that limits the localization (and scaling) of the UAV. The reduced density of points in feature-based monocular SLAM (Simultaneous Localization and Mapping) then limits the completeness of the delivered maps. In this paper, the real-time capabilities of a commercial, inexpensive UAV (DJI Tello) for indoor mapping are investigated. The work aims to assess its suitability for quick mapping in emergency conditions to support First Responders (FR) during rescue operations in collapsed buildings. The proposed solution only uses images in input and integrates SLAM and CNN-based (Convolutional Neural Networks) Single Image Depth Estimation (SIDE) algorithms to densify and scale the data and to deliver a map of the environment suitable for real-time exploration. The implemented algorithms, the training strategy of the network, and the first tests on the main elements of the proposed methodology are reported in detail. The results achieved in real indoor environments are also presented, demonstrating performances that are compatible with FRs’ requirements to explore indoor volumes before entering the building.

PAL-SLAM: a feature-based SLAM system for a panoramic annular lens.

Simultaneous localization and mapping (SLAM) is widely used in autonomous driving and intelligent robot positioning and navigation. In order to overcome the defects of traditional visual SLAM in rapid motion and bidirectional loop detection, we present a feature-based PAL-SLAM system for a panoramic-annular-lens (PAL) camera in this paper. We use a mask to extract and match features in the annular effective area of the images. A PAL-camera model, based on precise calibration, is used to transform the matched features onto a unit vector for subsequent processing, and a prominent inlier-checking metric is designed as an epipolar constraint in the initialization. After testing on large-scale indoor and outdoor PAL image dataset sequences, comprising of more than 12,000 images, the accuracy of PAL-SLAM is measured as typically below 1 cm. This result holds consistent in conditions when the camera rotates rapidly, or the Global Navigation Satellite System (GNSS) signals are blocked. PAL-SLAM can also detect unidirectional and bidirectional loop closures. Hence it can be used as a supplement or alternative to expensive commercial navigation systems, especially in urban environments where there are many signal obstructions such as buildings and bridges.

LIFT-SLAM: A deep-learning feature-based monocular visual SLAM method

The Simultaneous Localization and Mapping (SLAM) problem addresses the possibility of a robot to localize itself in an unknown environment and simultaneously build a consistent map of this environment. Recently, cameras have been successfully used to get the environment’s features to perform SLAM, which is referred to as visual SLAM (VSLAM). However, classical VSLAM algorithms can be easily induced to fail when either the motion of the robot or the environment is too challenging. Although new approaches based on Deep Neural Networks (DNNs) have achieved promising results in VSLAM, they still are unable to outperform traditional methods. To leverage the robustness of deep learning to enhance traditional VSLAM systems, we propose to combine the potential of deep learning-based feature descriptors with the traditional geometry-based VSLAM, building a new VSLAM system called LIFT-SLAM. Experiments conducted on KITTI and Euroc datasets show that deep learning can be used to improve the performance of traditional VSLAM systems, as the proposed approach was able to achieve results comparable to the state-of-the-art while being robust to sensorial noise. We enhance the proposed VSLAM pipeline by avoiding parameter tuning for specific datasets with an adaptive approach while evaluating how transfer learning can affect the quality of the features extracted.

Semantic Optimization of Feature-Based SLAM

The purpose of this paper is to provide reasonable recommendation and removal of inappropriate information for SLAM (Simultaneous Localization and Mapping) technology based on feature method. The methodology is to propose a semantic recognition of environment objects in the natural scene through object detection, which is a kind of bag of word method in SLAM problem between the key frames and object level, the method of establishing key frames, and the relationship between the target object levels, through the practical significance of the target object level to judge the merits of the target object level information, and then combined with key frames in the visual SLAM relations with relevant information, so as to get object level targets in each key frame and the relationship between the relevant information, so as to achieve through the object level semantic information to judge the merits of the key frames and screening, as well as to the key frames to judge the merits of the relevant information and screening purpose. The finding of the study is the above method can retain the information of high reliability and good stability for visual SLAM and process the key frames with poor reliability or low stability and the information related to key frames.

Line Flow Based Simultaneous Localization and Mapping

In this article, we propose a visual simultaneous localization and mapping (SLAM) method by predicting and updating line flows that represent sequential 2-D projections of 3-D line segments. While feature-based SLAM methods have achieved excellent results, they still face problems in challenging scenes containing occlusions, blurred images, and repetitive textures. To address these problems, we leverage a line flow to encode the coherence of line segment observations of the same 3-D line along the temporal dimension, which has been neglected in prior SLAM systems. Thanks to this line flow representation, line segments in a new frame can be predicted according to their corresponding 3-D lines and their predecessors along the temporal dimension. We create, update, merge, and discard line flows on-the-fly. We model the proposed line flow based SLAM (LF-SLAM) using a Bayesian network. Extensive experimental results demonstrate that the proposed LF-SLAM method achieves state-of-the-art results due to the utilization of line flows. Specifically, LF-SLAM obtains good localization and mapping results in challenging scenes with occlusions, blurred images, and repetitive textures.

Semi-Direct Monocular SLAM With Three Levels of Parallel Optimizations

In practical applications, how to use the complementary strengths of the direct and the feature-based methods for effective fusion may be the main challenge of simultaneous localization and mapping (SLAM). To solve this challenge, we propose the DO-SLAM, a novel fast and accurate semi-direct visual SLAM framework, which can maintain the direct method’s fast performance and the high precision and loop closure capability of the feature-based method. The direct method is used as the first half of the DO-SLAM to track the camera pose rapidly and robustly. The feature-based method is used as the second half of the DO-SLAM to refine the keyframe poses, perform loop closures, and build a globally consistent, long-term, sparse feature map that can be reused. The proposed pipeline fuses direct odometry and feature-based SLAM to perform three levels of parallel optimizations: (1) In the direct method module, the keyframe poses are estimated by minimizing the photometric error, (2) In the feature-based module, using the poses calculated by the inter-frame matching to correct and fuse the poses calculated by the direct method module as the initial poses, and the initial poses are optimized by the motion-only bundle adjustment, and (3) A pose graph optimization is used to achieve global map consistency in the presence of loop closures. Experimental evaluation on two benchmark datasets demonstrates that the proposed approach achieves higher accuracy and robustness on motion estimation compared to the other state-of-the-art methods.

Moving Object Segmentation and Detection for Robust RGBD-SLAM in Dynamic Environments

Localization accuracy is a fundamental requirement for Simultaneous Localization and Mapping (SLAM) systems. Traditional visual SLAM (vSLAM) schemes are usually based upon the assumption of static environments, so they do not perform well in dynamic environments. While a number of vSLAM frameworks have been reported for dynamic environments, the localization accuracy is usually unsatisfactory. In this article, we present a novel motion detection and segmentation method using Red Green Blue-Depth (RGB-D) data to improve the localization accuracy of feature-based RGB-D SLAM in dynamic environments. To overcome the problem due to undersegmentation generated by the semantic segmentation network, a mask inpainting method is developed to ensure the completeness of object segmentation. In the meantime, an optical flow-based motion detection method is proposed to detect dynamic objects from moving cameras, allowing robust detection by removing irrelevant information. Experiments performed on the public Technical University of Munich (TUM) RGB-D data set show that the presented scheme outperforms the state-of-art RGB-D SLAM systems in terms of trajectory accuracy, improving the localization accuracy of RGB-D SLAM in dynamic environments.

Study on the Influence of Image Noise on Monocular Feature-Based Visual SLAM Based on FFDNet.

Noise appears in images captured by real cameras. This paper studies the influence of noise on monocular feature-based visual Simultaneous Localization and Mapping (SLAM). First, an open-source synthetic dataset with different noise levels is introduced in this paper. Then the images in the dataset are denoised using the Fast and Flexible Denoising convolutional neural Network (FFDNet); the matching performances of Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF) and Oriented FAST and Rotated BRIEF (ORB) which are commonly used in feature-based SLAM are analyzed in comparison and the results show that ORB has a higher correct matching rate than that of SIFT and SURF, the denoised images have a higher correct matching rate than noisy images. Next, the Absolute Trajectory Error (ATE) of noisy and denoised sequences are evaluated on ORB-SLAM2 and the results show that the denoised sequences perform better than the noisy sequences at any noise level. Finally, the completely clean sequence in the dataset and the sequences in the KITTI dataset are denoised and compared with the original sequence through comprehensive experiments. For the clean sequence, the Root-Mean-Square Error (RMSE) of ATE after denoising has decreased by 16.75%; for KITTI sequences, 7 out of 10 sequences have lower RMSE than the original sequences. The results show that the denoised image can achieve higher accuracy in the monocular feature-based visual SLAM under certain conditions.