Simultaneous Localization And Mapping Framework Research Articles

Integrating Sparse Learning-Based Feature Detectors into Simultaneous Localization and Mapping-A Benchmark Study.

Simultaneous localization and mapping (SLAM) is one of the cornerstones of autonomous navigation systems in robotics and the automotive industry. Visual SLAM (V-SLAM), which relies on image features, such as keypoints and descriptors to estimate the pose transformation between consecutive frames, is a highly efficient and effective approach for gathering environmental information. With the rise of representation learning, feature detectors based on deep neural networks (DNNs) have emerged as an alternative to handcrafted solutions. This work examines the integration of sparse learned features into a state-of-the-art SLAM framework and benchmarks handcrafted and learning-based approaches by comparing the two methods through in-depth experiments. Specifically, we replace the ORB detector and BRIEF descriptor of the ORBSLAM3 pipeline with those provided by Superpoint, a DNN model that jointly computes keypoints and descriptors. Experiments on three publicly available datasets from different application domains were conducted to evaluate the pose estimation performance and resource usage of both solutions.

A Dynamic Scene Vision SLAM Method Incorporating Object Detection and Object Characterization

Simultaneous localization and mapping (SLAM) based on RGB-D cameras has been widely used for robot localization and navigation in unknown environments. Most current SLAM methods are constrained by static environment assumptions and perform poorly in real-world dynamic scenarios. To improve the robustness and performance of SLAM systems in dynamic environments, this paper proposes a new RGB-D SLAM method for indoor dynamic scenes based on object detection. The method presented in this paper improves on the ORB-SLAM3 framework. First, we designed an object detection module based on YOLO v5 and relied on it to improve the tracking module of ORB-SLAM3 and the localization accuracy of ORB-SLAM3 in dynamic environments. The dense point cloud map building module was also included, which excludes dynamic objects from the environment map to create a static environment point cloud map with high readability and reusability. Full comparison experiments with the original ORB-SLAM3 and two representative semantic SLAM methods on the TUM RGB-D dataset show that: the method in this paper can run at 30+fps, the localization accuracy improved to varying degrees compared to ORB-SLAM3 in all four image sequences, and the absolute trajectory accuracy can be improved by up to 91.10%. The localization accuracy of the method in this paper is comparable to that of DS-SLAM, DynaSLAM and the two recent target detection-based SLAM algorithms, but it runs faster. The RGB-D SLAM method proposed in this paper, which combines the most advanced object detection method and visual SLAM framework, outperforms other methods in terms of localization accuracy and map construction in a dynamic indoor environment and has a certain reference value for navigation, localization, and 3D reconstruction.

SLAM for Indoor Parking: A Comprehensive Benchmark Dataset and a Tightly Coupled Semantic Framework

For the task of autonomous indoor parking, various Visual-Inertial Simultaneous Localization And Mapping (SLAM) systems are expected to achieve comparable results with the benefit of complementary effects of visual cameras and the Inertial Measurement Units. To compare these competing SLAM systems, it is necessary to have publicly available datasets, offering an objective way to demonstrate the pros/cons of each SLAM system. However, the availability of such high-quality datasets is surprisingly limited due to the profound challenge of the groundtruth trajectory acquisition in the Global Positioning Satellite denied indoor parking environments. In this article, we establish BeVIS, a large-scale Be nchmark dataset with V isual (front-view), I nertial and S urround-view sensors for evaluating the performance of SLAM systems developed for autonomous indoor parking, which is the first of its kind where both the raw data and the groundtruth trajectories are available. In BeVIS, the groundtruth trajectories are obtained by tracking artificial landmarks scattered in the indoor parking environments, whose coordinates are recorded in a surveying manner with a high-precision Electronic Total Station. Moreover, the groundtruth trajectories are comprehensively evaluated in terms of two respects, the reprojection error and the pose volatility, respectively. Apart from BeVIS, we propose a novel tightly coupled semantic SLAM framework, namely VIS SLAM -2, leveraging V isual (front-view), I nertial, and S urround-view sensor modalities, specially for the task of autonomous indoor parking. It is the first work attempting to provide a general form to model various semantic objects on the ground. Experiments on BeVIS demonstrate the effectiveness of the proposed VIS SLAM -2. Our benchmark dataset BeVIS is publicly available at https://shaoxuan92.github.io/BeVIS .

A monocular SLAM system based on SIFT features for gastroscope tracking

During flexible gastroscopy, physicians have extreme difficulties to self-localize. Camera tracking method such as simultaneous localization and mapping (SLAM) has become a research hotspot in recent years, allowing tracking of the endoscope. However, most of the existing solutions have focused on tasks in which sufficient texture information is available, such as laparoscope tracking, and cannot be applied to gastroscope tracking since gastroscopic images have fewer textures than laparoscopic images. This paper proposes a new monocular SLAM framework based on scale-invariant feature transform (SIFT) and narrow-band imaging (NBI), which extracts SIFT features instead of oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (BRIEF) features from gastroscopic NBI images, and performs feature retention based on the response sorting strategy for achieving more matches. Experimental results show that the root mean squared error of the proposed algorithm can reach a minimum of 2.074mm, and the pose accuracy can be improved by up to 25.73% compared with oriented FAST and rotated BRIEF (ORB)-SLAM. SIFT features and response sorting strategy can achieve more accurate matching in gastroscopic NBI images than other features and homogenization strategy, and the proposed algorithm can also run successfully on real clinical gastroscopic data. The proposed algorithm has the potential clinical value to assist physicians in locating the gastroscope during gastroscopy.

Nonparametric Background Model-Based LiDAR SLAM in Highly Dynamic Urban Environments

In urban environments, simultaneous localization and mapping (SLAM) are essential for autonomous driving. Most light detection and ranging (LiDAR) SLAM methodologies have been developed for relatively static environments, despite real-world environments having many dynamic objects such as vehicles, bicycles, and pedestrians. This paper proposes an efficient and robust LiDAR SLAM. Our SLAM framework leverages the estimated background model to achieve robust motion estimation in dynamic urban environments. Based on probabilistic object estimation, the dynamic removal module estimates a nonparametric background model to recognize dynamic objects. This module estimates the probability of the difference of the range values from the accumulated LiDAR frames. Then, dynamic objects are removed by adapting the sensor velocity from the estimated ego-motion. In the local mapping module, our method optimizes the LiDAR motion considering the dynamic characteristics of LiDAR point clouds. Finally, the proposed method results in a global map with static point clouds and accurate LiDAR motion with global pose optimization. We tested the proposed method on the well-known public dataset (KITTI) and the custom dataset with complex environments, including various moving objects. Comparisons with state-of-the-art (SOTA) methods demonstrate that the our approach is more robust and efficient. For example, the proposed method performed an average 0.63% and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.18^{\circ }/100\;m$ </tex-math></inline-formula> errors on the KITTI dataset with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.96ms$ </tex-math></inline-formula> processing time that convinces real-time processing.

A monocular visual SLAM system augmented by lightweight deep local feature extractor using in-house and low-cost LIDAR-camera integrated device

ABSTRACT Simultaneous Localization and Mapping (SLAM) has been widely used in emergency response, self-driving and city-scale 3D mapping and navigation. Recent deep-learning based feature point extractors have demonstrated superior performance in dealing with the complex environmental challenges (e.g. extreme lighting) while the traditional extractors are struggling. In this paper, we have successfully improved the robustness and accuracy of a monocular visual SLAM system under various complex scenes by adding a deep learning based visual localization thread as an augmentation to the visual SLAM framework. In this thread, our feature extractor with an efficient lightweight deep neural network is used for absolute pose and scale estimation in real time using the highly accurate georeferenced prior map database at 20cm geometric accuracy created by our in-house and low-cost LiDAR and camera integrated device. The closed-loop error provided by our SLAM system with and without this enhancement is 1.03m and 18.28m respectively. The scale estimation of the monocular visual SLAM is also significantly improved (0.01 versus 0.98). In addition, a novel camera-LiDAR calibration workflow is also provided for large-scale 3D mapping. This paper demonstrates the application and research potential of deep-learning based vision SLAM with image and LiDAR sensors.

A Variational Bayesian-Based Simultaneous Localization and Mapping Method for Autonomous Underwater Vehicle Navigation

Simultaneous Localization and Mapping (SLAM) is a well-known solution for mapping and realizing autonomous navigation of an Autonomous Underwater Vehicle (AUV) in unknown underwater environments. However, the inaccurate time-varying observation noise will cause filtering divergence and reduce the accuracy of localization and feature estimation. In this paper, VB-AUFastSLAM based on the unscented-FastSLAM (UFastSLAM) and the Variational Bayesian (VB) is proposed. The UFastSLAM combines unscented particle filter (UPF) and unscented Kalman filter (UKF) to estimate the AUV poses and features. In addition, to resist the unknown time-varying observation noise, the method of Variational Bayesian learning is introduced into the SLAM framework. Firstly, the VB method is used to estimate the joint posterior probability of the AUV path and observation noise. The Inverse-Gamma distribution is used to model the observation noise and real-time noise parameters estimation is performed to improve the AUV localization accuracy. Secondly, VB is reused to estimate the noise parameters in the feature update stage to enhance the performance of the feature estimation. The proposed algorithms are first validated in an open-source simulation environment. Then, an AUV SLAM system based on the Inertial Navigation System (INS), Doppler Velocity Log (DVL), and single-beam Sonar are also built to verify the effectiveness of the proposed algorithms in the marine environment. The accuracy of the proposed methods can reach 0.742% and 0.776% of the range, respectively, which is much better than 1.825% and 1.397% of the traditional methods.

OG-SLAM: A real-time and high-accurate monocular visual SLAM framework

The challenge of improving the accuracy of monocular Simultaneous Localization and Mapping (SLAM) is considered, which widely appears in computer vision, autonomous robotics, and remote sensing. A new framework (ORB-GMS-SLAM (or OG-SLAM)) is proposed, which introduces the region-based motion smoothness into a typical Visual SLAM (V-SLAM) system. The region-based motion smoothness is implemented by integrating the Oriented Fast and Rotated Brief (ORB) features and the Grid-based Motion Statistics (GMS) algorithm into the feature matching process. The OG-SLAM significantly reduces the absolute trajectory error (ATE) on the key-frame trajectory estimation without compromising the real-time performance. This study compares the proposed OG-SLAM to an advanced V-SLAM system (ORB-SLAM2). The results indicate the highest accuracy improvement of almost 75% on a typical RGB-D SLAM benchmark. Compared with other ORB-SLAM2 settings (1800 key points), the OG-SLAM improves the accuracy by around 20% without losing performance in real-time. The OG-SLAM framework has a significant advantage over the ORB-SLAM2 system in that it is more robust for rotation, loop-free, and long ground-truth length scenarios. Furthermore, as far as the authors are aware, this framework is the first attempt to integrate the GMS algorithm into the V-SLAM.

Fusion VIO: An Extendable Sensor Fusion Framework for Unmaned Vehicles

Real-time perception based on the simultaneous localization and mapping (SLAM) technology is of immense application potential in the industry filed. In this work, we provide an extended Kalman filter (EKF) based visual SLAM framework named Fusion VIO, which contains HDR imaging system, hardware synchronization, object detection, filter based sensor fusion, and semantic fusion. Notably, Fusion VIO well fuses loop closure information, which takes effects in solving the error problem during operation of the robot and improving the robustness of the system. Besides, experimental validation shows that Fusion VIO not only maintains high accuracy but also cost one-third resources compared to the common VINS fusion. Our work demonstrates a novel sensor fusion framework for unmaned vehicles.

A point-line-plane primitives fused localization and object-oriented semantic mapping in structural indoor scenes

Visual SLAM (abbreviates ‘simultaneous localization and mapping’) is a promising solution for environment mapping. This study is devoted to a description of a semantically ensembled SLAM framework. For structural indoor scenes, the structured lines and planes can serve as the newly added constraints to improve the positioning accuracy. In this paper, we propose to comprehensively incorporate point-line-plane primitives and construct a tightly coupled camera pose estimator without any environment assumptions. In particular, the maximum number of extracted lines features is numerically determined. We further integrate a lightweight object mapping pipeline with the designed pose estimator. In this pipeline, the leveraging of fitted plane and cuboid landmarks enables an online, CPU-based dense mapping. The tests on ICL-NUIM and TUM benchmark datasets illustrate that, in comparison to ORB-SLAM2, PL-SLAM (Point and Line based SLAM), SP-SLAM (Supposed Plane SLAM) and PLP-SLAM (Point, Line and Plane fused SLAM), our design leads to superior performances in global consistency and system drift elimination. The feature detection and multi-level map reconstruction results are simultaneously provided.

DiT-SLAM: Real-Time Dense Visual-Inertial SLAM with Implicit Depth Representation and Tightly-Coupled Graph Optimization.

Recently, generating dense maps in real-time has become a hot research topic in the mobile robotics community, since dense maps can provide more informative and continuous features compared with sparse maps. Implicit depth representation (e.g., the depth code) derived from deep neural networks has been employed in the visual-only or visual-inertial simultaneous localization and mapping (SLAM) systems, which achieve promising performances on both camera motion and local dense geometry estimations from monocular images. However, the existing visual-inertial SLAM systems combined with depth codes are either built on a filter-based SLAM framework, which can only update poses and maps in a relatively small local time window, or based on a loosely-coupled framework, while the prior geometric constraints from the depth estimation network have not been employed for boosting the state estimation. To well address these drawbacks, we propose DiT-SLAM, a novel real-time Dense visual-inertial SLAM with implicit depth representation and Tightly-coupled graph optimization. Most importantly, the poses, sparse maps, and low-dimensional depth codes are optimized with the tightly-coupled graph by considering the visual, inertial, and depth residuals simultaneously. Meanwhile, we propose a light-weight monocular depth estimation and completion network, which is combined with attention mechanisms and the conditional variational auto-encoder (CVAE) to predict the uncertainty-aware dense depth maps from more low-dimensional codes. Furthermore, a robust point sampling strategy introducing the spatial distribution of 2D feature points is also proposed to provide geometric constraints in the tightly-coupled optimization, especially for textureless or featureless cases in indoor environments. We evaluate our system on open benchmarks. The proposed methods achieve better performances on both the dense depth estimation and the trajectory estimation compared to the baseline and other systems.

AGPC-SLAM: Absolute Ground Plane Constrained 3D Lidar SLAM

<h3>Abstract</h3> 3D lidar-based simultaneous localization and mapping (SLAM) is a well-recognized solution for mapping and localization applications. However, the typical 3D lidar sensor (e.g., Velodyne HDL-32E) only provides a very limited field of view vertically. As a result, the vertical accuracy of pose estimation suffers. This paper aims to alleviate this problem by detecting the absolute ground plane to constrain vertical pose estimation. Different from the conventional relative plane constraint, this paper employs the absolute plane distance to refine the position in the z-axis and the norm vector of the ground plane to constrain the attitude drift. Finally, relative positioning from lidar odometry, constraint from ground plane detection, and loop closure are integrated under a proposed factor graph-based 3D lidar SLAM framework (AGPC-SLAM). The effectiveness is verified using several data sets collected in Hong Kong.

Real-Time Dynamic SLAM Algorithm Based on Deep Learning

In the traditional visual simultaneous localization and mapping (SLAM), the strong static assumption leads to a large degradation in the accuracy of visual SLAM in dynamic environments. For this reason, many scholars incorporate semantic segmentation networks into the visual SLAM framework to extract dynamic information in images. However, most semantic segmentation networks consume a lot of computing time due to the large model size, which leads to the algorithm&#x2019;s inability to meet real-time requirements in practical applications. In this paper, a real-time visual SLAM algorithm based on deep learning is proposed. This novel algorithm is based on ORB-SLAM2, and a parallel semantic thread based on the lightweight object detection network YOLOv5s is designed, which enables us to get semantic information in the scene more quickly. In the tracking thread, an optimized homography matrix module is proposed, which utilizes semantic information to optimize and solve the homography matrix so that we can calculate a more accurate optical flow vector. In the optical flow module, the semantic information is used to narrow down the calculation range of the optical flow value to improve the real-time performance of the system, and the dynamic feature points in the image are removed by the optical flow mask to improve the accuracy of the system. Experimental results show that compared with ORB-SLAM2, DynaSLAM, and other excellent visual SLAM algorithms, this algorithm can effectively reduce the absolute trajectory error of visual SLAM in dynamic environments. Compared with other deep learning-based visual SLAM algorithms, the real-time performance of this algorithm is also significantly improved.

Hierarchical Multi-Level Information Fusion for Robust and Consistent Visual SLAM

Vision-based simultaneous localization and mapping (SLAM) has been indispensable for autonomous vehicles. However, in challenging environments, such as sparsely textured parking lots, the state-of-the-art methods often fail to obtain sufficiently reliable features for robust localization and consistent mapping. This paper proposes a multi-level visual information hierarchical fusion SLAM framework based on a multi-camera system. High-level features like object edges are extracted by different segmentation methods. With the guidance of high-level information, low-level key points are selected. The hierarchical fusion structure consists of two parts. High-level visual information is loosely fused with the wheel encoder to estimate initial data association. Then, low-level feature points of multiple views are tightly fused to facilitate robust localization. With such a structure, a consistent map can be built even under the condition that one view is lost. In addition, the lost view can be reinitialized in the single map with other information instead of waiting for re-localization or creating additional maps. Real-time experiments in autonomous parking scenario validate the robustness and consistency of the proposed method.

Visual SLAM Framework Based on Segmentation with the Improvement of Loop Closure Detection in Dynamic Environments

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

EIL‐SLAM: Depth‐enhanced edge‐based infrared‐LiDAR SLAM

AbstractTraditional simultaneous localization and mapping (SLAM) approaches that utilize visible cameras or light detection and rangings (LiDARs) frequently fail in dusty, low‐textured, or completely dark environments. To address this problem, this study proposes a novel approach by tightly coupling perception data from a thermal infrared camera and a LiDAR based on the advantages of the former. However, applying a thermal infrared camera directly to existing SLAM frameworks is difficult because of the sensor differences. Thus, a new infrared‐visual odometry method is developed by utilizing edge points as features to ensure the robustness of the state estimation. Furthermore, an edge‐based infrared‐LiDAR SLAM framework is developed to generate a dense depth map for recovering visual scale and to provide real‐time pose estimation at the same time throughout the day. An infrared‐visual and LiDAR‐integrated place recognition method is also introduced to achieve robust loop closure. Finally, several experiments are performed to illustrate the effectiveness of the proposed approach.

Semantics Aware Dynamic SLAM Based on 3D MODT.

The idea of SLAM (Simultaneous Localization and Mapping) being a solved problem revolves around the static world assumption, even though autonomous systems are gaining environmental perception capabilities by exploiting the advances in computer vision and data-driven approaches. The computational demands and time complexities remain the main impediment in the effective fusion of the paradigms. In this paper, a framework to solve the dynamic SLAM problem is proposed. The dynamic regions of the scene are handled by making use of Visual-LiDAR based MODT (Multiple Object Detection and Tracking). Furthermore, minimal computational demands and real-time performance are ensured. The framework is tested on the KITTI Datasets and evaluated against the publicly available evaluation tools for a fair comparison with state-of-the-art SLAM algorithms. The results suggest that the proposed dynamic SLAM framework can perform in real-time with budgeted computational resources. In addition, the fused MODT provides rich semantic information that can be readily integrated into SLAM.

DV-LOAM: Direct Visual LiDAR Odometry and Mapping

Self-driving cars have experienced rapid development in the past few years, and Simultaneous Localization and Mapping (SLAM) is considered to be their basic capabilities. In this article, we propose a direct vision LiDAR fusion SLAM framework that consists of three modules. Firstly, a two-staged direct visual odometry module, which consists of a frame-to-frame tracking step, and an improved sliding window based thinning step, is proposed to estimate the accurate pose of the camera while maintaining efficiency. Secondly, every time a keyframe is generated, a dynamic objects considered LiDAR mapping module is utilized to refine the pose of the keyframe to obtain higher positioning accuracy and better robustness. Finally, a Parallel Global and Local Search Loop Closure Detection (PGLS-LCD) module that combines visual Bag of Words (BoW) and LiDAR-Iris feature is applied for place recognition to correct the accumulated drift and maintain a globally consistent map. We conducted a large number of experiments on the public dataset and our mobile robot dataset to verify the effectiveness of each module in our framework. Experimental results show that the proposed algorithm achieves more accurate pose estimation than the state-of-the-art methods.

Fault-Diagnosing Deep-Visual-SLAM for 3D Change Object Detection

Although image change detection (ICD) methods provide good detection accuracy for many scenarios, most existing methods rely on place-specific background modeling. The time/space cost for such place-specific models is prohibitive for large-scale scenarios, such as long-term robotic visual simultaneous localization and mapping (SLAM). Therefore, we propose a novel ICD framework that is specifically customized for long-term SLAM. This study is inspired by the multi-map-based SLAM framework, where multiple maps can perform mutual diagnosis and hence do not require any explicit background modeling/model. We extend this multi-map-based diagnosis approach to a more generic single-map-based object-level diagnosis framework (i.e., ICD), where the self-localization module of SLAM, which is the change object indicator, can be used in its original form. Furthermore, we consider map diagnosis on a state-of-the-art deep convolutional neural network (DCN)-based SLAM system (instead of on conventional bag-of-words or landmark-based systems), in which the blackbox nature of the DCN complicates the diagnosis problem. Additionally, we consider a three-dimensional point cloud (PC)-based (instead of typical monocular color image-based) SLAM and adopt a state-of-the-art scan context PC descriptor for map diagnosis for the first time.

Necessary and Sufficient Conditions for Observability of SLAM-Based TDOA Sensor Array Calibration and Source Localization

Sensor array-based systems, which adopt time difference of arrival (TDOA) measurements among the sensors, have found many robotic applications. However, for existing frameworks and systems to be useful, the sensor array needs to be calibrated accurately. Of particular interest in this article are microphone array-based robot audition systems. In our recent work, by using a moving sound source, and the graph-based formulation of simultaneous localization and mapping (SLAM), we have proposed a framework for joint sound source localization and calibration of microphone array geometrical information, together with the estimation of microphone time offset and clock difference/drift rates. However, a thorough study on the identifiability question, termed observability analysis here, in the SLAM framework for microphone array calibration and sound source localization, is still lacking in the literature. In this article, we will fill the abovementioned gap via a Fisher information matrix approach. Motivated by the equivalence between the full column rankness of the Fisher information matrix and the Jacobian matrix, we leverage the structure of the latter associated with the SLAM formulation, and present necessary and sufficient conditions guaranteeing its full column rankness, which lead to parameter identifiability. We have thoroughly discussed the 3-D case with asynchronous (with both time offset and clock drifts, or with only one of them) and synchronous microphone array, respectively. These conditions are closely related to the motion varieties of the sound source and the microphone array configuration, and have intuitive and physical interpretations. Based on the established conditions, we have also discovered some particular cases where observability is impossible. Connections with calibration of other sensors will also be discussed, amongst others. To our best knowledge, this is the first systematic work on observability analysis of SLAM-based microphone array calibration and sound source localization. The tools and concepts used in this article are also applicable to other TDOA sensing modalities such as ultrawide band (UWB) sensors.