SLAM System Research Articles

A SLAM system based on Hidden Markov Models

We present a graph SLAM system based on Hidden Markov Models (HMM) where the sensor readings are represented with different symbols using a number of clustering techniques; then, the symbols are fused as a single prediction, to improve the accuracy rate, using a Dual HMM. Our system’s versatility allows to work with different types of sensors or fusion of sensors, and to implement, either active or passive, graph SLAM. The Toyota HSR (Human Support Robot) robot was used to generate the data set in both real and simulated competition environments. We tested our system in the kidnapped robot problem by training a representation, improving it online, and, finally, solving the SLAM problem.

ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM

This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main novelty is a feature-based tightly-integrated visual-inertial SLAM system that fully relies on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization phase. The result is a system that operates robustly in real-time, in small and large, indoor and outdoor environments, and is 2 to 5 times more accurate than previous approaches. The second main novelty is a multiple map system that relies on a new place recognition method with improved recall. Thanks to it, ORB-SLAM3 is able to survive to long periods of poor visual information: when it gets lost, it starts a new map that will be seamlessly merged with previous maps when revisiting mapped areas. Compared with visual odometry systems that only use information from the last few seconds, ORB-SLAM3 is the first system able to reuse in all the algorithm stages all previous information. This allows to include in bundle adjustment co-visible keyframes, that provide high parallax observations boosting accuracy, even if they are widely separated in time or if they come from a previous mapping session. Our experiments show that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature, and significantly more accurate. Notably, our stereo-inertial SLAM achieves an average accuracy of 3.6 cm on the EuRoC drone and 9 mm under quick hand-held motions in the room of TUM-VI dataset, a setting representative of AR/VR scenarios. For the benefit of the community we make public the source code.

Study of Indoor Visual SLAM System for Semi-autonomous Robot Platform

This study propose the use of heterogeneous visual landmarks, points and line segments, to achieve effective cooperation in indoor SLAM environments. In order to achieve un-delayed initialization required by the bearing-only observations, the well-known inverse-depth parameterization is adopted to estimate 3D points. Similarly, to estimate 3D line segments, we present a novel parameterization based on anchored Plücker coordinates, to which extensible endpoints are added

A Feature Based Laser SLAM Using Rasterized Images of 3D Point Cloud

An accurate and computationally efficient SLAM algorithm is vital for autonomous vehicles. Most modern SLAM systems use feature detection approaches to limit computational requirements. Feature detection through a 3D point cloud can be a computationally challenging task. In this paper, we propose a feature-based SLAM algorithm using 2D image projections of the 3D laser point cloud. We use a camera parameters matrix to rasterize the 3D point cloud to an image. Then ORB feature detector is applied to these images. The proposed method gives repeatable and stable features in a variety of environments. Based on such features, we can estimate the 6dof pose of the robot. For loop detection, we employ a 2-step approach, i.e., nearest key-frame detection and loop candidate verification by matching features extracted from rasterized LIDAR images. We evaluate the proposed system with implementation on the KITTI dataset. Through experimental results, we show that the algorithm presented in this paper can substantially reduce the computational cost of feature detection from the point cloud and the whole SLAM system while giving accurate results.

Instant Visual Odometry Initialization for Mobile AR.

Mobile AR applications benefit from fast initialization to display world-locked effects instantly. However, standard visual odometry or SLAM algorithms require motion parallax to initialize (see Figure 1) and, therefore, suffer from delayed initialization. In this paper, we present a 6-DoF monocular visual odometry that initializes instantly and without motion parallax. Our main contribution is a pose estimator that decouples estimating the 5-DoF relative rotation and translation direction from the 1-DoF translation magnitude. While scale is not observable in a monocular vision-only setting, it is still paramount to estimate a consistent scale over the whole trajectory (even if not physically accurate) to avoid AR effects moving erroneously along depth. In our approach, we leverage the fact that depth errors are not perceivable to the user during rotation-only motion. However, as the user starts translating the device, depth becomes perceivable and so does the capability to estimate consistent scale. Our proposed algorithm naturally transitions between these two modes. Our second contribution is a novel residual in the relative pose problem to further improve the results. The residual combines the Jacobians of the functional and the functional itself and is minimized using a Levenberg-Marquardt optimizer on the 5-DoF manifold. We perform extensive validations of our contributions with both a publicly available dataset and synthetic data. We show that the proposed pose estimator outperforms the classical approaches for 6-DoF pose estimation used in the literature in low-parallax configurations. Likewise, we show our relative pose estimator outperforms state-of-the-art approaches in an odometry pipeline configuration where we can leverage initial guesses. We release a dataset for the relative pose problem using real data to facilitate the comparison with future solutions for the relative pose problem. Our solution is either used as a full odometry or as a pre-SLAM component of any supported SLAM system (ARKit, ARCore) in world-locked AR effects on platforms such as Instagram and Facebook.

PSL-SLAM: a monocular SLAM system using points and structure lines in Manhattan World

PSL-SLAM: a monocular SLAM system using points and structure lines in Manhattan World

CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations

We propose a novel dense mapping framework for sparse visual SLAM systems which leverages a compact scene representation. State-of-the-art sparse visual SLAM systems provide accurate and reliable estimates of the camera trajectory and locations of landmarks. While these sparse maps are useful for localization, they cannot be used for other tasks such as obstacle avoidance or scene understanding. In this letter we propose a dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses, keyframes and sparse points produced by the SLAM system and predicts a dense depth image for every keyframe. We build on CodeSLAM [1] and use a variational autoencoder (VAE) which is conditioned on intensity, sparse depth and reprojection error images from sparse SLAM to predict an uncertainty-aware dense depth map. The use of a VAE then enables us to refine the dense depth images through multi-view optimization which improves the consistency of overlapping frames. Our mapper runs in a separate thread in parallel to the SLAM system in a loosely coupled manner. This flexible design allows for integration with arbitrary metric sparse SLAM systems without delaying the main SLAM process. Our dense mapper can be used not only for local mapping but also globally consistent dense 3D reconstruction through TSDF fusion. We demonstrate our system running with ORB-SLAM3 and show accurate dense depth estimation which could enable applications such as robotics and augmented reality.

A Life-Long SLAM Approach Using Adaptable Local Maps Based on Rasterized LIDAR Images

Most real-time autonomous robot applications require a robot to traverse through a dynamic space for a long time. In some cases, a robot needs to work in the same environment. Such applications give rise to the problem of a life-long SLAM system. Life-long SLAM presents two main challenges i.e. the tracking should not fail in a dynamic environment and the need for a robust and efficient mapping strategy. The system should update maps with new information; while also keeping track of older observations. But, mapping for a long time can require higher computational requirements. In this paper, we propose a solution to the problem of life-long SLAM. We represent the global map as a set of rasterized images of local maps along with a map management system responsible for updating local maps and keeping track of older values. We also present an efficient approach of using the bag of visual words method for loop closure detection and relocalization. We evaluate the performance of our system on the KITTI dataset and an indoor dataset. Our loop closure system reported recall and precision of above 90 percent. The computational cost of our system is much lower as compared to state-of-the-art methods. Our method reports lower computational requirements even for long-term operation.

LiDAR SLAM With Plane Adjustment for Indoor Environment

Planes ubiquitously exist in the indoor environment. This letter presents a real-time and low-drift LiDAR SLAM system using planes as the landmark for the indoor environment. Our algorithm includes three components: localization, local mapping and global mapping. The localization component performs real-time and global registration, instead of the scan-to-scan registration adopted in the state-of-the-art LiDAR odometry and mapping (LOAM) framework that yields lower fidelity poses. The local mapping component optimizes poses of the keyframes within a sliding window and parameters of the planes observed by these keyframes. The global mapping component conducts global plane adjustment (GPA) that jointly refines plane parameters and keyframe poses. The GPA is triggered when planes are revisited, rather than a place is revisited. This can establish constraints among remote places, and correct the drift without having to go back to a previously visited place. We adopt the point-to-plane distance to construct the cost functions of all the three components. Although this distance results in a large-scale least-squares problem that seems not suitable for real-time applications, we propose efficient algorithms to solve the resulting minimization problems by exploiting the special structure of the point-to-plane distance. Experimental results show that our algorithm achieves real-time performance and outperforms state-of-the-art LiDAR SLAM algorithms.

Keypoint Matching for Point Cloud Registration Using Multiplex Dynamic Graph Attention Networks

The registration of point clouds is a key ingredient of LiDAR-based SLAM systems and mapping approaches. A challenging task in this context is finding the right data association between 3D points. This paper proposes a novel and flexible graph network architecture to tackle the keypoint matching problem in an end-to-end fashion. Each layer of our multiplex dynamic graph attention network (MDGAT) utilizes an attention mechanism to dynamically construct a multiplex graph and reasons about the contextual information based on the point cloud data. It enriches the feature representation by recovering local information and by aggregating information along with the connections. We also design a scan matcher called MDGAT-matcher, which treats the registration problem as an optimal transport problem and uses the predictions of MDGAT as the cost. It builds upon sparse keypoints extracted from pairs of LiDAR scans. Eventually, MDGAT-matcher finds high-quality correspondences and at the same time handles non-matching points appropriately. We train our matcher using a novel gap loss guiding the network to learn a discriminative cognition about matching and non-matching 3D points. We thoroughly test our approach on the KITTI odometry benchmark. The experiments presented in this paper suggest that our approach outperforms state-of-the-art matching approaches and achieves a substantial improvement.

RS-SLAM: A Robust Semantic SLAM in Dynamic Environments Based on RGB-D Sensor

Most of the state-of-the-art visual simultaneous localization and mapping (V-SLAM) algorithms perform well in structured environments. However, the assumption of a static world limits further practical application. In this paper, we present RS-SLAM, a robust semantic RGB-D SLAM system. RS-SLAM is able to detect both moving and movable objects for high accuracy localization. A semantic segmentation model is applied to recognize the dynamic objects. Specifically, the segmentation result is refined using the context information based on Bayesian update, which provides the more accurate extraction of the interested region. With the refined results, a kind of movable object detection approach is developed. In addition, an updating strategy during mapping is introduced and a 3D semantic OctoMap only with a static world is built in real-time. Our SLAM system is evaluated both on public datasets and real-world environments. The results show that RS-SLAM outperforms the accuracy of standard visual SLAM baselines and is capable of constructing a clean static semantic OctoMap in a dynamic environment.

A Semantic SLAM System for Catadioptric Panoramic Cameras in Dynamic Environments.

When a traditional visual SLAM system works in a dynamic environment, it will be disturbed by dynamic objects and perform poorly. In order to overcome the interference of dynamic objects, we propose a semantic SLAM system for catadioptric panoramic cameras in dynamic environments. A real-time instance segmentation network is used to detect potential moving targets in the panoramic image. In order to find the real dynamic targets, potential moving targets are verified according to the sphere’s epipolar constraints. Then, when extracting feature points, the dynamic objects in the panoramic image are masked. Only static feature points are used to estimate the pose of the panoramic camera, so as to improve the accuracy of pose estimation. In order to verify the performance of our system, experiments were conducted on public data sets. The experiments showed that in a highly dynamic environment, the accuracy of our system is significantly better than traditional algorithms. By calculating the RMSE of the absolute trajectory error, we found that our system performed up to 96.3% better than traditional SLAM. Our catadioptric panoramic camera semantic SLAM system has higher accuracy and robustness in complex dynamic environments.

OverlapNet: a siamese network for computing LiDAR scan similarity with applications to loop closing and localization

Localization and mapping are key capabilities of autonomous systems. In this paper, we propose a modified Siamese network to estimate the similarity between pairs of LiDAR scans recorded by autonomous cars. This can be used to address both, loop closing for SLAM and global localization. Our approach utilizes a deep neural network exploiting different cues generated from LiDAR data. It estimates the similarity between pairs of scans using the concept of image overlap generalized to range images and furthermore provides a relative yaw angle estimate. Based on such predictions, our method is able to detect loop closures in a SLAM system or to globally localize in a given map. For loop closure detection, we use the overlap prediction as the similarity measurement to find loop closure candidates and integrate the candidate selection into an existing SLAM system to improve the mapping performance. For global localization, we propose a novel observation model using the predictions provided by OverlapNet and integrate it into a Monte-Carlo localization framework. We evaluate our approach on multiple datasets collected using different LiDAR scanners in various environments. The experimental results show that our method can effectively detect loop closures surpassing the detection performance of state-of-the-art methods and that it generalizes well to different environments. Furthermore, our method reliably localizes a vehicle in typical urban environments globally using LiDAR data collected in different seasons.

Blitz-SLAM: A semantic SLAM in dynamic environments

Static environment is a prerequisite for most of visual simultaneous localization and mapping systems. Such a strong assumption limits the practical application of most existing SLAM systems. When moving objects enter the camera’s view field, dynamic matching points will directly interrupt the camera localization, and the noise blocks formed by moving objects will contaminate the constructed map. In this paper, a semantic SLAM system working in indoor dynamic environments named Blitz-SLAM is proposed. The noise blocks in the local point cloud are removed by combining the advantages of semantic and geometric information of mask, RGB and depth images. The global point cloud map can be obtained by merging the local point clouds. We evaluate Blitz-SLAM on the TUM RGB-D dataset and in the real-world environment. The experimental results demonstrate that Blitz-SLAM can work robustly in dynamic environments and generate a clean and accurate global point cloud map simultaneously.

Panoramic annular SLAM with loop closure and global optimization.

In this paper, we propose panoramic annular simultaneous localization and mapping (PA-SLAM), a visual SLAM system based on a panoramic annular lens. A hybrid point selection strategy is put forward in the tracking front end, which ensures repeatability of key points and enables loop closure detection based on the bag-of-words approach. Every detected loop candidate is verified geometrically, and the Sim(3) relative pose constraint is estimated to perform pose graph optimization and global bundle adjustment in the back end. A comprehensive set of experiments on real-world data sets demonstrates that the hybrid point selection strategy allows reliable loop closure detection, and the accumulated error and scale drift have been significantly reduced via global optimization, enabling PA-SLAM to reach state-of-the-art accuracy while maintaining high robustness and efficiency.

Asynchronous event feature generation and tracking based on gradient descriptor for event cameras

Recently, the event camera has become a popular and promising vision sensor in the research of simultaneous localization and mapping and computer vision owing to its advantages: low latency, high dynamic range, and high temporal resolution. As a basic part of the feature-based SLAM system, the feature tracking method using event cameras is still an open question. In this article, we present a novel asynchronous event feature generation and tracking algorithm operating directly on event-streams to fully utilize the natural asynchronism of event cameras. The proposed algorithm consists of an event-corner detection unit, a descriptor construction unit, and an event feature tracking unit. The event-corner detection unit addresses a fast and asynchronous corner detector to extract event-corners from event-streams. For the descriptor construction unit, we propose a novel asynchronous gradient descriptor inspired by the scale-invariant feature transform descriptor, which helps to achieve quantitative measurement of similarity between event feature pairs. The construction of the gradient descriptor can be decomposed into three stages: speed-invariant time surface maintenance and extraction, principal orientation calculation, and descriptor generation. The event feature tracking unit combines the constructed gradient descriptor and an event feature matching method to achieve asynchronous feature tracking. We implement the proposed algorithm in C++ and evaluate it on a public event dataset. The experimental results show that our proposed method achieves improvement in terms of tracking accuracy and real-time performance when compared with the state-of-the-art asynchronous event-corner tracker and with no compromise on the feature tracking lifetime.

DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM

The assumption of scene rigidity is common in visual SLAM algorithms. However, it limits their applicability in populated real-world environments. Furthermore, most scenarios including autonomous driving, multi-robot collaboration and augmented/virtual reality, require explicit motion information of the surroundings to help with decision making and scene understanding. We present in this paper DynaSLAM II, a visual SLAM system for stereo and RGB-D camera configurations that tightly integrates the multi-object tracking capability. DynaSLAM II makes use of instance semantic segmentation and ORB features to track dynamic objects. The structures of the static scene and the dynamic objects are optimized jointly with the trajectories of both the camera and the moving agents within a novel bundle adjustment proposal. The 3D bounding boxes of the objects are also estimated and loosely optimized within a fixed temporal window. We demonstrate that tracking dynamic objects does not only provide rich clues for scene understanding but can be also beneficial for camera tracking.

GLO-SLAM: a slam system optimally combining GPS and LiDAR odometry

PurposeLarge-scale and precise three-dimensional (3D) map play an important role in autonomous driving and robot positioning. However, it is difficult to get accurate poses for mapping. On one hand, the global positioning system (GPS) data are not always reliable owing to multipath effect and poor satellite visibility in many urban environments. In another hand, the LiDAR-based odometry has accumulative errors. This paper aims to propose a novel simultaneous localization and mapping (SLAM) system to obtain large-scale and precise 3D map.Design/methodology/approachThe proposed SLAM system optimally integrates the GPS data and a LiDAR odometry. In this system, two core algorithms are developed. To effectively verify reliability of the GPS data, VGL (the abbreviation of Verify GPS data with LiDAR data) algorithm is proposed and the points from LiDAR are used by the algorithm. To obtain accurate poses in GPS-denied areas, this paper proposes EG-LOAM algorithm, a LiDAR odometry with local optimization strategy to eliminate the accumulative errors by means of reliable GPS data.FindingsOn the KITTI data set and the customized outdoor data set, the system is able to generate high-precision 3D map in both GPS-denied areas and areas covered by GPS. Meanwhile, the VGL algorithm is proved to be able to verify reliability of the GPS data with confidence and the EG-LOAM outperform the state-of-the-art baselines.Originality/valueA novel SLAM system is proposed to obtain large-scale and precise 3D map. To improve the robustness of the system, the VGL algorithm and the EG-LOAM are designed. The whole system as well as the two algorithms have a satisfactory performance in experiments.

Smartphone-Based Indoor Visual Navigation with Leader-Follower Mode

Existing indoor navigation solutions usually require pre-deployed comprehensive location services with precise indoor maps and, more importantly, all rely on dedicatedly installed or existing infrastructure. In this article, we present Pair-Navi, an infrastructure-free indoor navigation system that circumvents all these requirements by reusing a previous traveler’s (i.e., leader) trace experience to navigate future users (i.e., followers) in a Peer-to-Peer mode. Our system leverages the advances of visual simultaneous localization and mapping ( SLAM ) on commercial smartphones. Visual SLAM systems, however, are vulnerable to environmental dynamics in the precision and robustness and involve intensive computation that prohibits real-time applications. To combat environmental changes, we propose to cull non-rigid contexts and keep only the static and rigid contents in use. To enable real-time navigation on mobiles, we decouple and reorganize the highly coupled SLAM modules for leaders and followers. We implement Pair-Navi on commodity smartphones and validate its performance in three diverse buildings and two standard datasets (TUM and KITTI). Our results show that Pair-Navi achieves an immediate navigation success rate of 98.6%, which maintains as 83.4% even after 2 weeks since the leaders’ traces were collected, outperforming the state-of-the-art solutions by >50%. Being truly infrastructure-free, Pair-Navi sheds lights on practical indoor navigations for mobile users.

A novel methodology for the path alignment of visual SLAM in indoor construction inspection

Path alignment is the process of mapping an indoor construction inspection path reconstructed by a visual SLAM system onto a 2D map with user interaction required to pinpoint at least two common tie points. In practice, more points are often needed due to path distortions and linear transformations, potentially resulting in reduced productivity. This paper proposes a methodology that combines two novel algorithms for the path alignment: (1) PCA_STAN_ALGO applies principal component analysis to remove path distortions caused by the xz plane of a camera coordinate system not being parallel to the floor plane; and (2) GRPX_TRANS utilizes a graphical user interface to facilitate the path alignment. The proposed methodology enables the users to utilize just two tie points for successful path alignment. An experimental study showed that applying both PCA_STAN_ALGO and GRPX_TRANS saved about 50% in time compared to using only GRPX_TRANS, a result of needing minimal moving points.