Robust Loop Closure Detection Research Articles

Robust loop closure detection and relocalization with semantic-line graph matching constraints in indoor environments

Loop closure detection (LCD) plays an essential role in the Simultaneous Localization and Mapping (SLAM) process, effectively reducing cumulative trajectory errors. However, conventional LCD methods often encounter challenges when dealing with variations in illumination, changes in viewpoint, and environments with weak textures. This is due to their reliance on low-level geometric or image features. To address these issues, we propose a robust LCD method named SL-LCD, which integrates semantic information and line features to fully leverage the semantic content and line attributes within indoor scenes, thereby establishing a reliable feature correspondence between query images and loop closure images. For the retrieval of candidate closed-loop images, we construct a semantic-line-segment topological graph and introduce a graph matching algorithm to perform the LCD task. This approach fully exploits image features and spatial information to achieve closed-loop detection in complex indoor scenes. Furthermore, we present a semantic voxel-based generalized ICP (SVGICP) closed-loop relocalization algorithm tailored for challenging and complex indoor scenes, enhancing the accuracy of closed-loop relocalization in such scenarios. Experimental results demonstrate that the SL-LCD algorithm proposed in this paper surpasses state-of-the-art methods, accurately detecting closed loops, and effectively eliminating trajectory drift.

YES-SLAM: YOLOv7-enhanced-semantic visual SLAM for mobile robots in dynamic scenes

In dynamic scenes, moving objects will cause a significant error accumulation in robot’s pose estimation, and might even lead to tracking loss. In view of these problems, this paper proposes a semantic visual simultaneous localization and mapping algorithm based on YOLOv7. First, a light-weight network YOLOv7 is employed to acquire the semantic information of different objects in the scene, and flood filling and edge-enhanced techniques are combined to accurately and quickly separate the dynamic feature points from the extracted feature point set. In this way, the obtained static feature points with high-confidence are used to achieve the accurate estimation of robot’s pose. Then, according to the semantic information of YOLOv7, the motion magnitude of the robot, and the number of dynamic feature points in camera’s field-of-view, a high-performance keyframe selection strategy is constructed. On this basis, a robust loop closure detection method is developed by introducing the semantic information into the bag-of-words model, and global bundle adjustment optimization is performed on all keyframes and map points to obtain a global consistent pose graph. Finally, YOLOv7 is further utilized to carry out semantic segmentation on the keyframes, remove the dynamic objects in its semantic mask, and combine the point cloud pre-processing and octree map to build a 3D navigation semantic map. A series of simulations on TUM dataset and a case study in real scene clearly demonstrated the performance superiority of the proposed algorithms.

Mapping with Monocular Camera Sensor under Adversarial Illumination for Intelligent Vehicles.

High-precision maps are widely applied in intelligent-driving vehicles for localization and planning tasks. The vision sensor, especially monocular cameras, has become favoured in mapping approaches due to its high flexibility and low cost. However, monocular visual mapping suffers from great performance degradation in adversarial illumination environments such as on low-light roads or in underground spaces. To address this issue, in this paper, we first introduce an unsupervised learning approach to improve keypoint detection and description on monocular camera images. By emphasizing the consistency between feature points in the learning loss, visual features in dim environment can be better extracted. Second, to suppress the scale drift in monocular visual mapping, a robust loop-closure detection scheme is presented, which integrates both feature-point verification and multi-grained image similarity measurements. With experiments on public benchmarks, our keypoint detection approach is proven robust against varied illumination. With scenario tests including both underground and on-road driving, we demonstrate that our approach is able to reduce the scale drift in reconstructing the scene and achieve a mapping accuracy gain of up to 0.14 m in textureless or low-illumination environments.

Fast and robust loop-closure detection using deep neural networks and matrix transformation for a visual SLAM system

We investigate the loop-closure detection problem for a visual simultaneous localization and mapping (SLAM) system. Generally, a deep neural network is used to solve loop-closure detection problems. However, this method is time-consuming and occupies a large memory in practice, primarily owing to the high-dimensional nature of the data involved. Herein, we combine a pretrained convolutional neural network (CNN) model with a convolutional autoencoder to acquire a low-dimensional CNN vector as an image representation. Subsequently, the loop-closure problem is simulated by the single-image nearest-neighbor search method based on the cosine distance of the low-dimensional CNN vectors. In addition, to solve the perceptual aliasing problem in loop-closure detection, we empirically propose an efficient method for eliminating false positive loop closures by appropriately decomposing and reconstructing the similarity matrix. Furthermore, we verify through experiments that our method can effectively reduce the time and memory costs of loop-closure detection tasks while ensuring detection accuracy. Therefore, this method can provide a feasible solution to improve the real-time performance and robustness of visual SLAM systems.

Fast and Robust Loop-Closure Detection via Convolutional Auto-Encoder and Motion Consensus

Loop-closure detection is an indispensable module in the visual simultaneous localization and mapping (vSLAM) system. It typically consists of three main steps: image representation, loop-closure candidate selection, and loop-closure event verification. This article proposes a novel approach for loop-closure detection. In particular, we first introduce a lightweight convolutional auto-encoder network trained by the deep perceptual similarity loss for image representation. We then propose an image-to-sequence selection approach based on place sequence division and distance-weighted voting for loop-closure candidate selection. Furthermore, we propose a motion vector consensus constraint to improve locality preserving matching, which can be used for efficient loop-closure event verification that is robust for various complex environments. Extensive experiments have been conducted on four publicly available datasets. The results demonstrate that our method is able to achieve better recall performance than the state-of-the-art and meet the real-time requirement of vSLAM systems.

CVIDS: A Collaborative Localization and Dense Mapping Framework for Multi-Agent Based Visual-Inertial SLAM

Nowadays, visual SLAM (Simultaneous Localization And Mapping) has become a hot research topic due to its low costs and wide application scopes. Traditional visual SLAM frameworks are usually designed for single-agent systems, completing both the localization and the mapping with sensors equipped on a single robot or a mobile device. However, the mobility and work capacity of the single agent are usually limited. In reality, robots or mobile devices sometimes may be deployed in the form of clusters, such as drone formations, wearable motion capture systems, and so on. As far as we know, existing SLAM systems designed for multi-agents are still sporadic, and most of them have non-negligible limitations in functions. Specifically, on one hand, most of the existing multi-agent SLAM systems can only extract some key features and build sparse maps. On the other hand, schemes that can reconstruct the environment densely cannot get rid of the dependence on depth sensors, such as RGBD cameras or LiDARs. Systems that can yield high-density maps just with monocular camera suites are temporarily lacking. As an attempt to fill in the research gap to some extent, we design a novel collaborative SLAM system, namely CVIDS (Collaborative Visual-Inertial Dense SLAM), which follows a centralized and loosely coupled framework and can be integrated with any existing Visual-Inertial Odometry (VIO) to accomplish the co-localization and the dense reconstruction. Integrating our proposed robust loop closure detection module and two-stage pose-graph optimization pipeline, the co-localization module of CVIDS can estimate the poses of different agents in a unified coordinate system efficiently from the packed images and local poses sent by the client-ends of different agents. Besides, our motion-based dense mapping module can effectively recover the 3D structures of selected keyframes and then fuse their depth information to the global map for reconstruction. The superior performance of CVIDS is corroborated by both quantitative and qualitative experimental results. To make our results reproducible, the source code has been released at https://cslinzhang.github.io/CVIDS.

Semantic loop closure detection based on graph matching in multi-objects scenes

Robust loop-closure detection is essential for visual SLAM. Traditional methods often focus on the geometric and visual features in most scenes but ignore the semantic information provided by objects. Based on this consideration, we present a strategy that models the visual scene as semantic sub-graph by only preserving the semantic and geometric information from object detection. To align two sub-graphs efficiently, we use a sparse Kuhn–Munkres algorithm to speed up the search for correspondence among nodes. The shape similarity and the Euclidean distance between objects in the 3-D space are leveraged unitedly to measure the image similarity through graph matching. Furthermore, the proposed approach has been analyzed and compared with the state-of-the-art algorithms at several datasets as well as two indoor real scenes, where the results indicate that our semantic graph-based representation without extracting visual features is feasible for loop-closure detection at potential and competitive precision.

Robust Loop Closure Detection Integrating Visual–Spatial–Semantic Information via Topological Graphs and CNN Features

Loop closure detection is a key module for visual simultaneous localization and mapping (SLAM). Most previous methods for this module have not made full use of the information provided by images, i.e., they have only used the visual appearance or have only considered the spatial relationships of landmarks; the visual, spatial and semantic information have not been fully integrated. In this paper, a robust loop closure detection approach integrating visual–spatial–semantic information is proposed by employing topological graphs and convolutional neural network (CNN) features. Firstly, to reduce mismatches under different viewpoints, semantic topological graphs are introduced to encode the spatial relationships of landmarks, and random walk descriptors are employed to characterize the topological graphs for graph matching. Secondly, dynamic landmarks are eliminated by using semantic information, and distinctive landmarks are selected for loop closure detection, thus alleviating the impact of dynamic scenes. Finally, to ease the effect of appearance changes, the appearance-invariant descriptor of the landmark region is extracted by a pre-trained CNN without the specially designed manual features. The proposed approach weakens the influence of viewpoint changes and dynamic scenes, and extensive experiments conducted on open datasets and a mobile robot demonstrated that the proposed method has more satisfactory performance compared to state-of-the-art methods.

Illumination Robust Loop Closure Detection with the Constraint of Pose

Background: Loop closure detection is a crucial part in robot navigation and simultaneous location and mapping (SLAM). Appearance-based loop closure detection still faces many challenges, such as illumination changes, perceptual aliasing and increasing computational complexity. Methods: In this paper, we proposed a visual loop closure detection algorithm that combines illumination robust descriptor DIRD and odometry information. In this algorithm, a new distance function is built by fusing the Euclidean distance function and Mahalanobis distance function, which integrates the pose uncertainty of body and can dynamically adjust the threshold of potential loop closure locations. Then, potential locations are verified by calculating the similarity of DIRD descriptors. Results: The proposed algorithm is evaluated on KITTI and EuRoC datasets, and is compared with SeqSLAM algorithm, which is one of the state of the art loop closure detection algorithms. The results show that the proposed algorithm could effectively reduce the computing time and get better performance on P-R curve. Conclusion: The new loop closure detection method makes full use of odometry information and image appearance information. The application of the new distance function can effectively reduce the missed detection caused by odometry error accumulation. The algorithm does not require extracting image features or learning stage, and can realize real-time detection and run on the platform with limited computational power.

A novel loop closure detection method with the combination of points and lines based on information entropy

AbstractVisual simultaneous localization and mapping (visual‐SLAM) is a prominent technology for autonomous navigation of mobile robots. As a significant requirement for visual‐SLAM, loop closure detection (LCD) involves recognizing a revisited place, thereby helping visual‐SLAM eliminate accumulated errors and obtain consistent maps. Conventional LCD approaches mainly rely on point features to detect the loop. In challenging environments, the performance of point‐based LCD degrades, especially in low‐texture and perceptual aliasing environments. This paper presents a novel point and line‐based LCD. This approach allows for more robust LCD under an environment where point features are scarce and false‐positive loops are easily detected. First, point and line features are extracted to construct visual vocabularies (a point‐based vocabulary and a line‐based vocabulary) by using the bag‐of‐visual‐words model. Second, a novel weighting scheme is proposed by leveraging information entropy to combine the point‐based similarity score and line‐based similarity score to further improve the similarity evaluation accuracy of two images. Finally, a feature matching coherence check is added to determine the loop closure candidate searching area, which avoids false‐positive loops caused by a decrease in the robot's velocity or because the system is still. We compare the proposed method with point‐based LCD, line‐based LCD, and PL‐based (combining points and lines with the feature's number and dispersion) methods on public data sets in terms of precision and recall. The results reveal that the proposed method offers impressive results compared to other approaches.

Search and rescue under the forest canopy using multiple UAVs

We present a multi-robot system for GPS-denied search and rescue under the forest canopy. Forests are particularly challenging environments for collaborative exploration and mapping, in large part due to the existence of severe perceptual aliasing which hinders reliable loop closure detection for mutual localization and map fusion. Our proposed system features unmanned aerial vehicles (UAVs) that perform onboard sensing, estimation, and planning. When communication is available, each UAV transmits compressed tree-based submaps to a central ground station for collaborative simultaneous localization and mapping (CSLAM). To overcome high measurement noise and perceptual aliasing, we use the local configuration of a group of trees as a distinctive feature for robust loop closure detection. Furthermore, we propose a novel procedure based on cycle consistent multiway matching to recover from incorrect pairwise data associations. The returned global data association is guaranteed to be cycle consistent, and is shown to improve both precision and recall compared with the input pairwise associations. The proposed multi-UAV system is validated both in simulation and during real-world collaborative exploration missions at NASA Langley Research Center.

Integrating global and local image features for enhanced loop closure detection in RGB-D SLAM systems

Loop closure detection is essential for simultaneous localization and mapping systems to decrease accumulating drift of trajectory estimations. Robust loop closure detection is specifically important in large-scale mapping, but it gets more challenging as the mapping environment grows. This paper proposes a SLAM system utilizing a two-pass loop closure detection method to improve mapping accuracy in large-scale environments. The proposed system finds loop closure candidates by employing global and local image features together. After selecting a group of candidates by similarity of global features, the system applies keypoint matching on this group to improve scene matching accuracy and determines loop closure candidates. We extensively evaluate the system on the widely used TUM RGB-D dataset, which contains sequences of small to large-scale indoor environments, with respect to different parameter combinations. The results show that the proposed method increases accuracy substantially and achieves large-scale mapping with acceptable overhead.

Robust Loop Closure Detection Using Bayes Filters and CNN Features

This paper focuses on loop-closure detection (LCD) for a visual simultaneous localization and mapping (SLAM) system. We present a strategy that combines a Bayes filter and features from a pre-trained convolution neural network (CNN) to perform LCD. Rather than using features from only one layer, we fuse features from multiple layers based on spatial pyramid pooling. A flexible Bayes model is then formulated to integrate the sequential information and similarities that are computed by features at different scales. The introduction of a penalty factor and bidirectional propagation enables our approach to handle complex trajectories. We present extensive experiments on challenging datasets, and we compare our approach to state-of-the-art methods, to evaluate it. The results show that our approach can ensure remarkable performance under severe condition changes and handle trajectories that have different characteristics. We also show the advantages of Bayes filters over sequence matching in the experiments, and we analyze our feature fusion strategy by visualizing the activations of the CNN.

Formula omitted]-SLAM: A fast RGB-D SLAM approach for large indoor environments

We present an RGB-D based SLAM system capable of building consistent 3D maps of large indoor environments. The system performs motion estimation by using keypoint correspondences between frames. A novel data association approach is developed for accurate mapping with robust loop closure detection. To achieve this, the proposed method uses a Keyframe Autocorrelogram Database and an adaptive thresholding technique. Keyframe Autocorrelogram Database indexes and clusters past frames based on their spatial color correlations and returns short and long loop closure candidates as k-sized clusters by using a data structure that can effectively handle high dimensional data. The adaptive thresholding technique, which was introduced in our previous work, filters out more outliers in the clusters and selects better loop closure candidates. The proposed approach incrementally produces environment maps without needing any training step. The system is tested on widely used public datasets with a large group of sequences. The experimental results show the robustness and efficiency of our system in challenging conditions, in comparison to other state-of-the-art systems. The system produces accurate maps for medium and large-scale environments with stable and fast processing times on CPU. Thus, the proposed approach makes possible real-time operation with efficient implementations.

CNN Feature Boosted SeqSLAM for Real‐Time Loop Closure Detection

This paper proposes an efficient and robust Loop closure detection (LCD) method based on Convolutional neural network (CNN) feature. The primary method is called SeqCNNSLAM, in which both the outputs of the intermediate layer of a pre-trained CNN and the outputs of traditional sequence-based matching procedure are incorporated, making it possible to handle the viewpoint and condition variance properly. An acceleration algorithm for SeqCNNSLAM is developed to reduce the search range for the current image, resulting in a new LCD method called A-SeqCNNSLAM. To improve the applicability of A-SeqCNNSLAM to new environments, O-SeqCNNSLAM is proposed for online parameters adjustment in A-SeqCNNSLAM. In addition to the above work, we further put forward a promising idea to enhance SeqSLAM by integrating the both CNN features and VLAD's advantages called patch based SeqCNNSLAM (P-SeqCNNSLAM), and provide some preliminary experimental results to reveal its performance.

Sequence-based sparse optimization methods for long-term loop closure detection in visual SLAM

Loop closure detection is one of the most important module in Simultaneously Localization and Mapping (SLAM) because it enables to find the global topology among different places. A loop closure is detected when the current place is recognized to match the previous visited places. When the SLAM is executed throughout a long-term period, there will be additional challenges for the loop closure detection. The illumination, weather, and vegetation conditions can often change significantly during the life-long SLAM, resulting in the critical strong perceptual aliasing and appearance variation problems in loop closure detection. In order to address this problem, we propose a new Robust Multimodal Sequence-based (ROMS) method for robust loop closure detection in long-term visual SLAM. A sequence of images is used as the representation of places in our ROMS method, where each image in the sequence is encoded by multiple feature modalites so that different places can be recognized discriminatively. We formulate the robust place recognition problem as a convex optimization problem with structured sparsity regularization due to the fact that only a small set of template places can match the query place. In addition, we also develop a new algorithm to solve the formulated optimization problem efficiently, which guarantees to converge to the global optima theoretically. Our ROMS method is evaluated through extensive experiments on three large-scale benchmark datasets, which record scenes ranging from different times of the day, months, and seasons. Experimental results demonstrate that our ROMS method outperforms the existing loop closure detection methods in long-term SLAM, and achieves the state-of-the-art performance.

Global image signature for visual loop-closure detection

This work details a new method for loop-closure detection based on using multiple orthogonal projections to generate a global signature for each image of a video sequence. The new multi-projection function permits the detection of images corresponding to the same scene, but taken from different points of view. The signature generation process preserves enough information for robust loop-closure detection, although it transforms each image to a simple and compact representation. Thanks to these characteristics, a real-time operation is possible, even for long sequences with thousands of images. In addition, it has proved to work on very different scenarios without the need to change the parameters or to perform an onffline training stage, which makes it very independent on the environment and camera configuration. Results of an extensive set of experiments of the algorithm on several datasets, both indoors and outdoors and including underwater scenarios, are presented. Furthermore, an implementation, named HALOC, is available at a public repository as a C++ library for its use under the BSD license.