Appearance-based Loop Closure Detection Research Articles

LSGDDN-LCD: An appearance-based loop closure detection using local superpixel grid descriptors and incremental dynamic nodes

Loop Closure Detection (LCD) is an essential component of visual Simultaneous Localization and Mapping (SLAM) systems. It enables the recognition of previously visited scenes to eliminate pose and map estimate drifts arising from long-term exploration. However, current appearance-based LCD methods face significant challenges, including high computational costs, viewpoint changes, and dynamic objects in scenes. This paper introduced an online appearance based LCD using Local Superpixel Grids Descriptor (LSGD) and Dynamic Nodes (DN), i.e., LSGDDN-LCD, to find similarities between scenes via handcrafted features extracted from the LSGD. Additionally, we proposed the adaptive mechanism to group similar scenes called DynamicNodes, which incrementally adjusted the database in an online manner, allowing for efficient and online retrieval of previously viewed images without need of the pre-training. Experimental results confirmed that the LSGDDN-LCD significantly improved LCD precision–recall and efficiency, and outperformed several state-of-the-art (SOTA) approaches on public and our own datasets, indicating its great potential as a generic LCD framework. Our implementation of the LSGDDN-LCD approach and the datasets were open-sourced on GitHub (https://github.com/BaoshengZhang0/LSGDDN-LCD.git).

Fast loop closure detection using probabilistic integration of pose and appearance similarity

Loop closure detection is a key technique for robots to minimize the accumulated localization and mapping errors after long-time explorations of simultaneous localization and mapping. However, the requirement for efficiency and accuracy performance for mobile robot applications is not well satisfied. In this article, we propose a fast and accurate loop closure detection method by exploiting both pose-based and appearance-based information in a probabilistic manner, inspired by the complementarity between the pose-based and the appearance-based information. Our approach formulates a probability framework combing the pose-based loop closure detection probability and the appearance-based loop closure detection probability. In the proposed framework, the pose-based loop closure detection model is firstly derived from the nonlinear optimization model of odometry. Then the appearance-based loop similarity and the pose-based loop similarity are combined into a joint framework to improve the loop closure detection performance. We implemented our approach using C++ and ROS and thoroughly tested it on the publicly available datasets. The experiments presented in this article suggest that the proposed method can achieve high efficiency and accuracy performance on loop closure detection.

An Efficient Online Loop Closure Detection System With Local Spatial Co-Occurrence Information

As a vital component in simultaneous localization and mapping techniques, appearance-based loop closure detection (LCD) plays important roles in bounding the long-term drift errors. In this paper, an online LCD system based on the mutual co-occurrence information among visual features is proposed. First, a feature tracker module is designed to generate distinctive visual words, exploiting tracked words tool to improve efficiency. Then an incrementally built vocabulary is organized by a hierarchical navigable small world graph, where the visual words are indexed. To merge a homologous word into the existing one, the vocabulary applies an improved high dimensional online clustering method, which regards individual cluster as a normal distribution form. At the query phase, a list of candidate frames is located due to the co-occurrence constraint. Ultimately, the loop closure is specified by passing the temporal and similarity check, which avoids the memory consumption of historic image data. Validation tests based on public datasets and experimental sequence demonstrate the merit of low running time and memory cost, while the high-precision performance is retained in the system.

Incremental Bag of Words with Gradient Orientation Histogram for Appearance-Based Loop Closure Detection

This paper proposes a novel approach for appearance-based loop closure detection using incremental Bag of Words (BoW) with gradient orientation histograms. The presented approach involves dividing and clustering image blocks into local region features and representing them using gradient orientation histograms. To improve the efficiency of the loop closure detection process, the vocabulary Clustering Feature (CF) tree is generated and updated in real time using the Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH) algorithm, which is combined with an inverted index for the efficient selection of candidates and calculation of similarity. Moreover, temporally close and highly similar images are grouped to generate islands, which enhances the accuracy and efficiency of the loop closure detection process. The proposed approach is evaluated on publicly available datasets, and the results demonstrate high recall and precision.

Appearance-Based Loop Closure Detection via Locality-Driven Accurate Motion Field Learning

Loop closure detection (LCD) is of significant importance in simultaneous localization and mapping. It represents the robot&#x2019;s ability to recognize whether the current surrounding corresponds to a previously observed one. In this paper, we conduct this task in a two-step strategy: candidate frame selection and loop closure verification. The first step aims to search semantically similar images for the query one using features obtained by Key.Net with HardNet. Instead of adopting the traditional Bag-of-Words strategy, we utilize the aggregated selective match kernel to calculate the similarity between images. Subsequently, based on the potential property of motion field in the LCD scene, we propose a novel feature matching method, <i>i.e.</i>, exploiting the smoothness prior and learning the motion field for an image pair in a reproducing kernel Hilbert space (RKHS), to implement loop closure verification. Concretely, we formulate the learning problem into a Bayesian framework with latent variables indicating the true/false correspondences and a mixture model accounting for the distribution of data. Furthermore, we propose a locality-driven mechanism to enhance the local relevance of motion vectors and term the algorithm as locality-driven accurate motion field learning (LAL). To satisfy the requirement of efficiency in the LCD task, we use a sparse approximation and search a suboptimal solution for the motion field in the RKHS, termed as LAL&#x002A;. Extensive experiments are conducted on public datasets for feature matching and LCD tasks. The quantitative results demonstrate the effectiveness of our method over the current state-of-the-art, meanwhile showing its potential for long-term visual localization. The codes of LAL and LAL&#x002A; are publicly available at <uri>https://github.com/KN-Zhang/LAL</uri>.

Appearance-based loop closure detection combining lines and learned points for low-textured environments

Hand-crafted point descriptors have been traditionally used for visual loop closure detection. However, in low-textured environments, it is usually difficult to find enough point features and, hence, the performance of such algorithms degrade. Under this context, this paper proposes a loop closure detection method that combines lines and learned points to work, particularly, in scenarios where hand-crafted points fail. To index previous images, we adopt separate incremental binary Bag-of-Words (BoW) schemes for points and lines. Moreover, we adopt a binarization procedure for features’ descriptors to benefit from the advantages of learned features into a binary BoW model. Furthermore, image candidates from each BoW instance are merged using a novel query-adaptive late fusion approach. Finally, a spatial verification stage, which integrates appearance and geometry perspectives, allows us to enhance the global performance of the method. Our approach is validated using several public datasets, outperforming other state-of-the-art solutions in most cases, especially in low-textured scenarios.

Modest-vocabulary loop-closure detection with incremental bag of tracked words

A key feature in the context of simultaneous localization and mapping is loop-closure detection, a process determining whether the current robot’s environment perception coincides with previous observation. However, in long-term operations, both computational efficiency and memory requirements involved in an autonomous robot operation in uncontrolled environments, are of particular importance. The majority of approaches scale linearly with the environment’s size in terms of storage and query time. The article at hand presents an efficient appearance-based loop-closure detection pipeline, which encodes the traversed trajectory by a low amount of unique visual words generated on-line through feature tracking. The incrementally constructed visual vocabulary is referred to as the “Bag of Tracked Words.” A nearest-neighbor voting scheme is utilized to query the database and assign probabilistic scores to all visited locations. Exploiting the inherent temporal coherency in the loop-closure task, the produced scores are processed through a Bayesian filter to estimate the belief state about the robot’s location on the map. Also, a geometrical verification step ensures consistency between image matches. Management is also applied to the resulting vocabulary to reduce its growth rate and constraint the system’s computational complexity while improving its voting distinctiveness. The proposed approach’s performance is experimentally evaluated on several publicly available and challenging datasets, including hand-held, car-mounted, aerial, and ground trajectories. Results demonstrate the method’s adaptability, which retains high operational frequency in environments of up to 13 km and high recall rates for perfect precision, outperforming other state-of-the-art techniques. The system’s effectiveness is owed to the reduced vocabulary size, which is at least one order of magnitude smaller than other contemporary approaches. An open research-oriented source code has been made publicly available, which is dubbed as “BoTW-LCD.”

Illumination Robust Loop Closure Detection with the Constraint of Pose

Background: Loop closure detection is a crucial part in robot navigation and simultaneous location and mapping (SLAM). Appearance-based loop closure detection still faces many challenges, such as illumination changes, perceptual aliasing and increasing computational complexity. Methods: In this paper, we proposed a visual loop closure detection algorithm that combines illumination robust descriptor DIRD and odometry information. In this algorithm, a new distance function is built by fusing the Euclidean distance function and Mahalanobis distance function, which integrates the pose uncertainty of body and can dynamically adjust the threshold of potential loop closure locations. Then, potential locations are verified by calculating the similarity of DIRD descriptors. Results: The proposed algorithm is evaluated on KITTI and EuRoC datasets, and is compared with SeqSLAM algorithm, which is one of the state of the art loop closure detection algorithms. The results show that the proposed algorithm could effectively reduce the computing time and get better performance on P-R curve. Conclusion: The new loop closure detection method makes full use of odometry information and image appearance information. The application of the new distance function can effectively reduce the missed detection caused by odometry error accumulation. The algorithm does not require extracting image features or learning stage, and can realize real-time detection and run on the platform with limited computational power.

Learning second-order statistics for place recognition based on robust covariance estimation of CNN features

Appearance based loop closure detection plays an important role in visual simultaneous localization and mapping systems (vSLAM) by measuring similarity of the places and checking loops to reduce the accumulated error. Traditional loop closure methods execute place recognition by image retrieval with Bag-of-Word model, which forms an orderless representation of local feature descriptors. Convolutional neural networks (CNNs) based features have been investigated for place recognition, where the final descriptors usually are generated by first-order pooling, limiting the representation ability in challenging scenarios. To handle above issue, we introduce high-order statistics into place recognition by developing a novel adaptively normalized covariance pooling method for learning place representations in an end-to-end manner. The proposed method provides robust covariance matrix estimation of high-dimensional and small-size deep features by adaptive covariance normalization (AdaCN). Experimental results on place recognition in the urban environment and image retrieval tasks show that second-order representation is effective, especially for discriminating places with confusing objects, changes in viewpoint and illumination. Besides, the proposed adaptive normalization performs favorably against its counterparts based on Log-Euclidean Riemannian metric and Power-Euclidean metric, while our method is superior to the state-of-the-art place recognition approaches.

IBoW-LCD: An Appearance-Based Loop-Closure Detection Approach Using Incremental Bags of Binary Words

In this paper, we introduce iBoW-LCD, a novel appearance-based loop closure detection method. The presented approach makes use of an incremental Bag-of-Words (BoW) scheme based on binary descriptors to retrieve previously seen similar images, avoiding any vocabulary training stage usually required by classic BoW models. In addition, to detect loop closures, iBoW-LCD builds on the concept of dynamic islands, a simple but effective mechanism to group similar images close in time, which reduces the computational times typically associated to Bayesian frameworks. Our approach is validated using several indoor and outdoor public datasets, taken under different environmental conditions, achieving a high accuracy and outperforming other state-of-the-art solutions.

Affine-Invariant Geometric Constraints-Based High Accuracy Simultaneous Localization and Mapping

In this study we describe a new appearance-based loop-closure detection method for online incremental simultaneous localization and mapping (SLAM) using affine-invariant-based geometric constraints. Unlike other pure bag-of-words-based approaches, our proposed method uses geometric constraints as a supplement to improve accuracy. By establishing an affine-invariant hypothesis, the proposed method excludes incorrect visual words and calculates the dispersion of correctly matched visual words to improve the accuracy of the likelihood calculation. In addition, camera’s intrinsic parameters and distortion coefficients are adequate for this method. 3D measuring is not necessary. We use the mechanism of Long-Term Memory and Working Memory (WM) to manage the memory. Only a limited size of the WM is used for loop-closure detection; therefore the proposed method is suitable for large-scale real-time SLAM. We tested our method using the CityCenter and Lip6Indoor datasets. Our proposed method results can effectively correct the typical false-positive localization of previous methods, thus gaining better recall ratios and better precision.

Appearance-Based Loop Closure Detection for Online Large-Scale and Long-Term Operation

In appearance-based localization and mapping, loop-closure detection is the process used to determinate if the current observation comes from a previously visited location or a new one. As the size of the internal map increases, so does the time required to compare new observations with all stored locations, eventually limiting online processing. This paper presents an online loop-closure detection approach for large-scale and long-term operation. The approach is based on a memory management method, which limits the number of locations used for loop-closure detection so that the computation time remains under real-time constraints. The idea consists of keeping the most recent and frequently observed locations in a working memory (WM) that is used for loop-closure detection, and transferring the others into a long-term memory (LTM). When a match is found between the current location and one stored in WM, associated locations that are stored in LTM can be updated and remembered for additional loop-closure detections. Results demonstrate the approach's adaptability and scalability using ten standard datasets from other appearance-based loop-closure approaches, one custom dataset using real images taken over a 2-km loop of our university campus, and one custom dataset (7 h) using virtual images from the racing video game “Need for Speed: Most Wanted”.

PIRF-Nav 2.0: Fast and online incremental appearance-based loop-closure detection in an indoor environment

This paper presents a fast and online incremental solution for an appearance-based loop-closure detection problem in a dynamic indoor environment. Closing the loop in a dynamic environment has been an important topic in robotics for decades. Recently, PIRF-Nav has been reported as being successful in achieving high recall rate at precision 1. However, PIRF-Nav has three main disadvantages: (i) the computational expense of PIRF-Nav is beyond real-time, (ii) it utilizes a large amount of memory in the redundant process of keeping signatures of places, and (iii) it is ill-suited to an indoor environment. These factors hinder the use of PIRF-Nav in a general environment for long-term, high-speed mobile robotic applications. Therefore, this paper proposes two techniques: (i) new modified PIRF extraction that makes the system more suitable for an indoor environment and (ii) new dictionary management that can eliminate redundant searching and conserve memory consumption. The results show that our proposed method can complete tasks up to 12 times faster than PIRF-Nav with only a slight percentage decline in recall. In addition, we collected additional data from a university canteen crowded during lunch time. Even in this crowded indoor environment, our proposed method has better real-time processing performance compared with other methods.

Combining Odometry and Visual Loop-Closure Detection for Consistent Topo-Metrical Mapping

We address the problem of simultaneous localization and mapping (SLAM) by combining visual loop-closure detection with metrical information given by a robot odometry. The proposed algorithm extends a purely appearance-based loop-closure detection method based on bags of visual words [A. Angeli, D. Filliat, S. Doncieux and J.-A. Meyer, IEEE Transactions On Robotics, Special Issue on Visual SLAM 24 (2008) 1027–1037], which is able to detect when the robot has returned back to a previously visited place. An efficient optimization algorithm is used to integrate odometry information and to generate a consistent topo-metrical map much more usable for global localization and path planning. The resulting algorithm which only requires a monocular camera and robot odometry data, is real-time, incremental (i.e. it does not require any a priori information on the environment), and can be easily embedded on medium platforms.

Vast-scale Outdoor Navigation Using Adaptive Relative Bundle Adjustment

In this paper we describe a relative approach to simultaneous localization and mapping, based on the insight that a continuous relative representation can make the problem tractable at large scales. First, it is well known that bundle adjustment is the optimal non-linear least-squares formulation for this problem, in that its maximum-likelihood form matches the definition of the Cramer—Rao lower bound. Unfortunately, computing the maximum-likelihood solution is often prohibitively expensive: this is especially true during loop closures, which often necessitate adjusting all parameters in a loop. In this paper we note that it is precisely the choice of a single privileged coordinate frame that makes bundle adjustment costly, and that this expense can be avoided by adopting a completely relative approach. We derive a new relative bundle adjustment which, instead of optimizing in a single Euclidean space, works in a metric space defined by a manifold. Using an adaptive optimization strategy, we show experimentally that it is possible to solve for the full maximum-likelihood solution incrementally in constant time, even at loop closure. Our approach is, by definition, everywhere locally Euclidean, and we show that the local Euclidean estimate matches that of traditional bundle adjustment. Our system operates online in realtime using stereo data, with fast appearance-based loop closure detection. We show results on over 850,000 images that indicate the accuracy and scalability of the approach, and process over 330 GB of image data into a relative map covering 142 km of Southern England. To demonstrate a baseline sufficiency for navigation, we show that it is possible to find shortest paths in the relative maps we build, in terms of both time and distance. Query images from the web of popular landmarks around London, such as the London Eye or Trafalgar Square, are matched to the relative map to provide route planning goals.