Robust Visual Place Recognition Research Articles

Forest: A Lightweight Semantic Image Descriptor for Robust Visual Place Recognition

Visual place recognition (VPR) is the process of identifying previously visited places using visual information. It is crucial for a robot to achieve fast and accurate VPR under appearance and viewpoint changes. Inspired by human perception intuition, this letter proposes a lightweight semantic image descriptor called Forest for robust VPR. Specifically, for each semantic object in the image, three most recognizable reference objects are first found and ranked in descending order according to a customized measure. And then, categories, position and area of the semantic object itself, along with the categories of the three recognized reference objects, are used to encode the local descriptor of the object. As a result, the set of local descriptors is used as the global descriptor of the image. Performance of the proposed descriptor is evaluated and compared with six state-of-the-art VPR methods, Ibow-LCD, CoHOG, NetVLAD, LOST-X, Region VLAD, Patch-NetVLAD, over four public datasets of urban scenes, Extended-CMU Season, RobotCar Seasons v2, SYNTHIA and Kitti, using three Precision-Recall based metrics, AUC(Area Under Curve), Recall@100%Precision and Precision@100%Recall. The results show that our method maintains competitive accuracy in scenes with appearance and viewpoint changes and exhibits high robustness. At the same time, our method is robust to the noise of semantic segmentation and exhibits high real-time performance.

Unsupervised Appearance Map Abstraction for Indoor Visual Place Recognition With Mobile Robots

Visual Place Recognition (VPR), the task of identifying the place where an image has been taken from, is at the core of important robotic problems as relocalization, loop-closure detection or topological navigation. Even for indoors, the focus of this work, VPR is challenging for a number of reasons, including real-time performance when dealing with large image databases ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim10^{4}$</tex-math></inline-formula> ) (probably captured by different robots), or the avoidance of Perceptual Aliasing in environments with repetitive structures and scenes. In this letter, we tackle these issues by proposing an off-line mapping technique that abstracts a dense database of georeferenced images without particular order into a Multivariate Gaussian Mixture Model, by creating soft clusters in terms of their similarity in both pose and appearance. This abstract representation is obtained through an Expectation-Maximization algorithm and plays the role of a simplified map. Since querying this map yields a probability of being in a cluster, we exploit this “belief” within a Bayesian filter that regards previous query images and a topological map between clusters to perform more robust VPR. We evaluate our proposal in two different indoor datasets, demonstrating comparable VPR precision to querying the full database while incurring in shorter query times and handling Perceptual Aliasing for sequential navigation.

COLCONF: Collaborative ConvNet Features-based Robust Visual Place Recognition for Varying Environments

Several deep learning features were recently proposed for visual place recognition (VPR) purpose. Some of them use the information laid in the image sequences, while others utilize the regions of interest (ROIs) that reside in the feature maps produced by the CNN models. It was shown in the literature that features produced from a single layer cannot meet multiple visual challenges. In this work, we present a new collaborative VPR approach, taking the advantage of ROIs feature maps gathered and combined from two different layers in order to improve the recognition performance. An extensive analysis is made on extracting ROIs and the way the performance can differ from one layer to another. Our approach was evaluated over several benchmark datasets including those with viewpoint and appearance challenges. Results have confirmed the robustness of the proposed method compared to the state-of-the-art methods. The area under curve (AUC) and the mean average precision (mAP) measures achieve an average of 91% in comparison with 86% for Max Flow and 72% for CAMAL.

Robust Visual Place Recognition in Changing Environments Using Improved DTW

Recently, the methods based on Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in visual place recognition. CNN is a class of multilayer perceptrons, but unlike common multilayer perceptrons that it is not usually fully connected networks. It can acquire more general image features and make the image processing computationally manageable through filtering the connections by proximity. In this paper, we utilize the deep features generated by CNNs and the dynamic time warping (DTW) algorithm for image sequence place recognition. We propose a novel image similarity measurement, which is derived from cosine distance and can better distinguish match and mismatch. Meanwhile, we improve the DTW algorithm to design a local matching method that can reduce time complexity from O(n3) to O(n). To test the proposed method, four datasets (Nordland, Gardens Point, St. Lucia, and UoA datasets) are used as benchmarks; using two traverses in each dataset with one for reference and the other for testing. The results show high precision-recall characteristics of our method in the cases of severe appearance changes. Besides, our method achieves substantial improvements over the methods using the deep feature representations of a single image for recognition, which reflects that the spatiotemporal information contained in the image sequence is significant for the task of visual place recognition. Moreover, the proposed method also shows to outperform the classical sequence-based method SeqSLAM.

Local and Deep Features for Robust Visual Indoor Place Recognition

This study focuses on the area of visual indoor place recognition (e.g., in an office setting, automatically recognizing different places, such as offices, corridor, wash room, etc.). The potential applications include robot navigation, augmented reality, and image retrieval. However, the task is extremely demanding because of the variations in appearance in such dynamic setups (e.g., view-point, occlusion, illumination, scale, etc.). Recently, Convolutional Neural Network (CNN) has emerged as a powerful learning mechanism, able to learn deep higher-level features when provided with a comparatively big quantity of labeled training data. Here, we exploit the generic nature of CNN features for robust visual place recognition in the challenging COLD dataset. So, we employ the pre-trained CNNs (on the related tasks of object and scene classification) for deep feature extraction in the COLD images. We demonstrate that these off-the-shelf features, when combined with a simple linear SVM classifier, outperform their bag-of-features counterpart. Moreover, a simple combination scheme, combining the local bag-of-features and higher-level deep CNN features, produce outstanding results on the COLD dataset.

Learning to Fuse Multiscale Features for Visual Place Recognition

Efficient and robust visual place recognition is of great importance to autonomous mobile robots. Recent work has shown that features learned from convolutional neural networks achieve impressed performance with efficient feature size, where most of them are pooled or aggregated from a convolutional feature map. However, convolutional filters only capture the appearance of their perceptive fields, which lack the considerations on how to combine the multiscale appearance for place recognition. In this paper, we propose a novel method to build a multiscale feature pyramid and present two approaches to use the pyramid to augment the place recognition capability. The first approach fuses the pyramid to obtain a new feature map, which has an awareness of both the local and semi-global appearance, and the second approach learns an attention model from the feature pyramid to weight the spatial grids on the original feature map. Both approaches combine the multiscale features in the pyramid to suppress the confusing local features while tackling the problem in two different ways. Extensive experiments have been conducted on benchmark datasets with varying degrees of appearance and viewpoint variations. The results show that the proposed approaches achieve superior performance over the networks without the multiscale feature fusion and the multiscale attention components. Analyses on the performance of using different feature pyramids are also provided.

Place Recognition: An Overview of Vision Perspective

Place recognition is one of the most fundamental topics in the computer-vision and robotics communities, where the task is to accurately and efficiently recognize the location of a given query image. Despite years of knowledge accumulated in this field, place recognition still remains an open problem due to the various ways in which the appearance of real-world places may differ. This paper presents an overview of the place-recognition literature. Since condition-invariant and viewpoint-invariant features are essential factors to long-term robust visual place-recognition systems, we start with traditional image-description methodology developed in the past, which exploits techniques from the image-retrieval field. Recently, the rapid advances of related fields, such as object detection and image classification, have inspired a new technique to improve visual place-recognition systems, that is, convolutional neural networks (CNNs). Thus, we then introduce the recent progress of visual place-recognition systems based on CNNs to automatically learn better image representations for places. Finally, we close with discussions and mention of future work on place recognition.

BoCNF: efficient image matching with Bag of ConvNet features for scalable and robust visual place recognition

Recent advances in visual place recognition (VPR) have exploited ConvNet features to improve the recognition accuracy under significant environmental and viewpoint changes. However, it remains unsolved how to implement efficient image matching with high dimensional ConvNet features. In this paper, we tackle the problem of matching efficiency using ConvNet features for VPR, where the task is to accurately and quickly recognize a given place in large-scale challenging environments. The paper makes two contributions. First, we propose an efficient solution to VPR, based on the well-known bag-of-words (BoW) framework, to speed up image matching with ConvNet features. Second, in order to alleviate the problem of perceptual aliasing in BoW, we adopt a coarse-to-fine approach where we first, in the coarse stage, search for the top-K candidate images via BoW and then, in the fine stage, identify the best match among the candidates using a hash-based voting scheme. We conduct extensive experiments on six popular VPR datasets to validate the effectiveness of our method. Experimental results show that, in terms of recognition accuracy, our method is comparable to linear search, and outperforms other methods such as FABMAP and SeqSLAM by a significant margin. In terms of efficiecy, our method achieves a significant speed-up over linear search, with an average matching time as low as 23.5 ms per query on a dataset with 21K images.