Hierarchical Visual Place Recognition Based on Semantic-Aggregation

Xiaoting Song,Tao Lu,Baifan Chen,Hongyu Shen

doi:10.3390/app11209540

Abstract

A major challenge in place recognition is to be robust against viewpoint changes and appearance changes caused by self and environmental variations. Humans achieve this by recognizing objects and their relationships in the scene under different conditions. Inspired by this, we propose a hierarchical visual place recognition pipeline based on semantic-aggregation and scene understanding for the images. The pipeline contains coarse matching and fine matching. Semantic-aggregation happens in residual aggregation of visual information and semantic information in coarse matching, and semantic association of semantic edges in fine matching. Through the above two processes, we realized a robust coarse-to-fine pipeline of visual place recognition across viewpoint and condition variations. Experimental results on the benchmark datasets show that our method performs better than several state-of-the-art methods, improving the robustness against severe viewpoint changes and appearance changes while maintaining good matching-time performance. Moreover, we prove that it is possible for a computer to realize place recognition based on scene understanding.

Highlights

Visual place recognition (VPR) is a core task of localization [1,2,3] and loop closure detection [4,5] for mobile robots, which means that robots can accurately identify the same place according to the images under different conditions [6,7,8]
Such a coarse-to-fine hierarchical progress improves the accuracy of place recognition and matching helps to locate the query quickly, and fine matching helps to match the query accurately
The results show that the performance of hierarchical strategy with Candidates 10 and 15 is better than that of coarse matching only, and fine matching only on both two datasets

Summary

Introduction

Visual place recognition (VPR) is a core task of localization [1,2,3] and loop closure detection [4,5] for mobile robots, which means that robots can accurately identify the same place according to the images under different conditions [6,7,8]. Proposed to use the semantics-aware higher-order layers of deep neural networks for identifying specific places under 180 degrees viewpoint reversed They developed a descriptor normalization schemes to improve the robustness against appearance change. Aiming at the bucolic environments such as natural scenes with low texture and little semantic contents, but obvious appearance changes, Benbihi et al [31] proposed a global descriptor based on image topological and semantic information to achieve place recognition by matching semantic edges between two images. These works have shown that it is possible and efficient to apply image semantic information to VPR. We combine visual semantics and hierarchy and proof 14 pose hierarchical place recognition based on semantic aggregation, to minimize the influences of appearance and viewpoint variations

Hierarchical

Coarse

Semantic Edges Extraction and Description

Semantic

Datasets and Performance Evaluations

Performance Evaluations

Norland Dataset

Hierarchy or Single

Comparison with the State-of-Art Methods

Runtime

10. Even when

Conclusions