Beyond Holistic Descriptors, Keypoints, and Fixed Patches: Multiscale Superpixel Grids for Place Recognition in Changing Environments

Abstract

Vision-based place recognition in environments subject to severe appearance changes due to day–night cycles, changing weather or seasons is a challenging task. Existing methods typically exploit image sequences, holistic descriptors and/or training data. Each of these approaches limits the practical applicability, e.g., to constant viewpoints for usage of holistic image descriptors. Recently, the combination of local region detectors and descriptors based on convolutional neural networks showed to be a promising approach to overcome these limitations. However, established region detectors, e.g., keypoint detectors, showed severe problems to provide repetitive landmarks despite dramatically changed appearance of the environment. Thus, they are typically replaced by holistic image descriptors or fixedly arranged patches—both known to be sensitive towards viewpoint changes. In this letter, we present a novel local region detector, SP-Grid, that is particularly suited for the combination of severe appearance and viewpoint changes. It is based on multiscale image oversegmentations and is designed to combine the advantages of keypoints and fixed image patches by starting from an initial grid-like arrangement and subsequently adapting to the image content. The grid-like arrangement showed to be beneficial in the presence of severe appearance changes and the adaptation to the image content increases the robustness toward viewpoint changes. The experimental evaluation will show the benefit compared to existing local region detectors and holistic image descriptors.