Matchable image retrieval for large-scale UAV images: an evaluation of SfM-based reconstruction

Abstract

ABSTRACT Structure from Motion (SfM) is a 3D reconstruction framework that has achieved great success on large-scale Unmanned Aerial Vehicle (UAV) images. Due to the high time consumption of feature matching, overlapped match pairs are obtained by image retrieval to improve efficiency. Bag of words (BoW) is commonly used in existing SfM systems. However, the large number of local features and the high dimension of BoW vectors cause image retrieval time-consuming. Recently, the lower dimension learned global features and more efficient feature aggregation methods provide solutions to the problem. Besides, efficient approximate nearest neighbour (ANN) searching can further accelerate image retrieval. Thus, this study conducts an evaluation of image retrieval methods for UAV images in SfM-based reconstruction. First, image retrieval methods with varying combinations of feature descriptors, aggregation strategies, and NN searching algorithms are reviewed and configured for performance evaluation. Second, the selected methods are evaluated in SfM-based reconstruction, in which the image retrieval results are fed into the workflow to guide feature matching and then exploited to create the weighted view graph to achieve parallel SfM reconstruction. Finally, comprehensive tests are conducted to evaluate the performance of selected methods by using three large-scale UAV datasets. The experimental results show that: (1) for feature aggregation and NN searching, Vector of Locally Aggregated Descriptors (VLAD) has superior performance compared with other strategies, and Hierarchical Navigable Small World (HNSW) has better achievement in NNS; (2) among evaluated feature descriptors, with the combination of VLAD and HNSW, the retrieval accuracy of SIFT is still higher than that of the learned local and global features. In a word, the optimal image retrieval method consists of SIFT, VLAD and HNSW, whose retrieval accuracy is higher than BoW by about 2% and efficiency is around 100 times that of BoW.