Learned Local Features for Structure From Motion of UAV Images: A Comparative Evaluation

Abstract

Unmanned aerial vehicle (UAV) images have become the main remote sensing data sources for varying applications, and structure from motion (SfM) is the golden standard for resuming camera poses. Matching local feature descriptors is the prerequisite for the accurate and complete orientation of UAV images. Recently, some newly proposed learned methods have been shown to outperform the hand-crafted methods, such as the scale invariant feature transform (SIFT) and its variants, and almost all learned methods have been trained and evaluated by using images from the internet with varying focal lengths and varying size. It is of interest to investigate the performance of these learned methods with their pretrained models for feature detection and description in the context of the SfM-based orientation. Thus, this article conducts a comprehensive evaluation of both advanced hand-crafted and newly proposed learned detectors and descriptors by using four UAV datasets. The performance of these selected methods is compared in the context of feature matching and the SfM and (multiview stereo) MVS-based reconstruction. Experimental results demonstrate that the learned descriptors combined with the SIFT-like detectors can provide accurate and complete feature correspondences and achieve better or competitive performance in the SfM and MVS-based reconstruction. For UAV image orientation, the learned descriptors can be an alternative to the existing hand-crafted descriptors without their model retraining. The source codes of this evaluation would be made publicly available.