DCF-BoW: Build Match Graph Using Bag of Deep Convolutional Features for Structure From Motion

Abstract

Matching a large number of images is quite time-consuming for structure from motion (SfM) due to the image matching by comparing features between all image pairs. In this letter, a bag of deep convolutional features (DCF-BoW) model is proposed to create match graph to reduce the number of matches. First, the convolutional feature map of an image is extracted using the VGG-16 convolutional neural network trained on ImageNet. Then, each local region in the original image can be represented by a feature vector in the feature map. The feature vectors are normalized and used to construct a bag of words model, which could convert each image into a DCF-BoW representation. Finally, the match graph is constructed by selecting the top 10 images with the highest similarities, which are calculated by computing the distances between those DCF-BoW representations. The experiment results show that the proposed DCF-BoW can create the match graph effectively in short time and find the potential overlapping image pairs. The match graph created by the proposed DCF-BoW is better than those built by DBoW3 and VocabTree, which is clearly showed in precision-recall curve on the Urban data set. The results of the SfM reconstruction based on the match graph created by the proposed DCF-BoW are slightly worse than those of the exhaustive matching, while the number of matches is reduced by 97.4% and 92.1%, respectively, on the Urban and South Building data sets.