Towards a Fair Evaluation of Feature Extraction Algorithms Robustness in Structure from Motion

Abstract

Structure from Motion is a pipeline for 3D reconstruction in which the true geometry of an object or a scene is inferred from a sequence of 2D images. As feature extraction is usually the first phase in the pipeline, the reconstruction quality depends on the accuracy of the feature extraction algorithm. Fairly evaluating the robustness of feature extraction algorithms in the absence of reconstruction ground truth is challenging due to the considerable number of parameters that affect the algorithms' sensitivity and the tradeoff between reconstruction size and error. The evaluation methodology proposed in this paper is based on two elements. The first is using constrained 3D reconstruction, in which only fixed numbers of extracted and matched features are passed to subsequent phases. The second is comparing the 3D reconstructions using size-error curves (introduced in this paper) rather than the value of reconstruction size, error, or both. The experimental results show that the proposed methodology is more transparent.