Improving sparse graph attention for feature matching by informative keypoints exploration

Abstract

Feature matching refers to building correct point correspondences from two-view images. Typical methods create coarse matches merely based on the similarity of local descriptors, then additionally use geometrical constraints or coherence to get a clean match set. However, this indirect paradigm would be strictly limited by the pre-matching results. This can be alleviated by a GNN-based matching network that constructs correct correspondences directly from feature positions and descriptions. Attentional pooling is utilized to reduce the high computational complexity and memory usage by sparsing the graph attention. In this paper, we propose an enhanced sparse GNN, namely KeyGNN, for fast feature matching, by designing Guided Attentional Pooling to emphasize the informative cues in GNN layers. Specifically, we propose Informative Keypoints Exploration Module and Guided Sparse Module to respectively extract information-rich points and use them to guide the attentional pooling for better information preservation. This process is constrained by the proposed Structural Importance Ranking Loss and Descriptor Distribution Loss, that ensure the structural information can be correctly extracted and the generated feature vectors be well distributed in the deep space. Unlike existing techniques, our method can focus and preserve more on the main information in the sparse GNN stage, thus avoiding information loss and enhancing the accuracy. Extensive experiments on large and public datasets reveal that our method can obtain better performance than the state-of-the-arts in the tasks of camera pose estimation, fundamental matrix estimation, and visual localization. The runtime and memory test shows that our network takes low computational and memory usage, which is friendly to real-time vision tasks.