CE-Net: A Coordinate Embedding Network for Mismatching Removal

Abstract

Mismatching removal is at the core yet still a challenging problem in the photogrammetry and computer vision field. In this paper, we propose a coordinate embedding network (named CE-Net). We consider the mismatching problem as a graph node classification problem, and generate node descriptors by embedding point coordinates and aggregating geometric information from neighboring nodes based on self-attention and cross-attention mechanism. Finally, a binary classifier is used to separate node descriptors into two classes, namely matching inliers and outliers. Benefiting from the attention mechanism, firstly the node descriptors can get geometric information from “good neighbors” (i.e., matching inliers) and keep away from “bad neighbors” (i.e., matching outliers), improving the exactness of the descriptors; secondly the node descriptors can contain the information from both intra-graph and inter-graph, improving their distinctiveness. Experiments in testing datasets show that our proposed CE-Net achieves the state-of-the-art performance with a precision of 0.972, an outlier recall of 0.984, and an inlier recall of 0.963. Furthermore, CE-Net also outperforms the compared methods in real mismatching removal tasks in terms of positional accuracy, dispersion, and number of remaining point pairs, showing great potentials in practical applications. Our codes and data are available on https://github.com/csyhy1986 .