FEATURE MATCHING ENHANCEMENT USING THE GRAPH NEURAL NETWORK (GNN-RANSAC)

Abstract

Abstract. Improving the performance of feature matching plays a key role in computers vision and photogrammetry applications, such as fast image recognition, Structure from Motion (SFM), aerial triangulation, Visual Simultaneous Localization and Mapping (VSLAM), etc., where the RANSAC algorithm is frequently used for outlier detection; note that RANSAC is the most widely used robust approach in photogrammetry and computer vision for outlier detection. It is known that the outlier ratio used in RANSAC primarily determines the number of trial runs needed, which eventually, determines the computation time. Over time, different methods have been proposed to reject the false-positive correspondences and improve RANSAC, such as GR_RANSAC, SuperGlue, and LPRANSAC. The specific objective of this study is to propose a filtering algorithm based on Graph Neural Networks (GNN), as a pre-processing step before RANSAC, which can result in improvements for rejecting the outliers. The research is based on the idea that descriptors of corresponding points, as well as their spatial relationship, should be similar in image sequences. In graph representation, built by the adjacency matrix of data (nodes features), there should be similarity for corresponding points that are close to each other in the image domain. From the many GNNs techniques, Graph Attention Networks (GATs) were selected for this study as they assign different importance to each neighbour’s contribution as anisotropic operations, so the features of neighbour nodes are not considered in the same way, unlike other GNNs techniques. In our approach, we build a graph in each image, because the similarity of the two-dimensional spatial relationships between points in the image domain of consecutive images should be similar. Then during processing, points with any significantly different neighbours are considered as outliers. Next, the points can be updated in the GNN layer. GNN-RANSAC is tested experimentally on real image pairs. Clearly, the proposed pre-filtering increases the inlier ratio and results in faster convergence compared to ordinary RANSAC, making it attractive for real-time applications. Furthermore, there is no need to learn the features.