TSSN-Net: Two-step Sparse Switchable Normalization for learning correspondences with heavy outliers

Abstract

In this paper, we solve the problem of feature matching by designing an end-to-end network (called TSSN-Net). Given putative correspondences of feature points in two views, existing deep learning based methods formulate the feature matching problem as a binary classification problem. In these methods, a normalizer plays an important role in the networks. However, they adopt the same normalizer in all normalization layers of the entire networks, which will result in suboptimal performance. To address this problem, we propose a Two-step Sparse Switchable Normalization Block, which involves the advantage of adaptive normalization for different convolution layers from Sparse Switchable Normalization and robust global context information from Context Normalization. Moreover, to capture local information of correspondences, we propose a Multi-Scale Correspondence Grouping algorithm, by defining a multi-scale neighborhood representation, to search for consistent neighbors of each correspondence. Finally, with a series of convolution layers, the end-to-end TSSN-Net is proposed to learn correspondences with heavy outliers for feature matching. Our experimental results have shown that our network achieves the state-of-the-art performance on benchmark datasets.