MCNet: Multiscale Clustering Network for Two-View Geometry Learning and Feature Matching

Abstract

Dear Editor, The main components of multi-view geometry and computer vision are robust pose estimation and feature matching. This letter discusses how to recover two-view geometry and match features between a pair of images, and presents MCNet (a multiscale clustering network) as an algorithm for extracting multiscale features. It can identify the true inliers from the established putative correspondences, where outliers may degenerate the geometry estimation. In particular, the proposed MCNet is based on graph clustering, in which the embedded correspondence features are mapped to a number of clusters by graph pooling. We designed a multiscale clustering layer into the two-view correspondence learning framework in order to improve correspondence representation efficiency. As a consequence of the multi-group feature fusion, we also constructed the network architectures termed MCNet-U and MCNet-M, respectively, utilizing the UNet and Pyramid techniques. Based on experimental results, the proposed model achieves state-of-the-art performance on feature matching with heavy outliers under weak supervision.