Triple Robustness Augmentation Local Features for multi-source image registration

Abstract

With the proliferation of a wide variety of sensors, accurate multi-source image registration is crucial for many remote sensing image processing tasks. However, the registration of multi-source images faces the challenges of rotations, scales, and domain transformations caused by significant differences in shooting time, viewing angle, and sensor imaging modes. To cope with this problem, we propose a deep learning-based registration method named TRFeat, which aims to comprehensively improve the rotation, scale, and cross-domain robustness of local features. First, we introduce a special circular sampling convolutional layer to replace the standard square convolutional layer, in order to enhance the rotational robustness of local features. Second, we design a scale pyramid backbone network architecture to improve the robustness of the network to scale transformations. Third, we promote the use of hypercolumn domain alignment loss to extract cross-domain robust local descriptors for images from different sources. In addition, we develop a novel keypoint detection training framework based on iterative refinement supervision to obtain repeatable and reliable keypoints localization in multi-source images. Finally, we conduct thorough experiments on five multi-source datasets. Extensive experimental results validate that our TRFeat outperforms other state-of-the-art hand-crafted (e.g. RIFT) and deep learning-based methods (e.g. ASLFeat). Specifically, our TRFeat achieves an MMA@3 of 76.08% on the HPatches dataset and an RMSE of 3.38 on the Xiang dataset. The code is available at https://github.com/vignywang/TRFeat.