Metric Learning for Patch-Based 3-D Image Registration

Abstract

Patch-based image registration is a challenging problem in visual geometry, the crucial component of which is the selection of an appropriate similarity measure. The similarity measure participates in the objective calculation of the pose optimization, which determines the optimization convergence performance. In this paper, we propose learning a similarity metric of patches from reference and target images such that the pairwise patches with a small projection error receive high similarity scores. To achieve this objective, we designed and trained the classification, regression, and rank networks separately based on self-collected data sets. The network can directly output the projection error according to the patches, which is sensitive to the deviation of the pose transformation. We also designed evaluation criteria and validated the superior performance of the network’s outputs compared with the performance of traditional methods, such as the sum of absolute difference and the sum of squared differences. Note to Practitioners —Patch-based image registration is a basic field of study in visual geometry. The collected data sets, trained networks, and designed evaluation criteria in this paper can be used to conduct related research. In visual odometry or 3-D reconstruction, the pose optimization between frames is involved, and the network output can be used as an optimization objective. The patch size can be adjusted according to the speed and accuracy requirements.