Automatic Area-Based Registration of Optical and SAR Images Through Generative Adversarial Networks and a Correlation-Type Metric

Abstract

The automatic registration of multisensor remote sensing images is a highly challenging task due to the inherently different physical, statistical, and textural properties of the input data. In the present paper, this problem is addressed in the case of optical-SAR images by proposing a novel method based on deep learning and area-based registration concepts. The method integrates a conditional generative adversarial network (cGAN), an area-based cross-correlation-type $\ell^{2}$ similarity metric, and the COBYLA constrained maximization algorithm. Whereas correlation-type metrics are typically ineffective in the application to multisensor registration, the proposed approach allows exploiting the image translation capabilities of cGAN architectures to enable the use of an $\ell^{2}$ similarity metric, which favors high computational efficiency. Experiments with Sentinel-1 and Sentinel-2 data suggest the effectiveness of this strategy and the capability of the proposed method to achieve accurate registration.