A Rotation-Invariant Optical and SAR Image Registration Algorithm Based on Deep and Gaussian Features

Zeyi Li,Yihang Huang,Haitao Zhang

doi:10.3390/rs13132628

Abstract

Traditional feature matching methods of optical and synthetic aperture radar (SAR) used gradient are sensitive to non-linear radiation distortions (NRD) and the rotation between two images. To address this problem, this study presents a novel approach to solving the rigid body rotation problem by a two-step process. The first step proposes a deep learning neural network named RotNET to predict the rotation relationship between two images. The second step uses a local feature descriptor based on the Gaussian pyramid named Gaussian pyramid features of oriented gradients (GPOG) to match two images. The RotNET uses a neural network to analyze the gradient histogram of the two images to derive the rotation relationship between optical and SAR images. Subsequently, GPOG is depicted a keypoint by using the histogram of Gaussian pyramid to make one-cell block structure which is simpler and more stable than HOG structure-based descriptors. Finally, this paper designs experiments to prove that the gradient histogram of the optical and SAR images can reflect the rotation relationship and the RotNET can correctly predict them. The similarity map test and the image registration results obtained on experiments show that GPOG descriptor is robust to SAR speckle noise and NRD.

Highlights

With the rapid development of remote sensor technology, multimodal, and multispectral sensing data are generated
In this paper, inspired by the structure of the Siamese network, we propose a novel neural network framework to predict the rotation relationship between
For training the RotNET, we constructed a dataset based on gradient histogram based on the SEN1-2 dataset

Summary

Introduction

With the rapid development of remote sensor technology, multimodal, and multispectral sensing data are generated. Optical images accord with human vision and are easy interpretation but not more susceptible to cloud and fog. Utilizing the complementary information of the optical and SAR images of the same object in the different environments and spectra, we could get important application values in image fusions [2], pattern recognition [3], and change detection [4], etc. The effects of these applications are dependent on the accuracy of the optical and SAR registration. Because of the serious speckle noise, non-linear radiation distortions (NRD) of SAR images and the large irradiance differences between optical and SAR images, optical, and SAR registration is still a challenging task [5,6]

Methods

Results

Conclusion