NBR-Net: A Nonrigid Bidirectional Registration Network for Multitemporal Remote Sensing Images

Abstract

Remote sensing image registration is the basis of change detection, environmental monitoring, and image fusion. Under severe appearance differences, feature-based methods have difficulty in finding sufficient feature matches to solve the global transformation and tackling the local deformation caused by height undulations and building shadows. By contrast, nonrigid registration methods are more flexible than feature-based matching methods, while often ignoring the reversibility between images, resulting in misalignment and inconsistency. To this end, this article proposes a nonrigid bidirectional registration network (NBR-Net) to estimate the flow-based dense correspondence for remote sensing images. We first propose an external cyclic registration network to strengthen the registration reversibility and geometric consistency by registering Image A to Image B and then reversely registering back to Image A. Second, we design an internal iterative refinement strategy to optimize the rough predicted flow caused by large distortion and viewpoint difference. Extensive experiments demonstrate that our method shows a performance superior to the state-of-the-art models on the multitemporal satellite image dataset. Furthermore, we attempt to extend our method to heterogeneous remote sensing image registration, which is more common in the real world. Therefore, we test our pretrained model in a satellite and unmanned aerial vehicle (UAV) image registration task. Due to the cyclic registration mechanism and coarse-to-fine refinement strategy, the proposed approach obtains the best performance on two GPS-denied UAV image datasets. Our code will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/xuyingxiao/</uri> NBR-Net.