Remote Sensing Image Registration Research Articles

A Dual-Branch Network of Strip Convolution and Swin Transformer for Multimodal Remote Sensing Image Registration

Multimodal remote sensing image registration aims to achieve effective fusion and analysis of information by accurately aligning image data obtained by different sensors, thereby improving the accuracy and application value of remote sensing data in engineering. However, current advanced registration frameworks are unable to accurately register large-scale rigid distortions, such as rotation or scaling, that occur in multi-source remote sensing images. This paper presents a stable and high-precision end-to-end registration network that incorporates dual-branch feature extraction to address the stringent registration requirements encountered in practical engineering applications. The deep neural network consists of three parts: dual-branch feature extraction, affine parameter regression, and spatial transformation network. In the upper branch of the dual-branch feature extraction module, we designed a combination of multi-scale convolution and Swin Transformer to fully extract features of remote sensing images at different scales and levels to better understand the global structure and context information. In the lower branch, we incorporate strip convolution blocks to capture remote contextual information from various directions in multimodal images. Additionally, we introduce an efficient and lightweight ResNet module to enhance global features. At the same time, we developed a strategy to parallelize various convolution kernels in affine parameter regression networks, aiming to enhance the accuracy of transformation parameters and the robustness of the model. We conducted experiments on panchromatic–multispectral, infrared–optical, and SAR–optical image pairs with large-scale rigid transformations. The experimental results show that our method achieves the best registration effect.

DVF-NET: Bi-Temporal Remote Sensing Image Registration Network Based on Displacement Vector Field Fusion.

Accurate image registration is essential for various remote sensing applications, particularly in multi-temporal image analysis. This paper introduces DVF-NET, a novel deep learning-based framework for dual-temporal remote sensing image registration. DVF-NET integrates two displacement vector fields to address nonlinear distortions caused by significant variations between images, enabling more precise image alignment. A key innovation of this method is the incorporation of a Structural Attention Module (SAT), which enhances the model's ability to focus on structural features, improving the feature extraction process. Additionally, we propose a novel loss function design that combines multiple similarity metrics, ensuring more comprehensive supervision during training. Experimental results on various remote sensing datasets indicate that the proposed DVF-NET outperforms the existing methods in both accuracy and robustness, particularly when handling images with substantial geometric distortions such as tilted buildings. The results validate the effectiveness of our approach and highlight its potential for various remote sensing tasks, including change detection, land cover classification, and environmental monitoring. DVF-NET provides a promising direction for the advancement of remote sensing image registration techniques, offering both high precision and robustness in complex real-world scenarios.

A Line Feature-Based Rotation Invariant Method for Pre- and Post-Damage Remote Sensing Image Registration

The accurate registration of pre- and post-damage images plays a vital role in the change analysis of the target area and the subsequent work of damage effect assessment. However, due to the impact of shooting time and damaged areas, there are large background and regional differences between pre- and post-damage remote sensing images, and the existing image registration methods do not perform well. In this paper, a line feature-based rotation invariant image registration method is proposed for pre- and post-damage remote sensing images. First, we extract and screen straight line segments from the images before and after damage. Then, we design a new method to calculate the main direction of each line segment and rotate the image based on the current line segment’s main direction and the center coordinates. According to the spatial distribution (distance and angle) of the reference line segment relative to the remaining line segments, a line feature descriptor vector is constructed and matched for each line segment on the rotated image. Since the main edge contour can preserve more invariant features, this descriptor can be better applied to the registration of pre- and post-damage remote sensing images. Finally, we cross-pair the midpoints and endpoints of the matched line segments to improve the accuracy of subsequent affine transformation parameter calculations. In remote sensing images with large background and regional differences, the average registration precision of our method is close to 100%, and the root mean square error is about 1 pixel. At the same time, the rotation invariance of our method is verified by rotating the test images. In addition, the results of the comparative experiments show that the registration precision and error of the proposed method are better than those of the existing typical representative algorithms.

Registration of Remote Sensing Images by the Combination of Complex Nonlinear Diffusion and Phase Congruency Attributes

Registration of Remote Sensing Images by the Combination of Complex Nonlinear Diffusion and Phase Congruency Attributes

NCMNet: Neighbor Consistency Mining Network for Two-View Correspondence Pruning.

Correspondence pruning plays a crucial role in a variety of feature matching based tasks, which aims at identifying correct correspondences (inliers) from initial ones. Seeking consistent k-nearest neighbors in both coordinate and feature spaces is a prevalent strategy employed in previous approaches. However, the vicinity of an inlier contains numerous irregular false correspondences (outliers), which leads them to mistakenly become neighbors according to the similarity constraint of nearest neighbors. To tackle this issue, we propose a global-graph space to seek consistent neighbors with similar graph structures. This is achieved by using a global connected graph to explicitly render the affinity relationship between correspondences based on the spatial and feature consistency. Furthermore, to enhance the robustness of method for various matching scenes, we develop a neighbor consistency block to adequately leverage the potential of three types of neighbors. The consistency can be progressively mined by sequentially extracting intra-neighbor context and exploring inter-neighbor interactions. Ultimately, we present a Neighbor Consistency Mining Network (NCMNet) to estimate the parametric models and remove outliers. Extensive experimental results demonstrate that the proposed method outperforms other state-of-the-art methods on various benchmarks for two-view geometry estimation. Meanwhile, four extended tasks, including remote sensing image registration, point cloud registration, 3D reconstruction, and visual localization, are conducted to test the generalization ability.

A novel multimodal remote-sensing image registration algorithm using phase symmetry and rank-based local self-similarity

ABSTRACT To tackle the nonlinear radiation distortions (NRDs) in multimodal remote sensing images, this letter proposes a multimodal remote sensing image registration method that utilizes phase symmetry and rank-based local self-similarity (LSS). First, the local phase information of the images is utilized to construct a phase symmetry map, which is then used to detect features using the Features from Accelerated Segment Test (FAST) algorithm. Subsequently, a new feature descriptor that combines rank-based local self-similarity and phase congruency, is generated. Finally, the Fast Sample Consensus (FSC) method is employed to eliminate outliers. Experimental results on publicly available multimodal remote sensing image datasets demonstrate that the proposed method outperforms the state-of-the-art methods in terms of number of correct matches, precision, and root mean squared error (RMSE).

PRF-Net: A Progressive Remote Sensing Image Registration and Fusion Network.

Most of the existing fusion algorithms are not robust to unregistered input images. Even after image registration, nonlinear nonregistration may persist in the local areas of the images, leading to poor quality in the fused image. So, as to tackle these challenges, a progressive remote sensing image registration and fusion network is proposed in this article, and named PRF-Net, which is particularly useful when two images are from different platforms. First, a registration network is designed to register the input image patches, which includes a global spatial transform network (GSTN) and a local spatial warp network (LSWN). The GSTN is primarily used for coarse registration, applying rigid transformation to globally align the input images. After coarse registration, the preliminarily registered moving image is input into the LSWN for local fine-tuning to maximize correlation between the input image patches. Subsequently, the fine registered images are degraded and input into the fusion network to generate the fused image. To maintain sufficient spectral and spatial information of the fused image, a multiscale feature extraction (MSFE) block with a highly interpretable spatial details attention (SDA) block is designed, which can enhance the ability of fusion network to extract and preserve spatial details and spectral information. Three groups of experiments conducted on four types of remote sensing images give evidence of that the proposed PRF-Net exhibits excellent performance in both reduced and full resolutions, showcasing its outstanding registration and fusion quality.

Real-Time Registration of Unmanned Aerial Vehicle Hyperspectral Remote Sensing Images Using an Acousto-Optic Tunable Filter Spectrometer

Differences in field of view may occur during unmanned aerial remote sensing imaging applications with acousto-optic tunable filter (AOTF) spectral imagers using zoom lenses. These differences may stem from image size deformation caused by the zoom lens, image drift caused by AOTF wavelength switching, and drone platform jitter. However, they can be addressed using hyperspectral image registration. This article proposes a new coarse-to-fine remote sensing image registration framework based on feature and optical flow theory, comparing its performance with that of existing registration algorithms using the same dataset. The proposed method increases the structure similarity index by 5.2 times, reduces the root mean square error by 3.1 times, and increases the mutual information by 1.9 times. To meet the real-time processing requirements of the AOTF spectrometer in remote sensing, a development environment using VS2023+CUDA+OPENCV was established to improve the demons registration algorithm. The registration algorithm for the central processing unit+graphics processing unit (CPU+GPU) achieved an acceleration ratio of ~30 times compared to that of a CPU alone. Finally, the real-time registration effect of spectral data during flight was verified. The proposed method demonstrates that AOTF hyperspectral imagers can be used in real-time remote sensing applications on unmanned aerial vehicles.

Remote sensing image registration method based on synchronous atmospheric correction.

Image registration is a crucial preprocessing step in remote sensing applications, integrating information from multiple images to achieve synergistic advantages. Nevertheless, aerosols characterized by spatiotemporal heterogeneity can result in the blurring of remote-sensing images, thereby compromising the accuracy of image registration. This paper begins by analyzing the basic principles of atmospheric correction and image registration. The variations in atmospheric radiative contribution caused by aerosol changes in real-world scenarios were simulated, along with an examination of the relationship between atmospheric effects and the quantity of image features. Subsequently, addressing the challenge posed by insufficient synchronicity in aerosol parameters and the influence of atmospheric effects on remote sensing image registration, we propose a registration method based on synchronous atmospheric correction. This approach utilizes the Airborne Synchronous Monitoring Atmospheric Corrector (ASMAC) to obtain aerosol optical depth and column water vapor images for synchronous atmospheric correction of remote sensing images, along with the assessment of the registration transformation matrix. Finally, airborne experiments involving ASMAC and high-resolution cameras are conducted to validate the proposed method's improvement in remote sensing image registration accuracy. Experimental results demonstrate the effectiveness of the proposed method, showcasing an increase in the number of features and improvements in quantitative evaluation metrics. Specifically, the normalized correlation coefficient improved by up to 2.408%, while the normalized mutual information increased by a maximum of 1.395%, a maximum feature count and successfully matched features improvement of 21.1% and 38.5.

Automated Registration of Full Moon Remote Sensing Images Based on Triangulated Network Constraints

Abstract. The registration of full-moon remote sensing images constitutes a pivotal stage in the fusion analysis of multiple lunar remote sensing datasets. Addressing prevailing issues in automatic registration, such as the broad width of full-moon data, significant internal distortion, and texture distortion in high-latitude regions, this paper proposes a method for automatic matching and correction based on triangulation constraints. The approach employs a matching strategy progressing from coarse to fine and from sparse to dense. It optimizes and combines multiple existing matching algorithms, enhances the extraction of initial network points, constructs irregular triangulation networks using these points, conducts dense matching with each triangulation network as a basic unit, and introduces a geometric correction method based on triangulation network + grid (TIN + GRID) for the registration of full-moon data. For the matching of full-moon remote sensing images in high-latitude regions, a novel approach involving memory projection forward transformation-matching-projection inverse transformation is adopted. Through registration experiments with full-moon image data and an analysis of registration accuracy at different latitudes, the average mean square error is found to be less than 2 pixels. These results signify the efficacy of the proposed method in effectively addressing the automatic registration challenges encountered in full-moon remote sensing images.

PMA-Net: Progressive multi-stage adaptive feature learning for two-view correspondence

PMA-Net: Progressive multi-stage adaptive feature learning for two-view correspondence

Multimodal remote sensing image registration based on adaptive multi-scale PIIFD

Multimodal remote sensing image registration based on adaptive multi-scale PIIFD

Local feature matching using deep learning: A survey

Local feature matching using deep learning: A survey

Radiation-Variation Insensitive Coarse-to-Fine Image Registration for Infrared and Visible Remote Sensing Based on Zero-Shot Learning

Infrared and visible remote sensing image registration is significant for utilizing remote sensing images to obtain scene information. However, it is difficult to establish a large number of correct matches due to the difficulty in obtaining similarity metrics due to the presence of radiation variation between heterogeneous sensors, which is caused by different imaging principles. In addition, the existence of sparse textures in infrared images as well as in some scenes and the small number of relevant trainable datasets also hinder the development of this field. Therefore, we combined data-driven and knowledge-driven methods to propose a Radiation-variation Insensitive, Zero-shot learning-based Registration (RIZER). First, RIZER, as a whole, adopts a detector-free coarse-to-fine registration framework, and the data-driven methods use a Transformer based on zero-shot learning. Next, the knowledge-driven methods are embodied in the coarse-level matches, where we adopt the strategy of seeking reliability by introducing the HNSW algorithm and employing a priori knowledge of local geometric soft constraints. Then, we simulate the matching strategy of the human eye to transform the matching problem into a model-fitting problem and employ a multi-constrained incremental matching approach. Finally, after fine-level coordinate fine tuning, we propose an outlier culling algorithm that only requires very few iterations. Meanwhile, we propose a multi-scene infrared and visible remote sensing image registration dataset. After testing, RIZER achieved a correct matching rate of 99.55% with an RMSE of 1.36 and had an advantage in the number of correct matches, as well as a good generalization ability for other multimodal images, achieving the best results when compared to some traditional and state-of-the-art multimodal registration algorithms.

Feature matching based on Gaussian kernel convolution and minimum relative motion

Feature matching based on Gaussian kernel convolution and minimum relative motion

Hardware-Optimized Architecture of On-board Registration for Remote-Sensing Images ——Take SURF as an Example

Hardware-Optimized Architecture of On-board Registration for Remote-Sensing Images ——Take SURF as an Example

LM-Net: A Lightweight Matching Network for Remote Sensing Image Matching and Registration

LM-Net: A Lightweight Matching Network for Remote Sensing Image Matching and Registration

A Unified Deep Learning Network for Remote Sensing Image Registration and Change Detection

A Unified Deep Learning Network for Remote Sensing Image Registration and Change Detection

Aerial remote sensing image registration based on dense residual network of asymmetric convolution

Aerial remote sensing image registration based on dense residual network of asymmetric convolution

Aerial remote sensing image registration based on dense residual network of asymmetric convolution

Aerial remote sensing image registration based on dense residual network of asymmetric convolution