Remote Sensing Image Registration Research Articles

NCMNet: Neighbor Consistency Mining Network for Two-View Correspondence Pruning.

Correspondence pruning plays a crucial role in a variety of feature matching based tasks, which aims at identifying correct correspondences (inliers) from initial ones. Seeking consistent k-nearest neighbors in both coordinate and feature spaces is a prevalent strategy employed in previous approaches. However, the vicinity of an inlier contains numerous irregular false correspondences (outliers), which leads them to mistakenly become neighbors according to the similarity constraint of nearest neighbors. To tackle this issue, we propose a global-graph space to seek consistent neighbors with similar graph structures. This is achieved by using a global connected graph to explicitly render the affinity relationship between correspondences based on the spatial and feature consistency. Furthermore, to enhance the robustness of method for various matching scenes, we develop a neighbor consistency block to adequately leverage the potential of three types of neighbors. The consistency can be progressively mined by sequentially extracting intra-neighbor context and exploring inter-neighbor interactions. Ultimately, we present a Neighbor Consistency Mining Network (NCMNet) to estimate the parametric models and remove outliers. Extensive experimental results demonstrate that the proposed method outperforms other state-of-the-art methods on various benchmarks for two-view geometry estimation. Meanwhile, four extended tasks, including remote sensing image registration, point cloud registration, 3D reconstruction, and visual localization, are conducted to test the generalization ability.

PRF-Net: A Progressive Remote Sensing Image Registration and Fusion Network.

Most of the existing fusion algorithms are not robust to unregistered input images. Even after image registration, nonlinear nonregistration may persist in the local areas of the images, leading to poor quality in the fused image. So, as to tackle these challenges, a progressive remote sensing image registration and fusion network is proposed in this article, and named PRF-Net, which is particularly useful when two images are from different platforms. First, a registration network is designed to register the input image patches, which includes a global spatial transform network (GSTN) and a local spatial warp network (LSWN). The GSTN is primarily used for coarse registration, applying rigid transformation to globally align the input images. After coarse registration, the preliminarily registered moving image is input into the LSWN for local fine-tuning to maximize correlation between the input image patches. Subsequently, the fine registered images are degraded and input into the fusion network to generate the fused image. To maintain sufficient spectral and spatial information of the fused image, a multiscale feature extraction (MSFE) block with a highly interpretable spatial details attention (SDA) block is designed, which can enhance the ability of fusion network to extract and preserve spatial details and spectral information. Three groups of experiments conducted on four types of remote sensing images give evidence of that the proposed PRF-Net exhibits excellent performance in both reduced and full resolutions, showcasing its outstanding registration and fusion quality.

Real-Time Registration of Unmanned Aerial Vehicle Hyperspectral Remote Sensing Images Using an Acousto-Optic Tunable Filter Spectrometer

Differences in field of view may occur during unmanned aerial remote sensing imaging applications with acousto-optic tunable filter (AOTF) spectral imagers using zoom lenses. These differences may stem from image size deformation caused by the zoom lens, image drift caused by AOTF wavelength switching, and drone platform jitter. However, they can be addressed using hyperspectral image registration. This article proposes a new coarse-to-fine remote sensing image registration framework based on feature and optical flow theory, comparing its performance with that of existing registration algorithms using the same dataset. The proposed method increases the structure similarity index by 5.2 times, reduces the root mean square error by 3.1 times, and increases the mutual information by 1.9 times. To meet the real-time processing requirements of the AOTF spectrometer in remote sensing, a development environment using VS2023+CUDA+OPENCV was established to improve the demons registration algorithm. The registration algorithm for the central processing unit+graphics processing unit (CPU+GPU) achieved an acceleration ratio of ~30 times compared to that of a CPU alone. Finally, the real-time registration effect of spectral data during flight was verified. The proposed method demonstrates that AOTF hyperspectral imagers can be used in real-time remote sensing applications on unmanned aerial vehicles.

Remote sensing image registration method based on synchronous atmospheric correction

Image registration is a crucial preprocessing step in remote sensing applications, integrating information from multiple images to achieve synergistic advantages. Nevertheless, aerosols characterized by spatiotemporal heterogeneity can result in the blurring of remote-sensing images, thereby compromising the accuracy of image registration. This paper begins by analyzing the basic principles of atmospheric correction and image registration. The variations in atmospheric radiative contribution caused by aerosol changes in real-world scenarios were simulated, along with an examination of the relationship between atmospheric effects and the quantity of image features. Subsequently, addressing the challenge posed by insufficient synchronicity in aerosol parameters and the influence of atmospheric effects on remote sensing image registration, we propose a registration method based on synchronous atmospheric correction. This approach utilizes the Airborne Synchronous Monitoring Atmospheric Corrector (ASMAC) to obtain aerosol optical depth and column water vapor images for synchronous atmospheric correction of remote sensing images, along with the assessment of the registration transformation matrix. Finally, airborne experiments involving ASMAC and high-resolution cameras are conducted to validate the proposed method's improvement in remote sensing image registration accuracy. Experimental results demonstrate the effectiveness of the proposed method, showcasing an increase in the number of features and improvements in quantitative evaluation metrics. Specifically, the normalized correlation coefficient improved by up to 2.408%, while the normalized mutual information increased by a maximum of 1.395%, a maximum feature count and successfully matched features improvement of 21.1% and 38.5%

Automated Registration of Full Moon Remote Sensing Images Based on Triangulated Network Constraints

Abstract. The registration of full-moon remote sensing images constitutes a pivotal stage in the fusion analysis of multiple lunar remote sensing datasets. Addressing prevailing issues in automatic registration, such as the broad width of full-moon data, significant internal distortion, and texture distortion in high-latitude regions, this paper proposes a method for automatic matching and correction based on triangulation constraints. The approach employs a matching strategy progressing from coarse to fine and from sparse to dense. It optimizes and combines multiple existing matching algorithms, enhances the extraction of initial network points, constructs irregular triangulation networks using these points, conducts dense matching with each triangulation network as a basic unit, and introduces a geometric correction method based on triangulation network + grid (TIN + GRID) for the registration of full-moon data. For the matching of full-moon remote sensing images in high-latitude regions, a novel approach involving memory projection forward transformation-matching-projection inverse transformation is adopted. Through registration experiments with full-moon image data and an analysis of registration accuracy at different latitudes, the average mean square error is found to be less than 2 pixels. These results signify the efficacy of the proposed method in effectively addressing the automatic registration challenges encountered in full-moon remote sensing images.

PMA-Net: Progressive multi-stage adaptive feature learning for two-view correspondence

Establishing high-quality correspondences is a fundamental step for many computer vision tasks. Leveraging the local consistency of correct correspondences (i.e., inliers) has been a prevalent approach to distinguish them from incorrect correspondences (i.e., outliers). However, the random distribution of a significant proportion of outliers complicates the local neighborhood construction of inliers, making it inevitably contain many outliers, which compromises the reliability of the consistency information. In this paper, we propose a Progressive Multi-Stage Adaptive Feature Learning Network (PMA-Net) to solve this problem, which progressively and adaptively learns over multiple stages to remove random outliers embedded in the initial consistency. Specifically, we propose an Adaptive Dynamic Graph Construction (ADGC) module to construct a global topological graph with a high inlier ratio by calculating the affinity between the corresponding neighbors. Additionally, we also design a Feature Mapping Processing (FMP) block and Multi-Stage Prediction (MSP) block to improve the accuracy of subsequent local neighborhood information aggregation, as well as information loss due to channel dimension compression in the prediction phase. Experimental results in camera pose estimation, remote sensing image registration, and homography estimation demonstrate that the proposed PMA-Net performs better than other state-of-the-art methods on various public datasets. Code: https://github.com/XiaojieLi11/PMA-Net

Multimodal remote sensing image registration based on adaptive multi-scale PIIFD

Multimodal remote sensing image registration based on adaptive multi-scale PIIFD

Local feature matching using deep learning: A survey

Local feature matching enjoys wide-ranging applications in the realm of computer vision, encompassing domains such as image retrieval, 3D reconstruction, and object recognition. However, challenges persist in improving the accuracy and robustness of matching due to factors like viewpoint and lighting variations. In recent years, the introduction of deep learning models has sparked widespread exploration into local feature matching techniques. The objective of this endeavor is to furnish a comprehensive overview of local feature matching methods. These methods are categorized into two key segments based on the presence of detectors. The Detector-based category encompasses models inclusive of Detect-then-Describe, Joint Detection and Description, Describe-then-Detect, as well as Graph Based techniques. In contrast, the Detector-free category comprises CNN Based, Transformer Based, and Patch Based methods. Our study extends beyond methodological analysis, incorporating evaluations of prevalent datasets and metrics to facilitate a quantitative comparison of state-of-the-art techniques. The paper also explores the practical application of local feature matching in diverse domains such as Structure from Motion, Remote Sensing Image Registration, and Medical Image Registration, underscoring its versatility and significance across various fields. Ultimately, we endeavor to outline the current challenges faced in this domain and furnish future research directions, thereby serving as a reference for researchers involved in local feature matching and its interconnected domains. A comprehensive list of studies in this survey is available at https://github.com/vignywang/Awesome-Local-Feature-Matching.

Radiation-Variation Insensitive Coarse-to-Fine Image Registration for Infrared and Visible Remote Sensing Based on Zero-Shot Learning

Infrared and visible remote sensing image registration is significant for utilizing remote sensing images to obtain scene information. However, it is difficult to establish a large number of correct matches due to the difficulty in obtaining similarity metrics due to the presence of radiation variation between heterogeneous sensors, which is caused by different imaging principles. In addition, the existence of sparse textures in infrared images as well as in some scenes and the small number of relevant trainable datasets also hinder the development of this field. Therefore, we combined data-driven and knowledge-driven methods to propose a Radiation-variation Insensitive, Zero-shot learning-based Registration (RIZER). First, RIZER, as a whole, adopts a detector-free coarse-to-fine registration framework, and the data-driven methods use a Transformer based on zero-shot learning. Next, the knowledge-driven methods are embodied in the coarse-level matches, where we adopt the strategy of seeking reliability by introducing the HNSW algorithm and employing a priori knowledge of local geometric soft constraints. Then, we simulate the matching strategy of the human eye to transform the matching problem into a model-fitting problem and employ a multi-constrained incremental matching approach. Finally, after fine-level coordinate fine tuning, we propose an outlier culling algorithm that only requires very few iterations. Meanwhile, we propose a multi-scene infrared and visible remote sensing image registration dataset. After testing, RIZER achieved a correct matching rate of 99.55% with an RMSE of 1.36 and had an advantage in the number of correct matches, as well as a good generalization ability for other multimodal images, achieving the best results when compared to some traditional and state-of-the-art multimodal registration algorithms.

Feature matching based on Gaussian kernel convolution and minimum relative motion

Feature matching is a necessary and important step for remote sensing image registration, intended to establish reliable point correspondences between two sets of features. In this paper, we propose a feature registration model based on local relative motion, which combines Gaussian kernel convolution with relative motion (GRM) vector to obtain better results by removing wrong matches and improving the inlier point accuracy. We first establish putative matching based on the similarity between local descriptors. Then, the preliminary hypothetical matching point set is filtered using consistency with nearest neighbors among the inlier points to obtain a more accurate motion vector, and to fit the real motion vector through the Gaussian convolution kernel. Finally, we find the displacement between the fitted motion vector and the matching generated motion vector. And combine the displacement with the optimization model to find the inlier point set. Experimental results show that our GRM method outperforms related work, achieving better matching results.

Hardware-Optimized Architecture of On-board Registration for Remote-Sensing Images ——Take SURF as an Example

Hardware-Optimized Architecture of On-board Registration for Remote-Sensing Images ——Take SURF as an Example

A Unified Deep Learning Network for Remote Sensing Image Registration and Change Detection

A Unified Deep Learning Network for Remote Sensing Image Registration and Change Detection

Aerial remote sensing image registration based on dense residual network of asymmetric convolution

Aerial remote sensing image registration based on dense residual network of asymmetric convolution

Aerial remote sensing image registration based on dense residual network of asymmetric convolution

Aerial remote sensing image registration based on dense residual network of asymmetric convolution

Multimodal Remote Sensing Image Registration Based on Adaptive Spectrum Congruency

Multimodal Remote Sensing Image Registration Based on Adaptive Spectrum Congruency

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Robust Region Feature Extraction With Salient MSER and Segment Distance-Weighted GLOH for Remote Sensing Image Registration

Robust Region Feature Extraction With Salient MSER and Segment Distance-Weighted GLOH for Remote Sensing Image Registration

A method of UAV visible light remote sensing image registration based on eigenvector technique

As the advancement of unmanned aerial vehicle technology and deep learning technology, many deep learning technologies have been applied in the field of unmanned aerial vehicle image research. To improve the registration precision and efficiency of unmanned aerial vehicle visible light remote sensing images, this study uses the idea of eigenvectors to build an unmanned aerial vehicle visible light remote sensing image registration model. Firstly, the traditional registration model is optimized by multi-branch attention mechanism, and then the final remote sensing image registration model was constructed by combining convolutional neural network and mask prediction module. Testing the performance of the model, and the experiment findings expresses that the registration error of the final model is about 0.05, the registration precision is more than 0.95, and the registration time is within 2s, so it has a good performance of image registration. In conclusion, the image registration model designed by this research institute can improve the existing problems in the current registration field and provide new ideas for improving the image registration effect.

AA-LMM: Robust Accuracy-Aware Linear Mixture Model for Remote Sensing Image Registration

Remote sensing image registration has been widely applied in military and civilian fields, such as target recognition, visual navigation and change detection. The dynamic changes in the sensing environment and sensors bring differences to feature point detection in amount and quality, which is still a common and intractable challenge for feature-based registration approaches. With such multiple perturbations, the extracted feature points representing the same physical location in space may have different location accuracy. Most existing matching methods focus on recovering the optimal feature correspondences while they ignore the diversities of different points in position, which easily brings the model into a bad local extrema, especially when existing with the outliers and noises. In this paper, we present a novel accuracy-aware registration model for remote sensing. A soft weighting is designed for each sample to preferentially select more reliable sample points. To better estimate the transformation between input images, an optimal sparse approximation is applied to approach the transformation by multiple iterations, which effectively reduces the computation complexity and also improves the accuracy of approximation. Experimental results show that the proposed method outperforms the state-of-the-art approaches in both matching accuracy and correct matches.

Research on multi-source remote sensing image registration technology based on Baker mapping

ABSTRACT To address the issues of inaccurate estimation of registration parameters and high mismatch rate in feature based remote sensing image registration, a registration method based on global feature triangle similarity is proposed. This method utilizes the similarity principle of feature triangles to evaluate the global geometric similarity of matching feature points to eliminate mismatched points. In addition, due to the sensitivity of phase information in the frequency domain to spatial transformations and structural differences, as well as its robustness to lighting and noise, a phase structure consistency measurement method is proposed for developing feature point position adjustment strategies. The results indicate that the registration method proposed by the research institute achieved the lowest RMSE with a size of 1.51. In terms of IRMSE indicators, compared to the RANSAC measurement model, the PH SSIM measurement model has a mean decrease of 0.253. This indicates that the improved registration model proposed in the study has advantages in improving registration accuracy. The innovation of this study lies in constructing a matching feature point evaluation model to eliminate mismatched points, and proposing a remote sensing image registration method based on mismatch point removal and feature point position adjustment.