Remote Sensing Image Matching Research Articles

Robust Multimodal Image Matching Based on Radiation Invariant Phase Correlation

Abstract. Due to the influence of nonlinear radiation distortion and geometric deformation, achieving multimodal image matching remains a challenging task. To address these issues, this paper proposes a method called radiation invariant phase correlation (RIPC) to simultaneously estimate the rotation, scale, and displacement changes of multimodal image pairs. Firstly, based on the local structure characteristics of the image itself, we harness the nonlinear invariance of kernel canonical correlation analysis to devise the multimodal local self-correlation (MLSC) descriptor. This descriptor is resilient to nonlinear radiative differences, as well as local rotation and scale variations. Subsequently, we incorporate the log-polar coordinate transformation to capture the overall rotation and scale changes in the image, enabling independent representation of these factors on the Cartesian coordinate system. Finally, drawing upon the continuity of displacement estimation, as well as rotation and scale estimation, we construct a five-dimensional descriptor tailored for phase correlation. Extensive experiments conducted on five open-source datasets demonstrate that our proposed method surpasses state-of-the-art (SOTA) techniques in matching performance. Furthermore, our RIPC method achieves matching accuracy within 2-pixel threshold, which underscores its effectiveness in multimodal remote sensing image matching.

PDE-Constrained Scale Optimization Selection for Feature Detection in Remote Sensing Image Matching

Feature detection and matching is the key technique for remote sensing image processing and related applications. In this paper, a PDE-constrained optimization model is proposed to determine the scale levels advantageous for feature detection. A variance estimation technique is introduced to treat the observation optical images polluted by additive zero-mean Gaussian noise and determine the parameter of a nonlinear scale space governed by the partial differential equation. Additive Operator Splitting is applied to efficiently solve the PDE constraint, and an iterative algorithm is proposed to approximate the optimal subset of the original scale level set. The selected levels are distributed more uniformly in the total variation sense and helpful for generating more accurate and robust feature points. The experimental results show that the proposed method can achieve about a 30% improvement in the number of correct matches with only a small increase in time cost.

A dense matching method for remote sensing images fused with CPS denoising

Dense matching of remote sensing images is crucial for 3D reconstruction. This study proposes an enhanced dense matching method employing the CPS image denoising algorithm, aiming to boost the SGM algorithm's accuracy and efficiency in remote sensing image matching. The stereo image pair's quality is evaluated using the PSNR index, and a decision-making criterion based on the CPS algorithm is incorporated to determine the need for denoising. Preprocessing steps, including image cropping and pixel coordinate transformation, significantly reduce computational requirements. An epipolar line model, minimizing the disparity between two pixels, is used for calculations. This model is employed to construct an epipolar image, enhancing the accuracy and efficiency of the process. The study conducted experimental validation and analysis of the mismatch rate, running time, and denoising effect of the algorithm using the Middlebury 2021 stereo datasets. Additionally, the matching results of the World-View3 satellite stereo image pairs were visualized and analyzed. The experimental results indicate that the proposed algorithm reduces the average mismatch rate by 13.1% and increases the running speed by about 3 to 4 times compared to the SGBM algorithm. Specifically, the denoising effect reduces the mismatch rate of the reconstructed image by an average of 8.97%. The results indicate that the CPS method effectively addresses dense matching challenges in the presence of image blur and noise, thereby improving the operational efficiency and accuracy of the dense matching algorithm.

A radiometric and rotational invariant feature descriptor for multimodal remote sensing image matching using the monogenic signal

ABSTRACT Multimodal remote sensing image matching is still a challenging task, and its matching performance will highly influence the subsequent applications, such as image fusion, image mosaic and change detection. To tackle this problem, we propose a radiometric and rotational invariant descriptor (RRID) using the monogenic signal for multimodal image matching. First of all, a multiscale feature asymmetry measurement is proposed to obtain more reliable detected points, in which the multiscale feature asymmetry can be computed from the intermediate results from the monogenic signal; then a multiscale local phase orientation computation algorithm is designed for the main orientation assignment of each keypoint, in which the local phase orientation can also be provided by the monogenic signal construction; finally, a circular local window around each keypoint is extracted, a histogram of local phase orientations weighted by phase congruency is calculated for the log-polar feature description. RRID is compared with other state-of-the-art multimodal image matching methods on three publicly available datasets. Experimental results have demonstrated that our proposed method outperforms the compared methods in terms of precision, recall, and F1-measure criteria and also has achieved the lowest RMSEs.

Performance evaluation of state-of-the-art multimodal remote sensing image matching methods in the presence of noise

Performance evaluation of state-of-the-art multimodal remote sensing image matching methods in the presence of noise

Robust Multimodal Remote Sensing Image Matching Based on Enhanced Oriented Self-Similarity Descriptor

Robust Multimodal Remote Sensing Image Matching Based on Enhanced Oriented Self-Similarity Descriptor

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

A Heterogeneous Remote Sensing Image Matching Method for Urban Areas With Complex Terrain Based on 3D Spatial Relationship Constraints

A Heterogeneous Remote Sensing Image Matching Method for Urban Areas With Complex Terrain Based on 3D Spatial Relationship Constraints

Shape-Adaptive Modality Independent Region Descriptor for Multimodal Remote Sensing Image Matching

Shape-Adaptive Modality Independent Region Descriptor for Multimodal Remote Sensing Image Matching

Multimodal Remote Sensing Image Matching Based on Weighted Structure Saliency Feature

Multimodal Remote Sensing Image Matching Based on Weighted Structure Saliency Feature

Multi-Modal Image Registration Based on Phase Exponent Differences of the Gaussian Pyramid

In multi-modal images (MMI), the differences in their imaging mechanisms lead to large signal-to-noise ratio differences, which means that the matching of geometric invariance and the matching accuracy of the matching algorithms often cannot be balanced. Therefore, how to weaken the signal-to-noise interference of MMI, maintain good scale and rotation invariance, and obtain high-precision matching correspondences becomes a challenge for multimodal remote sensing image matching. Based on this, a lightweight MMI alignment of the phase exponent of the differences in the Gaussian pyramid (PEDoG) is proposed, which takes into account the phase exponent differences of the Gaussian pyramid with normalized filtration, i.e., it achieves the high-precision identification of matching correspondences points while maintaining the geometric invariance of multi-modal matching. The proposed PEDoG method consists of three main parts, introducing the phase consistency model into the differential Gaussian pyramid to construct a new phase index. Then, three types of MMI (multi-temporal image, infrared–optical image, and map–optical image) are selected as the experimental datasets and compared with the advanced matching methods, and the results show that the NCM (number of correct matches) of the PEDoG method displays a minimum improvement of 3.3 times compared with the other methods, and the average RMSE (root mean square error) is 1.69 pixels, which is the lowest value among all the matching methods. Finally, the alignment results of the image are shown in the tessellated mosaic mode, which shows that the feature edges of the image are connected consistently without interlacing and artifacts. It can be seen that the proposed PEDoG method can realize high-precision alignment while taking geometric invariance into account.

2OC: A General Automated Orientation and Orthorectification Method for Corona KH-4B Panoramic Imagery

Due to a lack of geographical reference information, complex panoramic camera models, and intricate distortions, including radiation, geometric, and land cover changes, it can be challenging to effectively apply the large number (800,000+) of high-resolution Corona KH-4B panoramic images from the 1960s and 1970s for surveying-related tasks. This limitation hampers their significant potential in the remote sensing of the environment, urban planning, and other applications. This study proposes a method called 2OC for the automatic and accurate orientation and orthorectification of Corona KH-4B images, which is based on generalized control information from reference images such as Google Earth orthophoto. (1) For the Corona KH-4B panoramic camera, we propose an adaptive focal length variation model that ensures accuracy and consistency. (2) We introduce a robust multi-source remote sensing image matching algorithm, which includes an accurate primary orientation estimation method, a multi-threshold matching enhancement strategy based on scale, orientation, and texture (MTE), and a model-guided matching strategy. These techniques are employed to extract high-accuracy generalized control information for Corona images with significant geometric distortions and numerous weak texture areas. (3) A time-iterative Corona panoramic digital differential correction method is proposed. The orientation and orthorectification results of KH-4B images from multiple regions, including the United States, Russia, Austria, Burkina Faso, Beijing, Chongqing, Gansu, and the Qinghai–Tibet Plateau in China, demonstrate that 2OC not only achieves automation but also attains a state-of-the-art level of generality and accuracy. Specifically, the standard deviation of the orientation is less than 2 pixels, the mosaic error of orthorectified images is approximately 1 pixel, and the standard deviation of ground checkpoints is better than 4 m. In addition, 2OC can provide a longer time series analysis of data from 1962 to 1972, benefiting various fields such as environmental remote sensing and archaeology.

Multisource remote sensing image matching based on expanded phase consistency

Multisource remote sensing image matching based on expanded phase consistency

Weak texture remote sensing image matching based on hybrid domain features and adaptive description method

AbstractWeak texture remote sensing image (WTRSI) has characteristics such as low reflectivity, high similarity of neighbouring pixels and insignificant differences between regions. These factors cause difficulties in feature extraction and description, which lead to unsuccessful matching. Therefore, this paper proposes a novel hybrid‐domain features and adaptive description (HFAD) approach to perform WTRSI matching. This approach mainly provides two contributions: (1) a new feature extractor that combines both the spatial domain scale space and the frequency domain scale space is established, where a weighted least square filter combined with a phase consistency filter is used to establish the frequency domain scale space; and (2) a new log‐polar descriptor of adaptive neighbourhood (LDAN) is established, where the neighbourhood window size of each descriptor is calculated according to the log‐normalised intensity value of feature points. This article prepares some remote sensing images under weak texture scenes which include deserts, dense forests, waters, ice and snow, and shadows. The data set contains 50 typical image pairs, on which the proposed HFAD was demonstrated and compared with state‐of‐the‐art matching algorithms (RIFT, HOWP, KAZE, POS‐SIFT and SIFT). The statistical results of the comparative experiment show that the HFAD can achieve the accuracy of matching within two pixels and confirm that the proposed algorithm is robust and effective.

StateNet: Deep State Learning for Robust Feature Matching of Remote Sensing Images.

Seeking good correspondences between two images is a fundamental and challenging problem in the remote sensing (RS) community, and it is a critical prerequisite in a wide range of feature-based visual tasks. In this article, we propose a flexible and general deep state learning network for both rigid and nonrigid feature matching, which provides a mechanism to change the state of matches into latent canonical forms, thereby weakening the degree of randomness in matching patterns. Different from the current conventional strategies (i.e., imposing a global geometric constraint or designing additional handcrafted descriptor), the proposed StateNet is designed to perform alternating two steps: 1) recalibrates matchwise feature responses in the spatial domain and 2) leverages the spatially local correlation across two sets of feature points for transformation update. For this purpose, our network contains two novel operations: adaptive dual-aggregation convolution (ADAConv) and point rendering layer (PRL). These two operations are differentiable, so our network can be inserted into the existing classification architecture to reduce the cost of establishing reliable correspondences. To demonstrate the robustness and universality of our approach, extensive experiments on various real image pairs for feature matching are conducted. Experiments reveal the superiority of our StateNet significantly over the state-of-the-art alternatives.

An Optical Remote Sensing Image Matching Method Based on the Simple and Stable Feature Database

Satellite remote sensing has entered the era of big data due to the increase in the number of remote sensing satellites and imaging modes. This presents significant challenges for the processing of remote sensing systems and will result in extremely high real-time data processing requirements. The effective and reliable geometric positioning of remote sensing images is the foundation of remote sensing applications. In this paper, we propose an optical remote sensing image matching method based on a simple stable feature database. This method entails building the stable feature database, extracting local invariant features that are comparatively stable from remote sensing images using an iterative matching strategy, and storing useful information about the features. Without reference images, the feature database-based matching approach potentially saves storage space for reference data while increasing image processing speed. To evaluate the performance of the feature database matching method, we train the feature database with various local invariant feature algorithms on different time phases of Gaofen-2 (GF-2) images. Furthermore, we carried out matching comparison experiments with various satellite images to confirm the viability and stability of the feature database-based matching method. In comparison with direct matching using the classical feature algorithm, the feature database-based matching method in this paper can essentially improve the correct rate of feature point matching by more than 30% and reduce the matching time by more than 40%. This method improves the accuracy and timeliness of image matching, potentially solves the problem of large storage space occupied by the reference data, and has great potential for fast matching of optical remote sensing images.

An InSAR Interference Fringe-Matching Algorithm Based on Mountain Branch Points

Interferometric synthetic aperture radar (InSAR) is an integrated navigation technique that can be used for aircraft positioning and attitude retrieval, regardless of weather conditions. The key aspect of the entire system is interference fringe matching, which has not been extensively researched in existing literature. To address this gap, this paper proposes a terrain-feature-based interference fringe-matching algorithm. The proposed algorithm first extracts mountain line features from the interference fringes and identifies mountain branch points as key points for feature matching. A threshold is set to eliminate false detections of mountain branch points caused by phase mutation. Matching is then carried out by combining the mountain line features and curvature design feature descriptors of the area around a branch point. The proposed algorithm is verified using interference fringe data obtained from an actual flight experiment in Inner Mongolia, China, compared to a reference interference fringe dataset with errors. The results show that, under the condition of position error, the proposed algorithm yields 72 matching inliers with a precision of 0.103, a recall of 0.128, and an F1-Score of 0.144. Compared to traditional algorithms, our proposed algorithm significantly improves the problem of mismatch and opens up new possibilities for downstream interference fringe matching navigation technology. Furthermore, the proposed algorithm provides a new approach for remote sensing image matching using terrain features.

A Multiscale Unsupervised Orientation Estimation Method With Transformers for Remote Sensing Image Matching

Estimating the orientations of remote sensing images is a very important step in remote sensing image matching and is now gradually receiving widespread attention. However, due to the inability to explicitly define the standard orientations of feature points, the current methods still produce feature point orientation estimation errors, resulting in reduced matching accuracy. In this letter, we propose a multiscale unsupervised orientation estimation method with transformers, in which we use a multiscale feature extraction module to aggregate rich semantic features and a transformer-based attention mechanism module to address robust feature extraction in weakly textured regions while predicting the orientations of feature points through a carefully designed loss function. We set up image matching experiments on remote sensing images in different scenes for comparison purposes, and the experimental results show that our proposed method achieves substantially improved orientation estimation accuracy and improved image matching performance.

Modality-Invariant Structural Feature Representation for Multimodal Remote Sensing Image Matching

Modality-Invariant Structural Feature Representation for Multimodal Remote Sensing Image Matching

Combining Phase Congruency and Self-Similarity Features for Multimodal Remote Sensing Image Matching

The structural features using self-similarity have become more popular for multimodal remote sensing image matching. However, mostly because of significant geometric distortions and nonlinear intensity differences between images, these methods produce a limited matching performance when directly applied to multimodal remote sensing images. To address that, we propose a novel feature descriptor named pyramid features of orientated self-similarity (POSS) for multimodal remote sensing image matching, which integrates phase congruency (PC) into the self-similarity model for better encoding structural information. Unlike these conventional self-similarity-based descriptors, the POSS is constructed by using PC instead of image intensity in a pyramid manner and thus has better discriminability and robustness to modality variations. In addition, we design a uniform multimoment of PC (UMMPC) detector for feature detection, which can improve the number, distribution, and repeatability of feature points. Experimental results demonstrate its superior applicability of the proposed method over the state-of-the-art methods, showing much better matching performance.