Number Of Correct Matches Research Articles

PDE-Constrained Scale Optimization Selection for Feature Detection in Remote Sensing Image Matching

Feature detection and matching is the key technique for remote sensing image processing and related applications. In this paper, a PDE-constrained optimization model is proposed to determine the scale levels advantageous for feature detection. A variance estimation technique is introduced to treat the observation optical images polluted by additive zero-mean Gaussian noise and determine the parameter of a nonlinear scale space governed by the partial differential equation. Additive Operator Splitting is applied to efficiently solve the PDE constraint, and an iterative algorithm is proposed to approximate the optimal subset of the original scale level set. The selected levels are distributed more uniformly in the total variation sense and helpful for generating more accurate and robust feature points. The experimental results show that the proposed method can achieve about a 30% improvement in the number of correct matches with only a small increase in time cost.

A lightweight deep convolutional network with inverted residuals for matching optical and SAR images

ABSTRACT Matching optical and synthetic aperture radar (SAR) images is often challenged by intricate geometric distortion and nonlinear radiation differences, leading to insufficient and unevenly distributed corresponding points. To tackle this issue, we propose a lightweight deep convolutional network with inverted residuals for optical and SAR image matching. Initially, a fully convolutional neural network (FCNN) is designed to extract high-level and semantic features, robustly capturing universal characteristics between optical and SAR images, adept at handling geometric distortion and nonlinear radiation changes. Notably, we integrate a lightweight architecture with inverted residuals into FCNN to adeptly extract local and global contextual information, facilitating feature reuse and minimizing the loss of crucial features. Additionally, a vector-refined module is deployed to refine dense features, filtering out redundant information. Subsequently, a coarse-to-fine strategy is employed to further eliminate gross errors or incorrect matches. Finally, we evaluate the performance of the proposed network in optical and SAR image matching against manually-designed methods and state-of-the-art deep learning techniques. Experimental results demonstrate that our network significantly surpasses existing methods in terms of the number of correct matches and matching accuracy. Specifically, our proposed network achieves at least a 2.8 times increase in correct matches and an 18% improvement in matching accuracy compared to existing methods.

Quasi-Dense Matching for Oblique Stereo Images through Semantic Segmentation and Local Feature Enhancement

This paper proposes a quasi-dense feature matching algorithm that combines image semantic segmentation and local feature enhancement networks to address the problem of the poor matching of image features because of complex distortions, considerable occlusions, and a lack of texture on large oblique stereo images. First, a small amount of typical complex scene data are used to train the VGG16-UNet, followed by completing the semantic segmentation of multiplanar scenes across large oblique images. Subsequently, the prediction results of the segmentation are subjected to local adaptive optimization to obtain high-precision semantic segmentation results for each planar scene. Afterward, the LoFTR (Local Feature Matching with Transformers) strategy is used for scene matching, enabling enhanced matching for regions with poor local texture in the corresponding planes. The proposed method was tested on low-altitude large baseline stereo images of complex scenes and compared with five classical matching methods. Results reveal that the proposed method exhibits considerable advantages in terms of the number of correct matches, correct rate of matches, matching accuracy, and spatial distribution of corresponding points. Moreover, it is well-suitable for quasi-dense matching tasks of large baseline stereo images in complex scenes with considerable viewpoint variations.

Radiation-Variation Insensitive Coarse-to-Fine Image Registration for Infrared and Visible Remote Sensing Based on Zero-Shot Learning

Infrared and visible remote sensing image registration is significant for utilizing remote sensing images to obtain scene information. However, it is difficult to establish a large number of correct matches due to the difficulty in obtaining similarity metrics due to the presence of radiation variation between heterogeneous sensors, which is caused by different imaging principles. In addition, the existence of sparse textures in infrared images as well as in some scenes and the small number of relevant trainable datasets also hinder the development of this field. Therefore, we combined data-driven and knowledge-driven methods to propose a Radiation-variation Insensitive, Zero-shot learning-based Registration (RIZER). First, RIZER, as a whole, adopts a detector-free coarse-to-fine registration framework, and the data-driven methods use a Transformer based on zero-shot learning. Next, the knowledge-driven methods are embodied in the coarse-level matches, where we adopt the strategy of seeking reliability by introducing the HNSW algorithm and employing a priori knowledge of local geometric soft constraints. Then, we simulate the matching strategy of the human eye to transform the matching problem into a model-fitting problem and employ a multi-constrained incremental matching approach. Finally, after fine-level coordinate fine tuning, we propose an outlier culling algorithm that only requires very few iterations. Meanwhile, we propose a multi-scene infrared and visible remote sensing image registration dataset. After testing, RIZER achieved a correct matching rate of 99.55% with an RMSE of 1.36 and had an advantage in the number of correct matches, as well as a good generalization ability for other multimodal images, achieving the best results when compared to some traditional and state-of-the-art multimodal registration algorithms.

Robust registration of multi-modal remote sensing images based on multi-dimensional oriented self-similarity features

Registration of multi-modal remote sensing images (MRSI) is crucial for unlocking the full potential of heterogeneous remote sensing imagery. However, achieving accurate registration among MRSI is challenging due to the trade-off between geometric invariance and matching accuracy, caused by differences in signal-to-noise ratio and nonlinear radiometric distortion (NRD) arising from varying imaging mechanisms. To tackle the challenge, this paper proposes a lightweight and hybrid feature-guided registration algorithm for MRSI called the hybrid registration algorithm based on multi-dimensional oriented self-similarity features (MOSS). MOSS leverages the advantages of multi-dimensional oriented self-similarity features to progressively enhance registration performance. In the hybrid feature coarse matching stage, oriented self-similarity features are extracted from MRSI, and their directional information is utilized for feature description to estimate the initial affine transformation. The fine matching under multi-dimensional oriented self-similarity features stage takes the outputs of the coarse matching stage to perform a template-like matching process. To evaluate the performance of MOSS, comprehensive experiments are conducted using six different combinations of MRSI, and seven state-of-the-art registration algorithms are selected for comparison. The experimental results demonstrate that MOSS outperforms the compared methods, with the number of correct matches being at least about 1.6 times higher than the comparison methods. Moreover, MOSS exhibits the lowest root mean square error across all experiments, with an average RMSE of 1.86 pixels, achieving an RMSE within 2 pixels. This highlights its effectiveness in achieving precise alignment and robust registration of MRSI.

Multi-Modal Image Registration Based on Phase Exponent Differences of the Gaussian Pyramid

In multi-modal images (MMI), the differences in their imaging mechanisms lead to large signal-to-noise ratio differences, which means that the matching of geometric invariance and the matching accuracy of the matching algorithms often cannot be balanced. Therefore, how to weaken the signal-to-noise interference of MMI, maintain good scale and rotation invariance, and obtain high-precision matching correspondences becomes a challenge for multimodal remote sensing image matching. Based on this, a lightweight MMI alignment of the phase exponent of the differences in the Gaussian pyramid (PEDoG) is proposed, which takes into account the phase exponent differences of the Gaussian pyramid with normalized filtration, i.e., it achieves the high-precision identification of matching correspondences points while maintaining the geometric invariance of multi-modal matching. The proposed PEDoG method consists of three main parts, introducing the phase consistency model into the differential Gaussian pyramid to construct a new phase index. Then, three types of MMI (multi-temporal image, infrared–optical image, and map–optical image) are selected as the experimental datasets and compared with the advanced matching methods, and the results show that the NCM (number of correct matches) of the PEDoG method displays a minimum improvement of 3.3 times compared with the other methods, and the average RMSE (root mean square error) is 1.69 pixels, which is the lowest value among all the matching methods. Finally, the alignment results of the image are shown in the tessellated mosaic mode, which shows that the feature edges of the image are connected consistently without interlacing and artifacts. It can be seen that the proposed PEDoG method can realize high-precision alignment while taking geometric invariance into account.

Improving the matching of deformable objects by learning to detect keypoints

We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. By leveraging true correspondences acquired by matching annotated image pairs with a specified descriptor extractor, we train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method, while outperforming the state-of-the-art keypoint detectors on real images of non-rigid objects by 20 p.p. We also apply our method on the complex real-world task of object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task. The source code and trained models are publicly available at https://github.com/verlab/LearningToDetect_PRL_2023.

Robust line segment mismatch removal using point-pair representation and Gaussian-uniform mixture formulation

Line segment matching (LSM) plays an important role in image matching, while it is always a challenging problem due to line fracture and high geometric complexity. In this paper, we propose a line segment mismatch removal to address the sensitivity to segment length and the fracture problem. The mismatch removal removes the massive false matches obtained by the descriptors like LBD. Specifically, we suggest calculating the endpoint error in LSM rather than the direction error in considering the error band and segment length. We propose a point-pair representation, and transform the LSM problem to be an intuitive point-pair alignment problem with a mixture model. The point-pair representation is more robust on the segment length than the linear equation representation used in RANSAC, which significantly preserves short line segments. Then, based on the point-pair representation, we propose a novel Directed Endpoint Drift method to solve the inherent fracture problem of line segments by allowing the endpoints to move along the line to complement the fracture. Compared with the recent learning-based method, the proposed method nearly doubles the number of correct matches on the remote sensing dataset. Especially, the recall is improved by 10% to 30% for both the short segments and the slightly fractured segments, compared with the popular RANSAC and MAGSAC++ methods. The code is available at https://github.com/shenliang16/DEpD.

SRTPN: Scale and Rotation Transform Prediction Net for Multimodal Remote Sensing Image Registration

How to recover geometric transformations is one of the most challenging issues in image registration. To alleviate the effect of large geometric distortion in multimodal remote sensing image registration, a scale and rotate transform prediction net is proposed in this paper. First, to reduce the scale between the reference and sensed images, the image scale regression module is constructed via CNN feature extraction and FFT correlation, and the scale of sensed image can be recovered roughly. Second, the rotation estimate module is developed for predicting the rotation angles between the reference and the scale-recovered images. Finally, to obtain the accurate registration results, LoFTR is employed to match the geometric-recovered images. The proposed registration network was evaluated on GoogleEarth, HRMS, VIS-NIR and UAV datasets with contrast differences and geometric distortions. The experimental results show that the number of correct matches of our model reached 74.6%, and the RMSE of the registration results achieved 1.236, which is superior to the related methods.

Optical and SAR Image Registration Based on Multi-Scale Orientated Map of Phase Congruency

Optical and Synthetic Aperture Radar (SAR) images are highly complementary, and their registrations are a fundamental task for other remote sensing applications. Traditional feature-matching algorithms fail to solve the significant nonlinear radiation difference (NRD) caused by different sensors. To address this problem, a robust registration algorithm with the multi-scale orientated map of phase congruency (MSPCO) is proposed. First, a nonlinear diffusion scale space is established to obtain the scale invariance of feature points. Compared with the linear Gaussian scale space, the nonlinear diffusion scale space can better preserve the edge and texture information. Second, to ensure the quantity and repeatability of features, corner points and edge points are detected on the moment map of phase congruency, respectively, which is the foundation to the next feature matching. Third, the MSPCO descriptor is constructed via the orientation of phase congruency (PCO). PCO is highly robust to NRD, and the different scales of PCOs enhance the robustness of the descriptor. Finally, a feature-matching strategy based on an effective scale ratio is proposed, which reduces the number of comparisons among features and improves computational efficiency. The experimental results show that the proposed method is better than the existing feature-based methods in terms of the number of correct matches and registration accuracy. The registration accuracy is only inferior to that of the most advanced template matching method, and the accuracy difference is within 0.3 pixels, which fully demonstrates the robustness and accuracy of the proposed method in optical and SAR image registration.

Multiscale structural feature transform for multi-modal image matching

Robust and reliable multi-modal image matching is an essential and challenging technique in the multi-modality involved scenarios for image fusion, image mosaic, and visual navigation. Existing image matching methods cannot simultaneously solve the scale and rotation distortion caused by different viewpoints and the nonlinear radiation distortion caused by different imaging environments or mechanisms. To address these problems, a multiscale structural feature transform method is proposed for multi-modal image matching, which consists of three blocks: multiscale structural feature detector, symmetric log-polar descriptor, and point matching with position correction. The multiscale structural feature detector constructs the phase congruency map based Difference of Gaussian image pyramid for the scale-invariant feature points detection to tackle the nonlinear radiation distortion. The feature points are detected within 10-neighborhood instead of 26-neighborhood to produce sufficient points on the phase congruency map with very sparse texture. The symmetric log-polar descriptor utilizes the multiscale structure principal direction and symmetric bins to improve the robustness of the descriptor to rotation and nonlinear radiation distortion. At last, the point matching with position correction corrects the position of feature points to compensate the position offset between different images caused by the scale and rotation distortion and the nonlinear radiation distortion, which increases the number of correct matches and improves the transformation consistency between matched points. In the experiment, the proposed method in this paper is compared with seven state-of-the-art methods. Experimental results verify the superiority of the proposed method and its robustness to the scale and rotation distortion and the nonlinear radiation distortion. In addition, we demonstrate the effectiveness of the proposed three blocks by the ablation experiments.

Robust feature matching via progressive smoothness consensus

Feature matching is a long-standing fundamental and critical problem in computer vision and photogrammetry. The indirect matching strategy has become a popular choice because of its high precision and generality, but it finds only a limited number of correct matches, and the mismatch removal phase does not utilize the critical feature descriptors. To this end, this paper proposes a novel and effective feature matching method, named Progressive Smoothness Consensus (PSC). Our PSC designs an objective function to directly construct correct matches from two feature point sets. To optimize the objective, we introduce a stepwise strategy, where a small but reliable match set with the smooth function is used as initialization, and then the correct match set is iteratively enlarged and optimized by match expansion and smooth function estimation, respectively. In addition, the local geometric constraint is added to the compact representation with a Fourier basis, thus improving the estimation precision. We perform the match expansion as a Bayesian formulation to exploit both the spatial distribution and feature description information, thus finding feasible matches to expand the match set. Extensive experiments on feature matching, homography & fundamental matrix estimation, and image registration are conducted, which demonstrate the advantages of our PSC against state-of-the-art methods in terms of generality and effectiveness. Our code is publicly available at https://github.com/XiaYifan1999/Robust-feature-matching-via-Progressive-Smoothness-Consensus.

Histogram of the orientation of the weighted phase descriptor for multi-modal remote sensing image matching

Multi-modal remote sensing image (MRSI) has nonlinear radiation distortion (NRD) and significant contrast differences to which image gradient features are usually sensitive. Although image phase features are more robust against NRD, they might not be much helpful in resolving the problems of directional inversion or phase extreme value mutations that are common in the phase feature calculation. To address these issues, a new MRSI matching method—“histogram of the orientation of weighted phase” (HOWP)—is proposed in this paper. This method distinguishes itself from other methods in three aspects: (1) a feature aggregation strategy is used to optimize feature points by extracting the corner and blob features separately; (2) a novel weighted phase orientation model is established to replace the traditional image gradient orientation features; and (3) a regularization-based log-polar descriptor is constructed to generate robust feature description vectors. To evaluate the performance of the proposed method, we selected 50 sets of typical MRSIs with translation, scale, and rotation differences for comparison with the other four state-of-the-art methods. The results show that our method is more resistant to radiometric distortion and the contrasting differences in MRSIs. It also performs better in tackling the problems of direction reversal and phase extreme value mutation, as evidenced by more, the number of correct matches (NCM). Since the method has improved the average NCM by 1.6–4.5 times, the average success rate by 35.5%, and the average rate of correct matches by 11.1% with an average root of mean-squared error of 1.93 pixels. Moreover, we have put forward an extended version of the HOWP method (Simplified-HOWP) when there is no image rotation, which manifests in an average 0.75 times improvement in NCM of Simplified-HOWP performance over that of the HOWP method. The executable code and test data are linked in https://skyearth.org/publication/project/HOWP/.

Two-Step Matching Method Based on Co-Occurrence Scale Space Combined with Second-Order Gaussian Steerable Filter

Multimodal images refer to images obtained by different sensors, and there are serious nonlinear radiation differences (NRDs) between multimodal images for photos of the same object. Traditional multimodal image matching methods cannot achieve satisfactory results in most cases. In order to better solve the NRD in multimodal image matching, as well as the rotation and scale problems, we propose a two-step matching method based on co-occurrence scale space combined with the second-order Gaussian steerable filter (G-CoFTM). We first use the second-order Gaussian steerable filter and co-occurrence filter to construct the image’s scale space to preserve the image’s edge and detail features. Secondly, we use the second-order gradient direction to calculate the images’ principal direction, and describe the images’ feature points through improved GLOH descriptors. Finally, after obtaining the rough matching results, the optimized 3DPC descriptors are used for template matching to complete the fine matching of the images. We validate our proposed G-CoFTM method on eight different types of multimodal datasets and compare it with five state-of-the-art methods: PSO-SIFT, CoFSM, RIFT, HAPCG, and LPSO. Experimental results show that our proposed method has obvious advantages in matching success rate (SR) and the number of correct matches (NCM). On eight different types of datasets, compared with CoFSM, RIFT, HAPCG, and LPSO, the mean SRs of G-CoFSM are 17.5%, 6.187%, 30.462%, and 32.21%, respectively, and the mean NCMs are 5.322, 11.503, 8.607, and 16.429 times those of the above four methods.

An efficient SIFT-based matching algorithm for optical remote sensing images

ABSTRACT Scale-invariant feature transform (SIFT) and its improved versions have been widely used to match the remote-sensing optical images. However, it is still a challenging task to get enough correct matching pairs by using SIFT-based methods. In this letter, an efficient matching scheme is proposed to increase the number of correct matches. The proposed matching scheme uses scale, orientation, and translation differences between the input images to remove the outliers and to retain the correct matches. At first, the initial matching candidates are obtained by a SIFT-based algorithm. Then, our proposed matching scheme is used to select the correct matching pairs. The proposed method can obtain more correct matches than the state-of-the-art methods. Experiments are performed on five pairs of optical images to verify the performance of the proposed method.

RI-LPOH: Rotation-Invariant Local Phase Orientation Histogram for Multi-Modal Image Matching

To better cope with the significant nonlinear radiation distortions (NRD) and severe rotational distortions in multi-modal remote sensing image matching, this paper introduces a rotationally robust feature-matching method based on the maximum index map (MIM) and 2D matrix, which is called the rotation-invariant local phase orientation histogram (RI-LPOH). First, feature detection is performed based on the weighted moment equation. Then, a 2D feature matrix based on MIM and a modified gradient location orientation histogram (GLOH) is constructed and rotational invariance is achieved by cyclic shifting in both the column and row directions without estimating the principal orientation separately. Each part of the sensed image’s 2D feature matrix is additionally flipped up and down to obtain another 2D matrix to avoid intensity inversion, and all the 2D matrices are concatenated by rows to form the final 1D feature vector. Finally, the RFM-LC algorithm is introduced to screen the obtained initial matches to reduce the negative effect caused by the high proportion of outliers. On this basis, the remaining outliers are removed by the fast sample consensus (FSC) method to obtain optimal transformation parameters. We validate the RI-LPOH method on six different types of multi-modal image datasets and compare it with four state-of-the-art methods: PSO-SIFT, MS-HLMO, CoFSM, and RI-ALGH. The experimental results show that our proposed method has obvious advantages in the success rate (SR) and the number of correct matches (NCM). Compared with PSO-SIFT, MS-HLMO, CoFSM, and RI-ALGH, the mean SR of RI-LPOH is 170.3%, 279.8%, 81.6%, and 25.4% higher, respectively, and the mean NCM is 13.27, 20.14, 1.39, and 2.42 times that of the aforementioned four methods.

MOTIF: MULTI-ORIENTATION TENSOR INDEX FEATURE DESCRIPTOR FOR SAR-OPTICAL IMAGE REGISTRATION

Abstract. The inherent speckle noise in synthetic aperture radar (SAR) images and the significant differences between SAR and optical images in nonlinear radiation give rise to the great difficulty in computing similarity between image features, improving detection accuracy of corresponding points and the efficiency of image matching, thus making the registration of SAR and optical images a long-standing challenging task. To address these issues, a new SAR-optical image registration method was proposed in this paper, namely, Multi-orientation Tensor Index Feature (MoTIF), which is characterized by a lightweight feature descriptor. Specifically, we firstly established a diffusion tensor model based on the information of image gradient orientation. Then, the model was parameterized using polar coordinates to help identify the MoTIF and get the array of indices of maximum value, with which we could draw a multi-orientation index map and thereupon construct the feature vector descriptors. To evaluate the proposed method, seven representative SAR-optical image pairs were tested along with a comparison with other four state-of-the-art methods. Results show that our MoTIF method outperforms the other methods in that it substantially de-speckles SAR images, overcomes nonlinear radiation distortions caused by the differences between SAR and optical images, and achieves high precision and efficiency in image registration. The average number of correct matches (NCM) of 151.0 and the root of mean-squared error (RMSE) of 1.66 pixels obtained by utilizing MoTIF with lower time consumption adds more evidence to its superior performance. The time consumption of the MoTIF method is better than that of the other four methods, and the calculation speed is 4 times faster than that of the LGHD method. Executable code and test data are published in the link https://skyearth.org/publication/project/MoTIF/

3MRS: An Effective Coarse-to-Fine Matching Method for Multimodal Remote Sensing Imagery

The fusion of image data from multiple sensors is crucial for many applications. However, there are significant nonlinear intensity deformations between images from different kinds of sensors, leading to matching failure. To address this need, this paper proposes an effective coarse-to-fine matching method for multimodal remote sensing images (3MRS). In the coarse matching stage, feature points are first detected on a maximum moment map calculated with a phase congruency model. Then, feature description is conducted using an index map constructed by finding the index of the maximum value in all orientations of convolved images obtained using a set of log-Gabor filters. At last, several matches are built through image matching and outlier removal, which can be used to estimate a reliable affine transformation model between the images. In the stage of fine matching, we develop a novel template matching method based on the log-Gabor convolution image sequence and match the template features with a 3D phase correlation matching strategy, given that the initial correspondences are achieved with the estimated transformation. Results show that compared with SIFT, and three state-of-the-art methods designed for multimodal image matching, PSO-SIFT, HAPCG, and RIFT, only 3MRS successfully matched all six types of multimodal remote sensing image pairs: optical–optical, optical–infrared, optical–depth, optical–map, optical–SAR, and day–night, with each including ten different image pairs. On average, the number of correct matches (NCM) of 3MRS was 164.47, 123.91, 4.88, and 4.33 times that of SIFT, PSO-SIFT, HAPCG, and RIFT for the successfully matched image pairs of each method. In terms of accuracy, the root-mean-square error of correct matches for 3MRS, SIFT, PSO-SIFT, HAPCG, and RIFT are 1.47, 1.98, 1.79, 2.83, and 2.45 pixels, respectively, revealing that 3MRS got the highest accuracy. Even though the total running time of 3MRS was the longest, the efficiency for obtaining one correct match is the highest considering the most significant number of matches. The source code of 3MRS and the experimental datasets and detailed results are publicly available.

LNIFT: Locally Normalized Image for Rotation Invariant Multimodal Feature Matching

Severe nonlinear radiation distortion (NRD) is the bottleneck problem of multimodal image matching. Although many efforts have been made in the past few years, such as the radiation-variation insensitive feature transform (RIFT) and the histogram of orientated phase congruency (HOPC), almost all these methods are based on frequency-domain information that suffers from high computational overhead and memory footprint. In this article, we propose a simple but very effective multimodal feature matching algorithm in the spatial domain, called locally normalized image feature transform (LNIFT). We first propose a local normalization filter to convert original images into normalized images for feature detection and description, which largely reduces the NRD between multimodal images. We demonstrate that normalized matching pairs have a much larger correlation coefficient than the original ones. We then detect oriented FAST and rotated brief (ORB) keypoints on the normalized images and use an adaptive nonmaximal suppression (ANMS) strategy to improve the distribution of keypoints. We also describe keypoints on the normalized images based on a histogram of oriented gradient (HOG), such as a descriptor. Our LNIFT achieves rotation invariance the same as ORB without any additional computational overhead. Thus, LNIFT can be performed in near real-time on images with 1024 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 1024$ </tex-math></inline-formula> pixels (only costs 0.32 s with 2500 keypoints). Four multimodal image datasets with a total of 4000 matching pairs are used for comprehensive evaluations, including synthetic aperture radar (SAR)–optical, infrared–optical, and depth–optical datasets. Experimental results show that LNIFT is far superior to RIFT in terms of efficiency (0.49 s versus 47.8 s on a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1024 \times 1024$ </tex-math></inline-formula> image), success rate (99.9% versus 79.85%), and number of correct matches (309 versus 119). The source code and datasets will be publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://ljy-rs.github.io/web</uri> .

A Novel Multiscale Adaptive Binning Phase Congruency Feature for SAR and Optical Image Registration

Automatic registration of synthetic aperture radar (SAR) and optical images is still a challenging problem because of the potential differences in geometric and intensity. In this work, we propose a robust and efficient method for improving the registration performance of SAR and optical images. Our work consists mainly of two steps, including the feature point detection stage and the feature description stage. In the first stage, we present a new feature extraction method, named nonlinear diffusion-based Harris-Laplace (NDHL) detector, which incorporates the nonlinear diffusion and a spatial consistency strategy into the Harris-Laplace detector, which can weaken the modality variations and enhance the structural features of the multimodal images, and can detect many more similar and highly repeatable feature points. In the second stage, we design a novel structural descriptor, named multiscale adaptive binning phase congruency (MABPC). The proposed MABPC descriptor encodes multiscale phase congruency features with an adaptive binning spatial structure, which brings an improvement of the robustness against geometric and nonlinear intensity discrepancies. Experimental results on both simulated and real image pairs show that the proposed registration method achieves encouraging performance improvements over other state-of-the-art methods. In comparison with the ROS-PC and RIFT methods, the proposed method obtains an average 54.6% improvement for the number of correct matches and has an average registration precision of 1.38 pixels.