Synthetic Aperture Radar Image Matching Research Articles

A lightweight deep convolutional network with inverted residuals for matching optical and SAR images

ABSTRACT Matching optical and synthetic aperture radar (SAR) images is often challenged by intricate geometric distortion and nonlinear radiation differences, leading to insufficient and unevenly distributed corresponding points. To tackle this issue, we propose a lightweight deep convolutional network with inverted residuals for optical and SAR image matching. Initially, a fully convolutional neural network (FCNN) is designed to extract high-level and semantic features, robustly capturing universal characteristics between optical and SAR images, adept at handling geometric distortion and nonlinear radiation changes. Notably, we integrate a lightweight architecture with inverted residuals into FCNN to adeptly extract local and global contextual information, facilitating feature reuse and minimizing the loss of crucial features. Additionally, a vector-refined module is deployed to refine dense features, filtering out redundant information. Subsequently, a coarse-to-fine strategy is employed to further eliminate gross errors or incorrect matches. Finally, we evaluate the performance of the proposed network in optical and SAR image matching against manually-designed methods and state-of-the-art deep learning techniques. Experimental results demonstrate that our network significantly surpasses existing methods in terms of the number of correct matches and matching accuracy. Specifically, our proposed network achieves at least a 2.8 times increase in correct matches and an 18% improvement in matching accuracy compared to existing methods.

Airborne Platform Three-Dimensional Positioning Method Based on Interferometric Synthetic Aperture Radar Interferogram Matching

As the demand for precise navigation of aircraft increases in modern society, researching high-precision, high-autonomy navigation systems is both theoretically valuable and practically significant. Because the inertial navigation system (INS) has systematic and random errors, its output information diverges. Therefore, it is necessary to combine them with other navigation systems for real-time compensation and correction of these errors. The SAR matching positioning and navigation system uses synthetic aperture radar (SAR) image matching for platform positioning and compensates for the drift caused by errors in the inertial measurement unit (IMU). Images obtained by SAR are matched with digital landmark data, and the platform’s position is calculated based on the SAR imaging geometry. However, SAR matching positioning faces challenges due to seasonal variations in SAR images, the need for typical landmarks for matching, and the lack of elevation information in two-dimensional SAR image matching. This paper proposes an airborne platform positioning method based on interferometric SAR (InSAR) interferogram matching. InSAR interferograms contain terrain elevation information, are less affected by seasonal changes, and provide higher positioning accuracy and robustness. By matching real-time InSAR-processed interferograms with simulated interferograms using a digital elevation model (DEM), three-dimensional position information about the matching points has been obtained. Subsequently, a three-dimensional positioning model for the platform has bene established using the unit line-of-sight vector decomposition method. In actual flight experiments using an FMCW Ku-band Interferometric SAR system, the proposed platform positioning framework demonstrated its ability to achieve precise positioning in the absence of signals from the global navigation satellite system (GNSS).

A Gaussian filter-based window for efficient SAR image matching

ABSTRACT Repeatability of the extracted features and the distinctiveness of the descriptors have been the critical factors in synthetic aperture radar (SAR) image matching. These factors are significantly affected by the presence of multiplicative speckle noise in SAR images. In order to address these issues, a Gaussian filter-based SAR image matching scheme is proposed in this paper. At first, the scale-invariant feature transform is combined with the Gaussian filter-based ratio operator for the extraction of more repeatable features from the input images. Then, the descriptors of the extracted features are constructed by using a Gaussian filter-based window. Finally, the input SAR images are matched through a feature-matching operation. The proposed method can obtain better repeatability and more correct matches than the state-of-the-art methods.

A Robust and High-Precision Three-Step Positioning Method for an Airborne SAR Platform

When airborne synthetic aperture radar (SAR) encounters long-time global navigation satellite system (GNSS) denial, the system cannot eliminate inertial navigation system (INS) accumulated drift. Platform positioning technology based on SAR image-matching is one of the important auxiliary navigation methods. This paper proposes a three-step positioning method for an airborne SAR platform, which can achieve the robust and high-precision estimation of platform position and velocity. Firstly, the motion model of the airborne SAR platform is established and a nonlinear overdetermined equation set of SAR Range-Doppler based on the ground-control points set obtained by SAR image-matching is constructed. Then, to overcome the ill-conditioned problem generated by the singular Jacobian matrix when solving the equations directly, a three-step robust and high-precision estimation of platform position and velocity is achieved through singular value decomposition and equation decoupling. Furthermore, the error transfer model of systematic and random platform positioning errors is derived. Finally, a set of semi-physical simulation experiments of airborne SAR is conducted to verify the effectiveness of the positioning method and the accuracy of the error model presented in this paper.

A Dual-Generator Translation Network Fusing Texture and Structure Features for SAR and Optical Image Matching

The matching problem for heterologous remote sensing images can be simplified to the matching problem for pseudo homologous remote sensing images via image translation to improve the matching performance. Among such applications, the translation of synthetic aperture radar (SAR) and optical images is the current focus of research. However, the existing methods for SAR-to-optical translation have two main drawbacks. First, single generators usually sacrifice either structure or texture features to balance the model performance and complexity, which often results in textural or structural distortion; second, due to large nonlinear radiation distortions (NRDs) in SAR images, there are still visual differences between the pseudo-optical images generated by current generative adversarial networks (GANs) and real optical images. Therefore, we propose a dual-generator translation network for fusing structure and texture features. On the one hand, the proposed network has dual generators, a texture generator, and a structure generator, with good cross-coupling to obtain high-accuracy structure and texture features; on the other hand, frequency-domain and spatial-domain loss functions are introduced to reduce the differences between pseudo-optical images and real optical images. Extensive quantitative and qualitative experiments show that our method achieves state-of-the-art performance on publicly available optical and SAR datasets. Our method improves the peak signal-to-noise ratio (PSNR) by 21.0%, the chromatic feature similarity (FSIMc) by 6.9%, and the structural similarity (SSIM) by 161.7% in terms of the average metric values on all test images compared with the next best results. In addition, we present a before-and-after translation comparison experiment to show that our method improves the average keypoint repeatability by approximately 111.7% and the matching accuracy by approximately 5.25%.

PM-Net: A Multi-Level Keypoints Detector and Patch Feature Learning Network for Optical and SAR Image Matching

Due to the differences in radiation and geometric characteristics of optical and synthetic aperture radar (SAR) images, there is still a huge challenge for accurate matching. In this paper, we propose a patch-matching network (PM-Net) to improve the matching performance of optical and SAR images. First, a multi-level keypoints detector (MKD) with fused high-level and low-level features is presented to extract more robust keypoints from optical and SAR images. Second, we use a two-channel network structure to improve the image patch matching performance. Benefiting from this design, the proposed method can directly learn the similarity between optical and SAR image patches without manually designing features and descriptors. Finally, the MKD and two-channel net-work are trained separately on GL3D and QXS-SAROPT data sets, and the PM-Net is tested on multiple pairs of optical and SAR images. The experimental results demonstrate that the proposed method outperforms four advanced image matching networks on qualitative and quantitative assessments. The quantitative experiment results show that using our method correct matching points numbers are increased by more than 1.15 times, the value of F1-measure is raised by an average of 7.4% and the root mean squared error (RMSE) is reduced by more than 15.3%. The advantages of MKD and the two-channel network are also verified through ablation experiments.

SAR Image Matching Based on Local Feature Detection and Description Using Convolutional Neural Network

Feature detection is a vital step for the image registration process whose target is the misalignment correction among images to increase the convergency level. Deep learning (DL) in remote sensing has become a worldwide sensation. Despite its huge potential, DL has not reached its intended target concerning the applications of Synthetic Aperture Radar (SAR) images. In this study, we focus on matching SAR images using a Convolutional Neural Network. The big challenge in this study is how to modify a pretrained Visual Geometry Group model based on the multispectral dataset to act as a SAR image feature detector where it does not require any prior knowledge about the nature of the SAR feature. Since SAR images have different characteristics from optical images such as SAR dynamic range and imaging geometry, some problems arise and should be considered during the matching process. Despite all these difficulties, results demonstrate the robustness of the registration process where it can provide descriptors that preserve the localization data of features. Also, the proposed approach provides reasonable results compared to the state-of-the-art methods and outperforms the correlation approach and ORB descriptor under scaling. In addition, it may be considered an end-to-end image matching tool for SAR images, although the calculations of fine matching parameters are included.

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

To fully explore the complementary information from optical and synthetic aperture radar (SAR) imageries, they need first to be coregistered with high accuracy. Due to the vast radiometric and geometric disparity, the problem to match high-resolution optical and SAR images is quite challenging. The present deep learning-based methods have shown advantages over the traditional approaches, but the performance increment is not significant. In this article, we explore a better network framework for high-resolution optical and SAR image matching from three aspects. First, we propose an effective multilevel feature fusion method, which helps to take advantage of both the low-level fine-grained features for precious feature location and the high-level semantic features for better discriminative ability. Second, a feature channel excitation procedure is conducted using a novel multifrequency channel attention module, which is able to make image features of different types and multiple levels effectively collaborate with each other and produce image matching features with high diversity. Third, the self-adaptive weighting loss is introduced, with which, each sample is assigned with an adaptive weighting factor, and therefore, information buried in all nearby samples can be better exploited. Under a pseudo-Siamese architecture, the proposed optical and SAR image matching network (OSMNet) is trained and tested on a large and diverse high-resolution optical and SAR dataset. Extensive experiments demonstrate that each component of the proposed deep framework helps to improve the matching accuracy. Also, the OSMNet shows overwhelming superior to the state-of-the-art handcrafted approaches on imageries of different land-cover types.

Phase Congruency Order-Based Local Structural Feature for SAR and Optical Image Matching

Automatic matching of synthetic aperture radar (SAR) and optical images is a fundamental task in many remote sensing applications. However, due to different imaging modalities, conventional matching methods provide limited performances. In this letter, based on the observation that structural features are maintained across different modality images, we propose a novel feature-based method to effectively address SAR and optical image matching. The proposed method is built on the phase congruency (PC) model and consists mainly of two stages. First, a modified version of the uniform nonlinear diffusion-based Harris (MUND-Harris) detector is introduced to extract the local features. Unlike the UND-Harris, MUND-Harris employs the PC instead of image intensity for feature extraction and thus obtain well-distributed and highly repeatable feature points. Second, a local structural descriptor, namely phase congruency order-based local structural (PCOLS), is designed for the extracted points. PCOLS is constructed in a grouping manner and further encodes image structures with an adaptive descriptor structure, which provides robustness against modality variations including significant geometric and intensity differences. Experiment results obtained on several SAR and optical image pairs demonstrate the encouraging performance of the proposed method.

A Novel Loss Function for Optical and SAR Image Matching: Balanced Positive and Negative Samples

Image matching is a primary technology for optical and synthetic aperture radar (SAR) image fusion but often shows limited performance due to the highly nonlinear differences between optical and SAR modalities. Recently, deep neural networks (DNNs) have been investigated to effectively extract nonlinear features for image matching tasks, where DNNs are trained based on the elaborated design of loss functions and a low loss value is often expected to obtain better image matching performance. In this letter, we first theoretically demonstrate that when the value of a state-of-the-art loss function decreases, the corresponding matching performance may not consistently improve due to the imbalanced effect of positive and negative samples. To tackle this issue, we proposed an improved loss function to train DNNs for image matching of SAR and optical images. We theoretically prove that the improved loss function ensures the improvement of the matching performance when the loss value decreases based on Taylor’s series expansion analysis. Experimental results on an open dataset with extensive optical and SAR image pairs show that 1) the proposed loss function is better than the original one in terms of image matching performance and 2) the combination of our loss function and existing multiscale convolutional gradient feature (MCGF)-based network provides better matching performance than other state-of-art approaches.

A SINS/SAR/GPS Fusion Positioning System Based on Sensor Credibility Evaluations

A reliable framework for SINS/SAR/GPS integrated positioning systems is proposed for the case that sensors are in critical environments. Credibility is used to describe the difference between the true error and the initial setting standard deviation. Credibility evaluation methods for inertial measurement unit (IMU), synthetic aperture radar (SAR), and global positioning system (GPS) are presented. In particular, IMU credibility is modeled by noises and constant drifts that are accumulated with time in a strapdown inertial navigation system (SINS). The quality of the SAR image decides the credibility of positioning based on SAR image matching. In addition, a cumulative residual chi-square test is used to evaluate GPS credibility. An extended Kalman filter based on a sensor credibility evaluation is introduced to integrate the measurements. The measurement of a sensor is either discarded when its credibility value is below a threshold or the variance matrix for the estimated state is otherwise adjusted. Simulations show that the final fusion positioning accuracy with credibility evaluation can be improved by 1–2 times compared to that without evaluation. In addition, the derived standard deviation correctly indicates the value of the position error with credibility evaluation. Moreover, the experiments on an unmanned ground vehicle partially verify the proposed evaluation method of GPS and the fusion framework in the actual environment.

Optimal texture image reconstruction method for improvement of SAR image matching

Image matching is an important step that is taken in most synthetic aperture radar (SAR) images applications. In this study, a method is proposed for texture image reconstruction to improve SAR image matching. This method consists of three major steps. First, textural features are extracted. Second optimal texture images (OTIs) are created using extracted features. Texture optimisation is conducted using a cat swarm optimisation algorithm. The cost function is the number of correct matched points that are obtained from speeded-up robust features (SURFs) image matching algorithm. Finally, corresponding points on stereo OTI pairs are matched. Experiments were conducted on spaceborne SAR image pairs consist of RADARSAT-2, ALOS-PALSAR, and Sentinel-1. Results demonstrate that the proposed method has better results in terms of the number of matched points compared to the result of the original SURF and scale-invariant feature transform methods as two common matching methods.

A Modified Local Binary Pattern Descriptor for SAR Image Matching

Image matching is an important step which is taken in most applications of synthetic aperture radar (SAR) images. In this letter, a method is proposed for SAR image matching which introduces a modified local binary pattern (LBP) as a descriptor. Multitextural feature LBP (MTF-LBP) uses the gray-level cooccurrence matrix to increase image texture information. MTF-LBP creates bit plane for each point candidate for matching. Then, using hamming distance, true matches are determined. Experiments are conducted on four spaceborne SAR image pairs including Radarsat-2, TerraSAR-X, ALOS-PALSAR, and Sentinel-1. The proposed method is compared with five common LBP approaches. The results indicate that the proposed method has a better performance in terms of the number of true matches.

A Gabor Odd Filter-Based Ratio Operator for SAR Image Matching

Synthetic aperture radar (SAR) image matching is a challenging issue in remote sensing as the images contain significant multiplicative speckle noise. Histogram of oriented gradients (HOG)-based descriptors have been used popularly for the matching of the SAR images. An appropriate gradient computation of the SAR images plays a significant role in the matching performance of these HOG-based descriptors. The ratio operators are preferred for computing the gradients in the SAR images as these are usually corrupted by the multiplicative noise. Recently, a multiscale Gabor odd filter-based ratio operator (GOFRO) has been proposed for the edge detection of the SAR images. In this letter, we utilize the GOFRO for SAR image matching. We propose a Gabor odd filter (GOF)-based descriptor, where the gradients are computed by the GOFRO. At first, scale-invariant feature transform (SIFT) features are extracted from the SAR images. A novel approach is presented to select the more repeatable SIFT features in SAR images. Then, the proposed GOF-based descriptor is formed for the SIFT features and, finally, feature matching is performed. Experiments on three sets of simulated and real SAR image pairs demonstrate the effectiveness of the proposed GOF-based descriptor for SAR image matching.

Efficient method for outlier removal in SAR image matching based on epipolar geometry

A method is proposed for removing falsely matched points after sparse matching in synthetic aperture radar (SAR) images. This method uses epipolar geometry as well as random sample consensus (RANSAC) to automatically remove false matches (outliers) among all points. The method randomly selects at least four points among all matched points. Then, using a bilinear equation, a generic relation is established between the pixel coordinates of a reference point in the master image and the parameters of the corresponding epipolar line in the slave image. Therefore, it is possible to calculate the epipolar line for all points using the bilinear function and then evaluate the accuracy of the matched points. Moreover, the proposed method is compared with the method that uses the RANSAC with an affine function, which is a common method for removing outliers. Here, the output of the scale-invariant feature transform algorithm is used to conduct the experiments. Furthermore, the SAR image pairs from southern Iran are utilised to evaluate the performance of the method. Experiments are conducted on TerraSAR-X images, and the results demonstrate that the proposed method has high reliability for removing false matches.

Exploring the Potential of Conditional Adversarial Networks for Optical and SAR Image Matching

Tasks such as the monitoring of natural disasters or the detection of change highly benefit from complementary information about an area or a specific object of interest. The required information is provided by fusing high accurate coregistered and georeferenced datasets. Aligned high-resolution optical and synthetic aperture radar (SAR) data additionally enable an absolute geolocation accuracy improvement of the optical images by extracting accurate and reliable ground control points (GCPs) from the SAR images. In this paper, we investigate the applicability of a deep learning based matching concept for the generation of precise and accurate GCPs from SAR satellite images by matching optical and SAR images. To this end, conditional generative adversarial networks (cGANs) are trained to generate SAR-like image patches from optical images. For training and testing, optical and SAR image patches are extracted from TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe. The artificially generated patches are then used to improve the conditions for three known matching approaches based on normalized cross-correlation (NCC), scale-invariant feature transform (SIFT), and binary robust invariant scalable key (BRISK), which are normally not usable for the matching of optical and SAR images. The results validate that a NCC-, SIFT-, and BRISK-based matching greatly benefit, in terms of matching accuracy and precision, from the use of the artificial templates. The comparison with two state-of-the-art optical and SAR matching approaches shows the potential of the proposed method but also revealed some challenges and the necessity for further developments.

An Improved Multi-Image Matching Method in Stereo-Radargrammetry

The new generation of synthetic aperture radar (SAR) sensors provides us with an opportunity to match multiple high-resolution SAR images. Moreover, the multiple SAR image matching methods have recently gained a lot of attention due to the fact that they can obtain more accurate, better distributed, and more reliable matches than the stereo matching methods. In this letter, we present an improved multi-image matching method to simultaneously identify matches from multiple SAR amplitude images. The proposed method makes better use of the relationships between the pixels in the deformed correlation window and integrates geometric and radiometric information from multiple SAR images. Experiments on Chinese Academy of Surveying and Mapping Synthetic Aperture Radar (CASMSAR) data sets demonstrate that the improved multi-image matching method is capable of providing more accurate and better distributed matches, as well as offering a better multi-image matching solution in stereo-radargrammetry under the conditions of geometric and radiometric distortions, especially in low-texture areas.

Similarity measures of full polarimetric SAR images fusion for improved SAR image matching

Abstract. China’s first airborne SAR mapping system (CASMSAR) developed by Chinese Academy of Surveying and Mapping can acquire high-resolution and full polarimetric (HH, HV, VH and VV) Synthetic aperture radar (SAR) data. It has the ability to acquire X-band full polarimetric SAR data at a resolution of 0.5m. However, the existence of speckles which is inherent in SAR imagery affects visual interpretation and image processing badly, and challenges the assumption that conjugate points appear similar to each other in matching processing. In addition, researches show that speckles are multiplicative speckles, and most similarity measures of SAR image matching are sensitive to them. Thus, matching outcomes of SAR images acquired by most similarity measures are not reliable and with bad accuracy. Meanwhile, every polarimetric SAR image has different backscattering information of objects from each other and four polarimetric SAR data contain most basic and a large amount of redundancy information to improve matching. Therefore, we introduced logarithmically transformation and a stereo matching similarity measure into airborne full polarimetric SAR imagery. Firstly, in order to transform the multiplicative speckles into additivity ones and weaken speckles' influence on similarity measure, logarithmically transformation have to be taken to all images. Secondly, to prevent performance degradation of similarity measure caused by speckles, measure must be free or insensitive of additivity speckles. Thus, we introduced a stereo matching similarity measure, called Normalized Cross-Correlation (NCC), into full polarimetric SAR image matching. Thirdly, to take advantage of multi-polarimetric data and preserve the best similarity measure value, four measure values calculated between left and right single polarimetric SAR images are fused as final measure value for matching. The method was tested for matching under CASMSAR data. The results showed that the method delivered an effective performance on experimental imagery and can be used for airborne SAR matching applications.

Enhanced Feature Based Mosaicing Technique for Visually and Geometrically Degraded Airborne Synthetic Aperture Radar Images

In airborne synthetic aperture radar (SAR), there was a major problem encountered in the area of image mosaic in the absence of platform information and sensor information (geocoding), when SAR is applied in large-scale scene and the platform faces large changes. In order to enhance real-time performance and robustness of image mosaic, enhancement based Speeded-Up Robust Features (SURF) mosaic method for airborne SAR is proposed in this paper. SURF is a novel scale-invariant and rotation-invariant feature. It is perfect in its high computation, speed and robustness. In this paper, When the SAR image is acquired, initially the image is enhanced by using local statistic techniques and SURF is applied for SAR image matching accord to its characteristic, and then acquires its invariant feature for matching. In the process of image matching, the nearest neighbor rule for initial matching is used, and the wrong points of the matches are removed through RANSAC fitting algorithm. The proposed algorithm is implemented in different SAR images with difference in scale change, rotation change and noise. The proposed algorithm is compared with other existing algorithms and the quantitative and qualitative measures are calculated and tabulated. The proposed algorithm is robust to changes and the threshold is varied accordingly to increase the matching rate more than 95 %.

A novel method for multi-angle SAR image matching

Multi-angle synthetic aperture radar (SAR) image matching is very challenging, because the same object may cause different backscattering patterns, heavily depending on the radar incident angle. A technique based on the relations between the invariant positions of ground targets among the reference and sensed images is proposed to accommodate the nonmatching patterns. It involves a target extraction using wavelet coefficient fusion, as well as a geometric voting matching routine for searching the matched control points (CPs) in the reference and sensed images, respectively. To accelerate the speed of the search, a robust, rapidly corresponding CPs determination strategy is exploited by utilizing the global spatial transformation model, as well as a procedure of outlier removal to ensure the desired accuracy. Meanwhile, the positions of the matched point pairs are relocated using mutual information. The final warping of the images according to the CPs is performed by using a polynomial function. The results show the possibility of matching multi-angle SAR images in general cases.