Synthetic Aperture Radar Image Patches Research Articles

A Wavelet-Driven Subspace Basis Learning Network for High-Resolution Synthetic Aperture Radar Image Classification

The feature learning strategy of convolutional neural networks (CNNs) learns the deep spatial features from high-resolution (HR) synthetic aperture radar (SAR) images while ignoring the speckle noise based on SAR imaging mechanism. In the feature learning module, the noise reduction by feature-adaptive projection guided by a powerful embedded wavelet feature reconstruction mechanism can effectively learn the deep feature statistics. In this paper, we present a Wavelet Driven Subspace Basis Learning Network (WDSBLN), following an encoder-decoder architecture, for HR SAR image classification. The powerful wavelet module (PWM), including wavelet decomposition and reconstruction, is employed for keeping the structures of learned features well under speckle noise. Specifically, a compact second-order feature enhancement mechanism is designed for improving the contour and edge information of low-frequency components in the feature decomposition stage, and a local feature attention module based on point-wise convolutional layer is adopted to aggregate the contextual information of local channel and reserves detail information in the high-frequency components. Then the reconstructed feature map is employed as a guided standard in the subspace basis learning (SBL) module. The SBL module, including basis generation (generating the subspace basis vectors) and subspace projection (transforming deep feature maps into a signal subspace), maintains the local structure of HR SAR image patches and acquires the robust feature statistics. We conduct evaluations on three real HR SAR image classification datasets, achieve superior performances as compared to other related networks.

PM-Net: A Multi-Level Keypoints Detector and Patch Feature Learning Network for Optical and SAR Image Matching

Due to the differences in radiation and geometric characteristics of optical and synthetic aperture radar (SAR) images, there is still a huge challenge for accurate matching. In this paper, we propose a patch-matching network (PM-Net) to improve the matching performance of optical and SAR images. First, a multi-level keypoints detector (MKD) with fused high-level and low-level features is presented to extract more robust keypoints from optical and SAR images. Second, we use a two-channel network structure to improve the image patch matching performance. Benefiting from this design, the proposed method can directly learn the similarity between optical and SAR image patches without manually designing features and descriptors. Finally, the MKD and two-channel net-work are trained separately on GL3D and QXS-SAROPT data sets, and the PM-Net is tested on multiple pairs of optical and SAR images. The experimental results demonstrate that the proposed method outperforms four advanced image matching networks on qualitative and quantitative assessments. The quantitative experiment results show that using our method correct matching points numbers are increased by more than 1.15 times, the value of F1-measure is raised by an average of 7.4% and the root mean squared error (RMSE) is reduced by more than 15.3%. The advantages of MKD and the two-channel network are also verified through ablation experiments.

Fast texture classification of denoised SAR image patches using GLCM on Spark

Classification of a synthetic aperture radar (SAR) image is an essential process for SAR image analysis and interpretation. Recent advances in imaging technologies have allowed data sizes to grow, and a large number of applications in many areas have been generated. However, analysis of high-resolution SAR images, such as classification, is a time-consuming process and high-speed algorithms are needed. In this study, classification of high-speed denoised SAR image patches by using Apache Spark clustering framework is presented. Spark is preferred due to its powerful open-source cluster-computing framework with fast, easy-to-use, and in-memory analytics. Classification of SAR images is realized on patch level by using the supervised learning algorithms embedded in the Spark machine learning library. The feature vectors used as the classifier input are obtained using gray-level cooccurrence matrix which is chosen to quantitatively evaluate textural parameters and representations. SAR image patches used to construct the feature vectors are first applied to the noise reduction algorithm to obtain a more accurate classification accuracy. Experimental studies were carried out using naive Bayes, decision tree, and random forest algorithms to provide comparative results, and significant accuracies were achieved. The results were also compared with a state-of-the-art deep learning method. TerraSAR-X images of high-resolution real-world SAR images were used as data.

Matching of TerraSAR-X derived ground control points to optical image patches using deep learning

High resolution synthetic aperture radar (SAR) satellites like TerraSAR-X are capable of acquiring images exhibiting an absolute geolocation accuracy within a few centimeters, mainly because of the availability of precise orbit information and by compensating range delay errors due to atmospheric conditions. In contrast, satellite images from optical missions generally exhibit comparably low geolocation accuracies because of the propagation of errors in angular measurements over large distances. However, a variety of remote sensing applications, such as change detection, surface movement monitoring or ice flow measurements, require precisely geo-referenced and co-registered satellite images. By using Ground Control Points (GCPs) derived from TerraSAR-X, the absolute geolocation accuracy of optical satellite images can be improved. For this purpose, the corresponding matching points in the optical images need to be localized. In this paper, a deep learning based approach is investigated for an automated matching of SAR-derived GCPs to optical image elements. Therefore, a convolutional neural network is pretrained with medium resolution Sentinel-1 and Sentinel-2 imagery and fine-tuned on precisely co-registered TerraSAR-X and Pléiades training image pairs to learn a common descriptor representation. By using these descriptors, the similarity of SAR and optical image patches can be calculated. This similarity metric is then used in a sliding window approach to identify the matching points in the optical reference image. Subsequently, the derived points can be utilized for co-registration of the underlying images. The network is evaluated over nine study areas showing airports and their rural surroundings from several different countries around the world. The results show that based on TerraSAR-X-derived GCPs, corresponding points in the optical image can automatically and reliably be identified with a pixel-level localization accuracy.

A SEMI-SUPERVISED APPROACH TO SAR-OPTICAL IMAGE MATCHING

Abstract. Matching synthetic aperture radar (SAR) and optical remote sensing imagery is a key first step towards exploiting the complementary nature of these data in data fusion frameworks. While numerous signal-based approaches to matching have been proposed, they often fail to perform well in multi-sensor situations. In recent years deep learning has become the go-to approach for solving image matching in computer vision applications, and has also been adapted to the case of SAR-optical image matching. However, the hitherto proposed techniques still fail to match SAR and optical imagery in a generalizable manner. These limitations are largely due to the complexities in creating large-scale datasets of corresponding SAR and optical image patches. In this paper we frame the matching problem within semi-supervised learning, and use this as a proxy for investigating the effects of data scarcity on matching. In doing so we make an initial contribution towards the use of semi-supervised learning for matching SAR and optical imagery. We further gain insight into the non-complementary nature of commonly used supervised and unsupervised loss functions, as well as dataset size requirements for semi-supervised matching.

Mining Hard Negative Samples for SAR-Optical Image Matching Using Generative Adversarial Networks

In this paper, we propose a generative framework to produce similar yet novel samples for a specified image. We then propose the use of these images as hard-negatives samples, within the framework of hard-negative mining, in order to improve the performance of classification networks in applications which suffer from sparse labelled training data. Our approach makes use of a variational autoencoder (VAE) which is trained in an adversarial manner in order to learn a latent distribution of the training data, as well as to be able to generate realistic, high quality image patches. We evaluate our proposed generative approach to hard-negative mining on a synthetic aperture radar (SAR) and optical image matching task. Using an existing SAR-optical matching network as the basis for our investigation, we compare the performance of the matching network trained using our approach to the baseline method, as well as to two other hard-negative mining methods. Our proposed generative architecture is able to generate realistic, very high resolution (VHR) SAR image patches which are almost indistinguishable from real imagery. Furthermore, using the patches as hard-negative samples, we are able to improve the overall accuracy, and significantly decrease the false positive rate of the SAR-optical matching task—thus validating our generative hard-negative mining approaches’ applicability to improve training in data sparse applications.

The Generalized Gamma-DBN for High-Resolution SAR Image Classification

With the increase of resolution, effective characterization of synthetic aperture radar (SAR) image becomes one of the most critical problems in many earth observation applications. Inspired by deep learning and probability mixture models, a generalized Gamma deep belief network (g Γ-DBN) is proposed for SAR image statistical modeling and land-cover classification in this work. Specifically, a generalized Gamma-Bernoulli restricted Boltzmann machine (gΓB-RBM) is proposed to capture high-order statistical characterizes from SAR images after introducing the generalized Gamma distribution. After stacking the g Γ B-RBM and several standard binary RBMs in a hierarchical manner, a gΓ-DBN is constructed to learn high-level representation of different SAR land-covers. Finally, a discriminative neural network is constructed by adding an additional predict layer for different land-covers over the constructed deep structure. Performance of the proposed approach is evaluated via several experiments on some high-resolution SAR image patch sets and two large-scale scenes which are captured by ALOS PALSAR-2 and COSMO-SkyMed satellites respectively.

Exploring the Potential of Conditional Adversarial Networks for Optical and SAR Image Matching

Tasks such as the monitoring of natural disasters or the detection of change highly benefit from complementary information about an area or a specific object of interest. The required information is provided by fusing high accurate coregistered and georeferenced datasets. Aligned high-resolution optical and synthetic aperture radar (SAR) data additionally enable an absolute geolocation accuracy improvement of the optical images by extracting accurate and reliable ground control points (GCPs) from the SAR images. In this paper, we investigate the applicability of a deep learning based matching concept for the generation of precise and accurate GCPs from SAR satellite images by matching optical and SAR images. To this end, conditional generative adversarial networks (cGANs) are trained to generate SAR-like image patches from optical images. For training and testing, optical and SAR image patches are extracted from TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe. The artificially generated patches are then used to improve the conditions for three known matching approaches based on normalized cross-correlation (NCC), scale-invariant feature transform (SIFT), and binary robust invariant scalable key (BRISK), which are normally not usable for the matching of optical and SAR images. The results validate that a NCC-, SIFT-, and BRISK-based matching greatly benefit, in terms of matching accuracy and precision, from the use of the artificial templates. The comparison with two state-of-the-art optical and SAR matching approaches shows the potential of the proposed method but also revealed some challenges and the necessity for further developments.

Analytical Modeling of Rough Surface SAR Images Under Small Perturbation Approximation

In high-resolution synthetic aperture radar (SAR) images, the spatial pattern of a homogenous region conveys rich information regarding the specific scatterers being imaged. Analytical modelings of both scattering and imaging processes are critical to better interpret a specific scatterer’s appearances in SAR imagery and thereafter quantitatively retrieve its physical parameters. Small-scale rough surface scattering represents one of the mostly common scattering mechanisms and yet its SAR image characteristics have not been well studied. This becomes more critical in millimeter-wave/terahertz regime as smooth surfaces would become slightly rough under millimeter/submillimeter wavelengths. In this paper, we recast the small perturbation method (SPM) approximation of rough surface scattering under the deterministic finite-length surface condition. By ignoring evanescent waves, a simplified SPM solution for rough facet is derived as well as the analytical form of its SAR image under the conventional setup. Then, we reformulate the scattering imaging process of rough facet as a signal processing chain that further reveals the underlying mechanism of rough surface as imaged by radar. The proposed method is extensively validated against the method of moments in terms of both scattering coefficients and imaging patterns. It is found that under the conventional SPM validity condition, the error of scattering coefficients is less than 1.5 dB, while the correlation between SPM-calculated and MoM-calcualted SAR images is larger than 0.9. The proposed model of rough facet SAR image could serve as the theoretical basis for parameter inversion and surface reconstruction. Preliminary cases of the inversion of rough surface spectrum using both simulated and real SAR image patches are presented.

Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images

Improving the geo-localization of optical satellite images is an important pre-processing step for many remote sensing tasks like monitoring by image time series or scene analysis after sudden events. These tasks require geo-referenced and precisely co-registered multi-sensor data. Images captured by the high resolution synthetic aperture radar (SAR) satellite TerraSAR-X exhibit an absolute geo-location accuracy within a few decimeters. These images represent therefore a reliable source to improve the geo-location accuracy of optical images, which is in the order of tens of meters. In this paper, a deep learning-based approach for the geo-localization accuracy improvement of optical satellite images through SAR reference data is investigated. Image registration between SAR and optical images requires few, but accurate and reliable matching points. These are derived from a Siamese neural network. The network is trained using TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe, in order to learn the two-dimensional spatial shifts between optical and SAR image patches. Results confirm that accurate and reliable matching points can be generated with higher matching accuracy and precision with respect to state-of-the-art approaches.

SAR Image Content Retrieval Based on Fuzzy Similarity and Relevance Feedback

This paper presents a new content-based synthetic aperture radar (SAR) image retrieval method to search out SAR image patches, which consists of two essential parts: an initial retrieval and later refined results. To obtain the proper initial retrievals, we develop a similarity measure named region-based fuzzy matching (RFM) to evaluate the similarities between SAR image patches. First, to reduce the negative influence of speckle noise, we segment the SAR image patches into brightness-texture regions at the superpixel level rather than the pixel level. Second, a multiscale edge detector is utilized to resolve the multiscale property of the SAR image patches, and then the edge regions of the SAR image patches are defined by those edge features. Third, to overcome the segmented uncertainty and the blurry boundaries, the obtained regions are described by fuzzy features. Finally, the RFM similarity between two SAR image patches is converted into the linear summation of the resemblance between different fuzzy feature sets. After we obtain the initial retrieval results, the multiple relevance feedback (MRF) scheme is proposed to refine the original ranked list. In this scheme, different relevance feedback approaches are carried out respectively, and then their results are fused to improve the initial retrieval. In addition, a new kernel function based on the RFM measure is developed for MRF. The encouraging experimental results counted on a manually constructed ground truth SAR image patch dataset demonstrate that our retrieval method is effective for SAR images compared with some existing approaches proposed in the remote sensing community

Semisupervised Discriminant Feature Learning for SAR Image Category via Sparse Ensemble

Terrain scene classification plays an important role in various synthetic aperture radar (SAR) image understanding and interpretation. This paper presents a novel approach to characterize SAR image content by addressing category with a limited number of labeled samples. In the proposed approach, each SAR image patch is characterize by a discriminant feature which is generated in a semisupervised manner by utilizing a spare ensemble learning procedure. In particular, a nonnegative sparse coding procedure is applied on the given SAR image patch set to generate the feature descriptors first. The set is combined with a limited number of labeled SAR image patches and an abundant number of unlabeled ones. Then, a semisupervised sampling approach is proposed to construct a set of weak learners, in which each one is modeled by a logistic regression procedure. The discriminant information can be introduced by projecting SAR image patch on each weak learner. Finally, the features of SAR image patches are produced by a sparse ensemble procedure which can reduce the redundancy of multiple weak learners. Experimental results show that the proposed discriminant feature learning approach can achieve a higher classification accuracy than several state-of-the-art approaches.

Discriminant deep belief network for high-resolution SAR image classification

Classification plays an important role in many fields of synthetic aperture radar (SAR) image understanding and interpretation. Many scholars have devoted to design features to characterize the content of SAR images. However, it is still a challenge to design discriminative and robust features for SAR image classification. Recently, the deep learning has attracted much attention and has been successfully applied in many fields of computer vision. In this paper, a novel feature learning approach that is called discriminant deep belief network (DisDBN) is proposed to learning high-level features for SAR image classification, in which the discriminant features are learned by combining ensemble learning with a deep belief network in an unsupervised manner. Firstly, some subsets of SAR image patches are selected and marked with pseudo-labels to train weak classifiers. Secondly, the specific SAR image patch is characterized by a set of projection vectors that are obtained by projecting the SAR image patch onto each weak decision space spanned by each weak classifier. Finally, the discriminant features are generated by feeding the projection vectors to a DBN for SAR image classification. Experimental results demonstrate that better classification performance can be achieved by the proposed approach than the other state-of-the-art approaches.

Change detection in synthetic aperture radar images based on non-local means with ratio similarity measurement

Image change detection is of widespread interest due to a large number of applications in diverse disciplines. In this study, a novel change detection approach for synthetic aperture radar (SAR) images based on a non-local means algorithm is proposed. A non-local means technique is introduced to generate a difference image by using complete information from a pair of observed images. To take the characteristics of SAR images into account, a new ratio-based relativity measurement between two speckled SAR image patches based on a ratio distance is proposed. Theoretical analysis indicates that the ratio distance is valid for SAR images. The probability density function of the ratio distance is deduced to map the distance into a relativity value. Furthermore, the ratio distance and the probability density function are both parameter-free. The new non-local means technique is successfully applied to extend the classical mean-ratio detector for SAR image detection. Experimental results on real SAR images show that the proposed approach is robust to speckle noise and effective for the detection of change information between multitemporal SAR images.

Ratio-Detector-Based Feature Extraction for Very High Resolution SAR Image Patch Indexing

With the advent of very high resolution (VHR) synthetic aperture radar (SAR) images, local content description is becoming a critical issue for indexing. Conventional SAR image analysis techniques, like segmentation and pixel-level classification, are likely to fail as high-level semantic description should be considered for better discrimination. Therefore, we propose to use image-patch-based analysis method for SAR image interpretation. Inspired by ratio edge detector, in this letter, a new feature extraction method represented by the mean ratios in different directions is proposed for VHR SAR image content characterization. Based on the mean ratio, two simple yet powerful and robust features are proposed for SAR image patch indexing. One is the bag-of-word model using not only the basic statistics, i.e., local mean and variance, but also the mean ratios in different directions. The second one is an adaptation of the Weber local descriptor to SAR images by substituting the gradient with the ratio of mean differences in vertical and horizontal directions. To evaluate the proposed features, image patch indexing based on active learning using a SAR image database consisting of high-resolution TerraSAR-X patches is performed. Comparison with the state-of-the-art features, particularly texture features, has shown improved performance for SAR image categorization.