Radiometric Distortions Research Articles

Automatic Methodology for Forest Fire Mapping with SuperDove Imagery.

The global increase in wildfires due to climate change highlights the need for accurate wildfire mapping. This study performs a proof of concept on the usefulness of SuperDove imagery for wildfire mapping. To address this topic, we present an automatic methodology that combines the use of various vegetation indices with clustering algorithms (bisecting k-means and k-means) to analyze images before and after fires, with the aim of improving the precision of the burned area and severity assessments. The results demonstrate the potential of using this PlanetScope sensor, showing that the methodology effectively delineates burned areas and classifies them by severity level, in comparison with data from the Copernicus Emergency Management Service (CEMS). Thus, the potential of the SuperDove satellite sensor constellation for fire monitoring is highlighted, despite its limitations regarding radiometric distortion and the absence of Short-Wave Infrared (SWIR) bands, suggesting that the methodology could contribute to better fire management strategies.

Radargrammetric 3D Imaging through Composite Registration Method Using Multi-Aspect Synthetic Aperture Radar Imagery

Interferometric synthetic aperture radar (InSAR) and tomographic SAR measurement techniques are commonly used for the three-dimensional (3D) reconstruction of complex areas, while the effectiveness of these methods relies on the interferometric coherence among SAR images with minimal angular disparities. Radargrammetry exploits stereo image matching to determine the spatial coordinates of corresponding points in two SAR images and acquire their 3D properties. The performance of the image matching process directly impacts the quality of the resulting digital surface model (DSM). However, the presence of speckle noise, along with dissimilar geometric and radiometric distortions, poses considerable challenges in achieving accurate stereo SAR image matching. To address these aforementioned challenges, this paper proposes a radargrammetric method based on the composite registration of multi-aspect SAR images. The proposed method combines coarse registration using scale invariant feature transform (SIFT) with precise registration using normalized cross-correlation (NCC) to achieve accurate registration between multi-aspect SAR images with large disparities. Furthermore, the multi-aspect 3D point clouds are merged using the proposed radargrammetric 3D imaging method, resulting in the 3D imaging of target scenes based on multi-aspect SAR images. For validation purposes, this paper presents a comprehensive 3D reconstruction of the Five-hundred-meter Aperture Spherical radio Telescope (FAST) using Ka-band airborne SAR images. It does not necessitate prior knowledge of the target and is applicable to the detailed 3D imaging of large-scale areas with complex structures. In comparison to other SAR 3D imaging techniques, it reduces the requirements for orbit control and radar system parameters. To sum up, the proposed 3D imaging method with composite registration guarantees imaging efficiency, while enhancing the imaging accuracy of crucial areas with limited data.

Robust registration of multi-modal remote sensing images based on multi-dimensional oriented self-similarity features

Registration of multi-modal remote sensing images (MRSI) is crucial for unlocking the full potential of heterogeneous remote sensing imagery. However, achieving accurate registration among MRSI is challenging due to the trade-off between geometric invariance and matching accuracy, caused by differences in signal-to-noise ratio and nonlinear radiometric distortion (NRD) arising from varying imaging mechanisms. To tackle the challenge, this paper proposes a lightweight and hybrid feature-guided registration algorithm for MRSI called the hybrid registration algorithm based on multi-dimensional oriented self-similarity features (MOSS). MOSS leverages the advantages of multi-dimensional oriented self-similarity features to progressively enhance registration performance. In the hybrid feature coarse matching stage, oriented self-similarity features are extracted from MRSI, and their directional information is utilized for feature description to estimate the initial affine transformation. The fine matching under multi-dimensional oriented self-similarity features stage takes the outputs of the coarse matching stage to perform a template-like matching process. To evaluate the performance of MOSS, comprehensive experiments are conducted using six different combinations of MRSI, and seven state-of-the-art registration algorithms are selected for comparison. The experimental results demonstrate that MOSS outperforms the compared methods, with the number of correct matches being at least about 1.6 times higher than the comparison methods. Moreover, MOSS exhibits the lowest root mean square error across all experiments, with an average RMSE of 1.86 pixels, achieving an RMSE within 2 pixels. This highlights its effectiveness in achieving precise alignment and robust registration of MRSI.

GRiD: Guided Refinement for Detector-Free Multimodal Image Matching.

Multimodal image matching is essential in image stitching, image fusion, change detection, and land cover mapping. However, the severe nonlinear radiometric distortion (NRD) and geometric distortions in multimodal images severely limit the accuracy of multimodal image matching, posing significant challenges to existing methods. Additionally, detector-based methods are prone to feature point offset issues in regions with substantial modal differences, which also hinder the subsequent fine registration and fusion of images. To address these challenges, we propose a guided refinement for detector-free multimodal image matching (GRiD) method, which weakens feature point offset issues by establishing pixel-level correspondences and utilizes reference points to guide and correct matches affected by NRD and geometric distortions. Specifically, we first introduce a detector-free framework to alleviate the feature point offset problem by directly finding corresponding pixels between images. Subsequently, to tackle NRD and geometric distortion in multimodal images, we design a guided correction module that establishes robust reference points (RPs) to guide the search for corresponding pixels in regions with significant modality differences. Moreover, to enhance RPs reliability, we incorporate a phase congruency module during the RPs confirmation stage to concentrate RPs around image edge structures. Finally, we perform finer localization on highly correlated corresponding pixels to obtain the optimized matches. We conduct extensive experiments on four multimodal image datasets to validate the effectiveness of the proposed approach. Experimental results demonstrate that our method can achieve sufficient and robust matches across various modality images and effectively suppress the feature point offset problem.

A method for extracting 3D‐element based on local cost metrics resistant to radiometric distortion for realistic media application

AbstractThe intensity values of pairs of stereoscopic images could be affected by radiometric elements such as camera exposure, illumination direction, etc. Therefore, it is a challenging problem to get consistent and exact disparity values under a variety of real‐world environment. Here, the proposed approach extracts disparity values based on a local cost metric and is resistant to global and local radiometric changes. It is designed in a way that disparity values are extracted using local uniformity of pixel values between pairs of stereoscopic images. The proposed metric‐based method is defined as the resistant local cost metric (RLCM) model. As a result of the experiment, the proposed model performed better than the comparative methods under experimental conditions in which the radiometric distortion between pairs of stereoscopic images was significantly different. The average peak signal‐to‐noise ratio (PSNR) value of the RLCM method is 20.0 dB under the global radiometric change condition and 19.3 dB under the local condition, showing the highest value compared to the comparison group. Consequentially, it is demonstrated that the proposed method is less sensitive to various radiometric changes.

Vegetation Subtype Classification of Evergreen Broad-Leaved Forests in Mountainous Areas Using a Hierarchy-Based Classifier

Evergreen broad-leaved forests with rich biodiversity play a key role in stabilizing global vegetation productivity and maintaining land carbon sinks. However, quantitative and accurate classification results for humid, evergreen, broad-leaved forests (HEBF) and semi-humid evergreen broad-leaved forests (SEBF) with different vegetation productivity and significant differences in species composition are lacking. Remote sensing technology brings the possibility of vegetation subtype classification. Taking the mountainous evergreen broad-leaved forests distributed in Sichuan Province as an example, this study proposed a hierarchy-based classifier combined with environmental variables to quantitatively classify the two vegetation subtypes with different ecological characteristics but similar image features. Additionally, we applied Sun–Canopy–Sensor and C parameter(SCS + C) topographic correction to preprocess the images, effectively correcting the radiometric distortion and enhancing the accuracy of vegetation classification. Finally, achieving an overall accuracy (OA) of 87.91% and a Kappa coefficient of 0.76, which is higher than that of directly using the classifier to classify the two vegetation subtypes. The study revealed the widespread distribution of evergreen broad-leaved forests in Sichuan, with a clear boundary between the distribution areas of HEBF and SEBF. The HEBF in the east is located in the basin and the low marginal mountains; the SEBF is located in the southwest dry valley. The methods employed in this study offer an effective approach to vegetation classification in mountainous areas. The findings can provide guidance for ecological engineering construction, ecological protection, and agricultural and livestock development.

Colour-weighted rank transform and improved dynamic programming for fast and accurate stereo matching

ABSTRACT Real-time stereo matching with high accuracy is a dynamic research topic; it is attractive in diverse computer vision applications. This paper presents a stereo-matching algorithm that produces high-quality disparity map while maintaining real-time performance. The proposed stereo-matching method is based on three per-pixel difference measurements with adjustment elements. The absolute differences and the gradient matching are combined with a colour-weighted extension of complete rank transform to reduce the effect of radiometric distortion. The disparity calculation is realized using improved dynamic programming that optimizes along and across all scanlines. It solves the inter-scanline inconsistency problem and increases the matching accuracy. The proposed algorithm is implemented on parallel high-performance graphic hardware using the Compute Unified Device Architecture to reach over 240 million disparity evaluations per second. The processing speed of our algorithm reaches 98 frames per second on 240 × 320-pixel images and 32 disparity levels. Our method ranks fourth in terms of accuracy and runtime for quarter-resolution images in the Middlebury stereo benchmark.

Robust Cost Volume Generation Method for Dense Stereo Matching in Endoscopic Scenarios.

Stereo matching in binocular endoscopic scenarios is difficult due to the radiometric distortion caused by restricted light conditions. Traditional matching algorithms suffer from poor performance in challenging areas, while deep learning ones are limited by their generalizability and complexity. We introduce a non-deep learning cost volume generation method whose performance is close to a deep learning algorithm, but with far less computation. To deal with the radiometric distortion problem, the initial cost volume is constructed using two radiometric invariant cost metrics, the histogram of gradient angle and amplitude descriptors. Then we propose a new cross-scale propagation framework to improve the matching reliability in small homogenous regions without increasing the running time. The experimental results on the Middlebury Version 3 Benchmark show that the performance of the combination of our method and Local-Expansion, an optimization algorithm, ranks top among non-deep learning algorithms. Other quantitative experimental results on a surgical endoscopic dataset and our binocular endoscope show that the accuracy of the proposed algorithm is at the millimeter level which is comparable to the accuracy of deep learning algorithms. In addition, our method is 65 times faster than its deep learning counterpart in terms of cost volume generation.

SAR Radiometric Calibration Based on Differential Geometry: From Theory to Experimentation on SAOCOM Imagery

The modeling and simulation of topography-induced imaging distortions are crucial for consistent radiometric information exploitation in current and forthcoming SAR-based Earth observation missions with a high spatial and temporal resolution, with relevance in several applications. In this paper, for the first time, we specifically investigate the compensation of topography-induced radiometric distortions affecting SAR images acquired by the L-band Argentinian satellite SAOCOM. We adopt a recently developed calibration method relying on an analytical formulation derived in the rigorous framework of the differential geometry of surfaces. We first provide an original interpretation of the analytical formulation, thus providing further insights into the relevant area-stretching-based formalism. Then, the numerical implementation of the method is specialized to systematically process the data acquired by SAOCOM sensors; hence, the resulting sensor-specific prototype solution processor is employed in this study. Finally, experiments performed over a real scenario in the southern part of Italy, characterized by large topography variations, are presented and discussed, thus elucidating the effectiveness of the adopted method applied to SAOCOM images. The adopted effective SAR calibration strategy opens up the way to its operational use in large-scale SAOCOM data processing.

3D compressive imaging system with a single photon-counting detector.

For photon-counting based compressive imaging systems, it is difficult to obtain 3D image with intensity and depth information precisely due to the dead time and shot noise effect of photon-counting detectors. In this study, we design and achieve a 3D compressive imaging system using a single photon-counting detector. To overcome the radiometric distortion arising from the dead time and shot noise, considering the response mechanism of photon-counting detectors, a Bayesian posterior model is derived and a Reversible jump Markov chain Monte Carlo (RJMCMC)-based method is proposed to iteratively obtain model parameters. Experimental and simulation results indicate that the 3D image of targets can be effectively and accurately reconstructed with a smaller number of repeated illuminations and no longer restricted by the photon flux conditions (i.e., breaking through the upper limit of the received signal level). The proposed Bayesian RJMCMC-based radiometric correction method is not only beneficial to single-photon 3D compressive imaging system, but also to any other photon-counting based systems, e.g., photon-counting lidars. In addition, limiting condition of recovering the actual photon number for photon-counting imaging or lidar systems is also quantitatively analyzed, which is of great significance to the system scheme design.

PERFORMANCE EVALUATION OF LEARNING-BASED METHODS FOR MULTISPECTRAL SATELLITE IMAGE MATCHING

Abstract. Multispectral image registration is one of the most critical requirements to achieve reliable remote sensing goals such as change detection, image fusion, etc., due to providing complementary knowledge of the scene. On the one hand, this issue has always been a hot topic of research according to significant appearance differences, including geometric and nonlinear radiometric distortions. On the other hand, developing deep learning methods promises precise results in image processing and, in particular, image registration. It is no longer limited to low-level information structures, such as intensity and gradients. However, it is possible to provide more reliable results by extracting various high-level features and removing feature engineering. Therefore, we need extensive experiments in multispectral image registration to determine an efficient and robust method. To this end, this paper evaluates six well-known recently proposed learning-based feature descriptors, including LOFTR, TFeat, HardNet8, HardNet, SosNet, and HyNet, against geometric distortions within real multispectral images. Evaluations demonstrate the general superiority of the HardNet8 descriptor due to extracting high-level features within eight convolution layers.

SSS Radiometric Distortion Correction Based on Variational Retinex Framework With Consideration for Sediment Characteristics

SSS Radiometric Distortion Correction Based on Variational Retinex Framework With Consideration for Sediment Characteristics

Radiometric Correction Model and Land Cover Classification of Snow-Covered Terrains for ICESat-2 Photon-Counting Lidar

The signal strength is a fundamental parameter in radiometric applications for satellite lidars. Different from full-waveform lidars, photon-counting lidars cannot record the returned signal strength but only respond to the presence of the photon event and may miss some returned photons due to the dead time effect, i.e., introduce radiometric distortion. Based on the lidar equation and the response mechanism of photon-counting detectors, we propose a radiometric correction model to remove the impact of the nonlinear response and dead time of detectors for the photon-counting lidar borne on ICESat-2. The returned signal photon number is corrected by the proposed model with respect to the photon event number per shot (PNPS) and surface slope derived from ATL03/ATL08 products. Then, the optical throughput calibration factor of ICESat-2 is obtained from ATL06 products over high Antarctic plateau where has given reflectance and clear atmosphere, which is generally equal to 0.52. The atmospheric attenuation induced by the molecular, cloud, and aerosol is calculated from ATL09 products. In addition, the corrected radiometric parameters including the calculated surface reflectance and apparent surface reflectance (ASR) are applied to classify land cover types along laser tracks over snow-covered terrains. The results indicate that the signal strength and calibration constant are reliable after corrections, but the atmospheric attenuation is sometimes inaccurate, which further influences the derived surface reflectance. In classifications, the overall accuracy and Kappa coefficient based on the corrected ASR can achieve the best classification results with 88.80% and 0.69. The proposed radiometric correction model is very essential to radiometric applications for photon-counting lidars such as ICESat-2, especially for data captured on ice and bare land with relatively high reflectance.

Histogram of the orientation of the weighted phase descriptor for multi-modal remote sensing image matching

Multi-modal remote sensing image (MRSI) has nonlinear radiation distortion (NRD) and significant contrast differences to which image gradient features are usually sensitive. Although image phase features are more robust against NRD, they might not be much helpful in resolving the problems of directional inversion or phase extreme value mutations that are common in the phase feature calculation. To address these issues, a new MRSI matching method—“histogram of the orientation of weighted phase” (HOWP)—is proposed in this paper. This method distinguishes itself from other methods in three aspects: (1) a feature aggregation strategy is used to optimize feature points by extracting the corner and blob features separately; (2) a novel weighted phase orientation model is established to replace the traditional image gradient orientation features; and (3) a regularization-based log-polar descriptor is constructed to generate robust feature description vectors. To evaluate the performance of the proposed method, we selected 50 sets of typical MRSIs with translation, scale, and rotation differences for comparison with the other four state-of-the-art methods. The results show that our method is more resistant to radiometric distortion and the contrasting differences in MRSIs. It also performs better in tackling the problems of direction reversal and phase extreme value mutation, as evidenced by more, the number of correct matches (NCM). Since the method has improved the average NCM by 1.6–4.5 times, the average success rate by 35.5%, and the average rate of correct matches by 11.1% with an average root of mean-squared error of 1.93 pixels. Moreover, we have put forward an extended version of the HOWP method (Simplified-HOWP) when there is no image rotation, which manifests in an average 0.75 times improvement in NCM of Simplified-HOWP performance over that of the HOWP method. The executable code and test data are linked in https://skyearth.org/publication/project/HOWP/.

Mountainous SAR Image Registration Using Image Simulation and an L2E Robust Estimator

Synthetic Aperture Radar (SAR) is one of the most widely utilized methods to extract elevation information and identify large-scale deformations in mountainous areas. Homologous points in stereo SAR image pairs are difficult to identify due to complex geometric and radiometric distortions. In this paper, a new approach for mountainous area images is suggested. Firstly, a simulated SAR image and a look-up table based on DEM data are generated by a range-Doppler model and an empirical formula. Then, a point matching RPM-L2E algorithm is used to match images obtained by the simulation and in real-time to indirectly obtain the feature points of the real SAR images. Finally, the accurate registration of mountainous areas in the SAR images is achieved by a polynomial transform. Experimental verification is performed by using the data of mountainous SAR images from the same sensor and different sensors. When the registration accuracy of the method is compared with that of two state-of-the-art image registration algorithms, better outcomes are experimentally shown. The suggested approach can effectively solve the registration problem of SAR images of mountainous areas, and can overcome the disadvantages of poor adaptability and low accuracy of traditional SAR image registration methods for mountainous areas.

Performance Evaluation of Multiple Pan-Sharpening Techniques on NDVI: A Statistical Framework

Pan-sharpening is a pixel-level image fusion process whereby a lower-spatial-resolution multispectral image is merged with a higher-spatial-resolution panchromatic one. One of the drawbacks of this process is that it may introduce spectral or radiometric distortion. The degree to which distortion is introduced is dependent on the imaging sensor, the pan-sharpening algorithm employed, and the context of the scene analyzed. Studies that evaluate the quality of pan-sharpening algorithms often fail to account for changes in geographic context and are agnostic to any specific applications of an end user. This research proposes an evaluation framework to assess the effects of six widely used pan-sharpening algorithms on normalized difference vegetation index (NDVI) calculation in five contextually diverse geographic locations. Output image quality is assessed by comparing the empirical cumulative density function of NDVI values that are calculated by using pre-sharpened and sharpened imagery. The premise is that an effective algorithm will generate a sharpened multispectral image with a cumulative NDVI distribution that is similar to the pre-sharpened image. Research results revealed that, generally, the Gram–Schmidt algorithm introduces a significant degree of spectral distortion regardless of sensor and spatial context. In addition, higher-spatial-resolution imagery is more susceptible to spectral distortions upon pan-sharpening. Furthermore, variability in cumulative density of spectral information in fused images justifies the application of an analytical framework to assist users in selecting the most effective methods for their intended application.

Tensor-based keypoint detection and switching regression model for relative radiometric normalization of bitemporal multispectral images

ABSTRACT In some remote sensing applications, such as unsupervised change detection, bitemporal multispectral images must be first aligned/harmonized radiometrically. For doing so, Many Relative Radiometric Normalization (RRN) algorithms exist; however, most suffer from misregistration problems and can only operate on geo/co-registered image pairs, while unregistered multispectral pairs are required. To tackle this situation, keypoint-based RRN methods were introduced, which can radiometrically calibrate unregistered/registered image pairs using keypoint matching algorithms. However, they ignore the spatial and spectral characteristics of spectral bands of input images, resulting in potential RRN errors. They also employ a linear mapping function for RRN modelling, which can not handle non-linear radiometric distortions. To address these limitations, this paper proposes a robust algorithm for RRN of bitemporal multispectral images, using a new extension of SURF detector for multispectral images, namely the Weighted Spectral Structure Tensor SURF (WSST-SURF), and a flexible Switching Regression (SR) model. Taking advantage of the tensor theory, WSST-SURF efficiently preserves both spatial and spectral information distributed over all bands of multispectral images for keypoint detection, resulting in extracting reliable inliers (or keypoints) for RRN. An adaptive SR model is introduced based on the normalized mutual information, accurately approximating a linear/non-linear relationship between inliers in multispectral images. Six unregistered multispectral image pairs captured by inter/intra remote sensing sensors were employed to validate the efficacy of the proposed method. The results indicate that adopting spectral tensor-based SURF methods in the RRN process exhibits better local and global performance than using the original SURF. Furthermore, the proposed method outperforms the existing conventional RRN methods in terms of accuracy and visual quality, indicating its competence for RRN of bitemporal multispectral images with high illumination, viewpoint, and scale differences.

A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES

Abstract. Currently, multimodal remote sensing images have complex geometric and radiometric distortions, which are beyond the reach of classical hand-crafted feature-based matching. Although keypoint matching methods have been developed in recent decades, most manual and deep learning-based techniques cannot effectively extract highly repeatable keypoints. To address that, we design a Siamese network with self-supervised training to generate similar keypoint feature maps between multimodal images, and detect highly repeatable keypoints by computing local spatial- and channel-domain peaks of the feature maps. We exploit the confidence level of keypoints to enable the detection network to evaluate potential keypoints with end-to-end trainability. Unlike most trainable detectors, it does not require the generation of pseudo-ground truth points. In the experiments, the proposed method is evaluated using various SAR and optical images covering different scenes. The results prove its superior keypoint detection performance compared with current state-of-art matching methods based on keypoints.

Anisotropic Total Variation Regularized Low-Rank Approximation for SSS Images Radiometric Distortion Correction

Radiometric distortion caused by the time-varying gain (TVG), beam patterns, angular responses, and sonar altitude variations, highly degrades the quality of side-scan sonar (SSS) images. Thus, radiometric distortion correction becomes a fundamental step for SSS image processing which holds vital importance for geomorphic applications. However, existing methods cannot take the prior information of the acoustic illumination component as well as the feature of seafloor into consideration well, which would easily cause damage to the image and also always be powerless for residual stripe noise. In this paper, a novel radiometric correction method is proposed. First, we give a detailed analysis of the SSS imaging theory based on the Lambert’s law as well as the prior knowledge about the characteristics of SSS images. Then, incorporating the prior of the SSS imaging process, the low-rank constraint is specifically introduced for the illumination component, while the anisotropic total variation (ATV) constraint is used to constraint the albedo component, combining other constraints, a decomposition model is proposed to correct the radiometric distortion based on the SSS imaging theory. And an alternative minimization method has been adopted to solve the proposed model effectively. Experiments proved the validity of the proposed method.

A Multiscale Framework With Unsupervised Learning for Remote Sensing Image Registration

Registration for multisensor or multimodal image pairs with a large degree of distortions is a fundamental task for many remote sensing applications. To achieve accurate and low-cost remote sensing image registration, we propose a multiscale framework with unsupervised learning, named MU-Net. Without costly ground truth labels, MU-Net directly learns the end-to-end mapping from the image pairs to their transformation parameters. MU-Net stacks several deep neural network (DNN) models on multiple scales to generate a coarse-to-fine registration pipeline, which prevents the backpropagation from falling into a local extremum and resists significant image distortions. We design a novel loss function paradigm based on structural similarity, which makes MU-Net suitable for various types of multimodal images. MU-Net is compared with traditional feature-based and area-based methods, as well as supervised and other unsupervised learning methods on the optical-optical, optical-infrared, optical-synthetic aperture radar (SAR), and optical-map datasets. Experimental results show that MU-Net achieves more comprehensive and accurate registration performance between these image pairs with geometric and radiometric distortions. We share the code implemented by Pytorch at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yeyuanxin110/MU-Net</uri> .