Multitemporal Remote-sensing Images Research Articles

LIRRN: Location-Independent Relative Radiometric Normalization of Bitemporal Remote-Sensing Images.

Relative radiometric normalization (RRN) is a critical pre-processing step that enables accurate comparisons of multitemporal remote-sensing (RS) images through unsupervised change detection. Although existing RRN methods generally have promising results in most cases, their effectiveness depends on specific conditions, especially in scenarios with land cover/land use (LULC) in image pairs in different locations. These methods often overlook these complexities, potentially introducing biases to RRN results, mainly because of the use of spatially aligned pseudo-invariant features (PIFs) for modeling. To address this, we introduce a location-independent RRN (LIRRN) method in this study that can automatically identify non-spatially matched PIFs based on brightness characteristics. Additionally, as a fast and coregistration-free model, LIRRN complements keypoint-based RRN for more accurate results in applications where coregistration is crucial. The LIRRN process starts with segmenting reference and subject images into dark, gray, and bright zones using the multi-Otsu threshold technique. PIFs are then efficiently extracted from each zone using nearest-distance-based image content matching without any spatial constraints. These PIFs construct a linear model during subject-image calibration on a band-by-band basis. The performance evaluation involved tests on five registered/unregistered bitemporal satellite images, comparing results from three conventional methods: histogram matching (HM), blockwise KAZE, and keypoint-based RRN algorithms. Experimental results consistently demonstrated LIRRN's superior performance, particularly in handling unregistered datasets. LIRRN also exhibited faster execution times than blockwise KAZE and keypoint-based approaches while yielding results comparable to those of HM in estimating normalization coefficients. Combining LIRRN and keypoint-based RRN models resulted in even more accurate and reliable results, albeit with a slight lengthening of the computational time. To investigate and further develop LIRRN, its code, and some sample datasets are available at link in Data Availability Statement.

Thick Cloud Removal in Multitemporal Remote Sensing Images via Low-Rank Regularized Self-Supervised Network

Thick Cloud Removal in Multitemporal Remote Sensing Images via Low-Rank Regularized Self-Supervised Network

Snow Cover Detection Using Multi-Temporal Remotely Sensed Images of Fengyun-4A in Qinghai-Tibetan Plateau

Differentiating between snow and clouds presents a formidable challenge in the context of mapping snow cover over the Qinghai–Tibetan Plateau (QTP). The frequent presence of cloudy conditions severely complicates the discrimination of snow cover from satellite imagery. To accurately monitor the spatiotemporal evolution of snow cover, it is imperative to address these challenges and enhance the segmentation schemes employed for snow cover assessment. In this study, we devised a pixel-wise classification algorithm based on Support Vector Machine (SVM) called the 3-D Orientation Gradient algorithm (3-D OG), which captures the variations of the gradient direction of snow and clouds in spatiotemporal dimensions based on geostationary satellite “Fengyun-4A” (FY-4A) multi-spectral and multi-temporal optical imagery. This algorithm assumes that the speed and direction of clouds and snow are different in the process of movement leading to their discrepancy of gradient characteristics in time and space. Therefore, in this algorithm, the gradient of the images in the spatiotemporal dimensions is calculated first, and then the movement angle and trend are obtained based on that. Finally, the feature space is composed of the multi-spectral image, gradient image, and movement feature maps, which are used as the input of the SVM. Our results demonstrate that the proposed algorithm can identify snow and clouds more accurately during snowfall by utilizing the FY-4A’s high temporal resolution image. Weather station data, which was collected during snowstorms in the QTP, were used for evaluating the accuracy of our algorithm. It is demonstrated that the overall accuracy of snow cover segmentation by using the 3-D OG algorithm is improved by at least 12% and 10% as compared to snow products of Fengyun-2 and MODIS, respectively. Overall, the proposed algorithm has overcome the axial swing errors existing in Geostationary satellites and is successfully applied to cloud and snow segmentation in QTP. Furthermore, our study underscores that the visible and near-infrared bands of Fengyun-4A can be used for near real-time snow cover monitoring with high performance using the 3-D OG algorithm.

A Framework for Retrieving Soil Organic Matter by Coupling Multi-Temporal Remote Sensing Images and Variable Selection in the Sanjiang Plain, China

Soil organic matter (SOM) is an important soil property for agricultural production. Rising grain demand has increased the intensity of cultivated land development in the Sanjiang Plain of China, and there is a strong demand for SOM monitoring in this region. Therefore, Baoqing County of the Sanjiang Plain, an important grain production area, was considered the study area. In the study, we proposed a framework for high-accuracy SOM retrieval by coupling multi-temporal remote sensing (RS) images and variable selection algorithms. A total of 73 surface soil samples (0–20 cm) were collected in 2010, and Landsat 5 images acquired during the bare soil period (April, May, and June) were selected from 2008 to 2011. Three variable selection algorithms, namely, Genetic Algorithm, Random Frog and Competitive Adaptive Reweighted Sampling (CARS), were combined with Partial Least Squares Regression (PLSR) to build SOM retrieval models on the spectral bands and indices of the images. The results using a single-date image showed that the combination of variable selection algorithms and PLSR outperformed using PLSR alone, and CARS showed the best performance (R2 = 0.34, RMSE = 15.66 g/kg) among all the algorithms. Therefore, only CARS was applied to SOM retrieval in the different year interval groups. To investigate the effect of the image acquisition time, all images were divided into various year interval groups, and the resulting images were then stacked. The results using multi-temporal images showed that the SOM retrieval accuracy improved as the year interval lengthened. The optimal result (R2 = 0.59, RMSE = 11.81 g/kg) was obtained from the 2008–2011 group, wherein the difference indices derived from the images of 2009, 2010, and 2011 dominated the selected spectral variables. Moreover, the spatial prediction of SOM based on the optimal model was consistent with the distribution of SOM. Our study suggested that the proposed framework that couples stacked multi-temporal RS images with variable selection algorithms has potential for SOM retrieval.

Correction: Yu et al. Landslide Susceptibility Mapping Based on Multitemporal Remote Sensing Image Change Detection and Multiexponential Band Math. Sustainability 2023, 15, 2226

The authors would like to make the following corrections about the published paper [...]

Spatial-spectral-temporal connective tensor network decomposition for thick cloud removal

Thick cloud removal is a fundamental and challenging remote sensing image processing problem, which is beneficial to subsequent analysis and application. Fortunately, multi-temporal remote sensing images (MTRSIs) contain highly correlated spatial–spectral–temporal (SST) information that makes it possible to solve the thick cloud removal problem. To explore the rich SST relationships of MTRSIs effectively and efficiently, this paper proposes a semi-blind thick cloud removal method via SST connective tensor network decomposition (SSTC-CR). Specifically, the proposed model performs the subspace representation along the spectral mode of the image at each time node, and then introduces the tensor network decomposition to characterize the intrinsic relationship of the fourth-order tensor consisting of all intrinsic images under subspace representation. Besides, we develop a proximal alternating minimization-based algorithm to tackle the newly-built thick cloud removal model. In the developed algorithm, we integrate the cloud removal and mask refinement by updating optimization variables and the cloud masks alternately. Extensive numerical experiments on simulated and real-world MTRSIs substantiate that the proposed method outperforms other state-of-the-art tensor modeling and deep learning-based methods, especially for color protection. The code is available at https://github.com/zhaoxile/SSTC_CR.

The spatial distribution and expansion of Eucalyptus in its hotspots: Implications on agricultural landscapes

Fast coppicing plantations like Eucalyptus are becoming an ever increasingly important land use system globally, including the Eucalyptus hotspot highlands of Northwestern Ethiopia. However, comprehensive information regarding species composition is essential for proper planning and policy decisions. The current study mapped the spatial distribution of Eucalyptus globulus (hereafter referred to as Eucalyptus) and identified the key push factors for its expansion. The study used a mapping procedure that uses Landsat imagery together with ground truth data based on supervised training of a pixel-by-pixel classification algorithm within image regions to distinguish areas of Eucalyptus plantations from other classes. High-resolution multispectral and multi-temporal remote-sensing images were combined with ground truth data to produce robust features of Eucalyptus plantation distribution maps. Heckman's Two-Stage econometric model was also employed for determining the major driving factors of Eucalyptus expansion. The results of the mapping algorithm were Eucalyptus plantation distribution maps of 30 × 30 m resolution that showed temporal changes from 1999 to 2021. The findings revealed that Eucalyptus coverage increased by 55% during the period from 1999 to 2010 and the change expressively increased to 69% in 2021 with respect to the reference period. The study also found that a number of push factors influenced the size of land planted with Eucalyptus. The developed maps showing the spatial distribution and expansion of Eucalyptus will help policymakers properly manage the ecosystems and agricultural landscapes of Eucalyptus growing areas.

Thick Cloud Removal for Multitemporal Remote Sensing Images: When Tensor Ring Decomposition Meets Gradient Domain Fidelity

Thick clouds in remote sensing (RS) images deteriorate the visual quality and hinder subsequent applications. The emerging multitemporal RS images with rich temporal information bring the opportunity for cloud removal. How to effectively exploit the rich temporal information of the multitemporal RS images remains challenging. As multitemporal RS images with the same geographic scene, the spatial gradient of RS images at different time nodes has a resemblance, which can guide the reconstruction of the cloudy region. Motivated by this, we suggest a gradient domain fidelity with respect to the guided gradient for thick cloud removal in multitemporal RS images, which faithfully preserves the fine edges and textures compared to the original pixel domain fidelity. Armed with the gradient domain fidelity, we propose a low-rank tensor ring decomposition model (TRGFid) for the thick cloud removal problem. In the proposed model, the guided gradient of the cloudy region is availably estimated by using the regression method from the cloud-free region of different time nodes. Moreover, we develop an efficient proximal alternating minimization-based algorithm for solving the proposed nonconvex model. Extensive simulated and real experiments show that the proposed method outperforms its competitors, and preserves fine edges and textures.

A Robust Oriented Filter-Based Matching Method for Multisource, Multitemporal Remote Sensing Images

A Robust Oriented Filter-Based Matching Method for Multisource, Multitemporal Remote Sensing Images

Unsupervised Domain Factorization Network for Thick Cloud Removal of Multitemporal Remotely Sensed Images

Unsupervised Domain Factorization Network for Thick Cloud Removal of Multitemporal Remotely Sensed Images

The Impact of Central Heating on the Urban Thermal Environment Based on Multi-Temporal Remote Sensing Images

Research on the impact of anthropogenic heat discharge in a thermal environment is significant in climate change research. Central heating is more common in the winter in Northeast China as an anthropogenic heat. This study investigates the impact of central heating on the thermal environment in Shenyang, Changchun, and Harbin based on multi-temporal land surface temperature retrieval from remote sensing. An equivalent heat island index method was proposed to overcome the problem of the method based on a single-phase image, which cannot evaluate all the central heating season changes. The method improves the comprehensiveness of a thermal environment evaluation by considering the long-term heat accumulation. The results indicated a significant increase in equivalent heat island areas at night with 22.1%, 17.3%, and 19.5% over Shenyang, Changchun, and Harbin. The increase was significantly positively correlated with the central heating supply (with an R-value of 0.89 for Shenyang, 0.93 for Changchun, and 0.86 for Harbin; p < 0.05). The impact of central heating has a more significant effect than the air temperature. The results provide a reference for future studies of urban thermal environment changes.

Outlier Reconstruction of NDVI for Vegetation-Cover Dynamic Analyses

The normalized difference vegetation index (NDVI) contains important data for providing vegetation-cover information and supporting environmental analyses. However, understanding long-term vegetation cover dynamics remains challenging due to data outliers that are found in cloudy regions. In this article, we propose a sliding-window-based tensor stream analysis algorithm (SWTSA) for reconstructing outliers in NDVI from multitemporal optical remote-sensing images. First, we constructed a tensor stream of NDVI that was calculated from clear-sky optical remote-sensing images corresponding to seasons on the basis of the acquired date. Second, we conducted tensor decomposition and reconstruction by SWTSA. Landsat series remote-sensing images were used in experiments to demonstrate the applicability of the SWTSA. Experiments were carried out successfully on the basis of data from the estuary area of Salween River in Southeast Asia. Compared with random forest regression (RFR), SWTSA has higher accuracy and better reconstruction capabilities. Results show that SWTSA is reliable and suitable for reconstructing outliers of NDVI from multitemporal optical remote-sensing images.

Automatic Relative Radiometric Normalization of Bi-Temporal Satellite Images Using a Coarse-to-Fine Pseudo-Invariant Features Selection and Fuzzy Integral Fusion Strategies

Relative radiometric normalization (RRN) is important for pre-processing and analyzing multitemporal remote sensing (RS) images. Multitemporal RS images usually include different land use/land cover (LULC) types; therefore, considering an identical linear relationship during RRN modeling may result in potential errors in the RRN results. To resolve this issue, we proposed a new automatic RRN technique that efficiently selects the clustered pseudo-invariant features (PIFs) through a coarse-to-fine strategy and uses them in a fusion-based RRN modeling approach. In the coarse stage, an efficient difference index was first generated from the down-sampled reference and target images by combining the spectral correlation, spectral angle mapper (SAM), and Chebyshev distance. This index was then categorized into three groups of changed, unchanged, and uncertain classes using a fast multiple thresholding technique. In the fine stage, the subject image was first segmented into different clusters by the histogram-based fuzzy c-means (HFCM) algorithm. The optimal PIFs were then selected from unchanged and uncertain regions using each cluster’s bivariate joint distribution analysis. In the RRN modeling step, two normalized subject images were first produced using the robust linear regression (RLR) and cluster-wise-RLR (CRLR) methods based on the clustered PIFs. Finally, the normalized images were fused using the Choquet fuzzy integral fusion strategy for overwhelming the discontinuity between clusters in the final results and keeping the radiometric rectification optimal. Several experiments were implemented on four different bi-temporal satellite images and a simulated dataset to demonstrate the efficiency of the proposed method. The results showed that the proposed method yielded superior RRN results and outperformed other considered well-known RRN algorithms in terms of both accuracy level and execution time.

Land Use/Land Cover Evaluation Using Trajectory Maps Based on Landsat TM/OLI in Southwest China

This study mainly aims to detect the county-level spatio-temporal variability of LULCC (Land Use and Land Cover Change) spatial patterns in Southwest China. Multi-temporal Remote Sensing (RS) images (Landsat TM/OLI in 2000, 2005, 2010, and 2015) were applied to extract land use/cover types at each of the four-time nodes using the Support Vector Machine (SVM) method. Then, the trajectory map methodology was adopted to identify the spatio-temporal distribution characteristics of LULCC patterns throughout the given time series. According to the results, the area of unused land decreased continuously, 0.094% total. An evident decline of grassland by 2.17% was documented, and a notable increase was observed in forestland by 63.94 km2 during the period from 2000 to 2015. Water bodies, built-up land, and unused land showed no significant change throughout the study period. The conversion from grassland to forestland and vice-versa (code 13 or 31) was prominent due to an adjustment made to local forestry policy during the first two periods (2000-2005 and 2005-2010), accounting for 17.55% and 17.56% of the study region, respectively. The anaphase trajectory transition type occupied the smallest area of all the trajectory maps. By contrast, the repetitive trajectory was the leading land-use transition type and covered the largest area. Trajectory analysis provides an effective approach for detecting the spatio-temporal changes in LULCC patterns.

Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset

Analyzing land cover changes with multi-temporal remote sensing (RS) images is crucial for environmental protection and land planning. In this paper, we explore Remote Sensing Image Change Captioning (RSICC), a new task aiming at generating human-like language descriptions for the land cover changes in multi-temporal RS images. We propose a novel Transformer-based RSICC model (RSICCformer). It consists of three main components: 1) a CNN-based feature extractor to generate high-level features of RS image pairs, 2) a dual-branch Transformer encoder to improve the feature discrimination capacity for the changes, and 3) a caption decoder to generate sentences describing the differences. The dual-branch Transformer encoder consists of a hierarchy of processing stages to capture and recognize multiple changes of interest. Concretely, we use the bi-temporal feature differences as keys to enhance image features (queries) from each temporal image in the dual-branch Transformer encoder. To explore the RSICC task, we build a large-scale dataset named LEVIR-CC, which contains 10077 pairs of bi-temporal RS images and 50385 sentences describing the differences between images. We benchmark existing state-of-the-art synthetic image change captioning methods on the LEVIR-CC dataset, and our RSICCformer outperforms previous methods with a significant margin (+4.98% on BLEU-4 and +9.86% on CIDEr-D). The attention visualization results also suggest that our model can focus on changes of interest and ignore irrelevant changes.

Unsupervised Change Detection of Multispectral Remote Sensing Images Based on Deep Difference Feature Variance Maximization

Remote sensing (RS) image change detection (CD) is a basic application in the military and people's livelihood. The imaging results of multitemporal and multispectral RS images have interference effects such as spectral information difference, imaging light difference, sensor noise, so the accuracy and robustness of traditional CD algorithms are restricted. In this study, we propose an unsupervised fully connected multivariate alteration detection (UFMAD) algorithm. First, UFMAD is used to extract the deep difference feature of normalized multitemporal RS images. The weighted map was obtained by using the chi-square distance of the deep difference feature and weighted with the input data. Then, the UFMAD is iteratively optimized under the three proposed constraints of variance maximization, similarity maximization, and uncorrelation of deep features. Finally, the deep feature was clustered to get the change results. By analyzing the experimental results of six advanced CD algorithms on three sets of data, the highest relative detection accuracy of UFMAD has increased by 7.0% . It is verified that the algorithm proposed in this thesis has obvious performance advantages.

Open Set Semantic Segmentation for Multitemporal Crop Recognition

Multitemporal remote-sensing images play a key role as a source of information for automated crop mapping and monitoring. The spatial/spectral pattern evolution along time provides information about the dynamics of the crops and are very useful for productivity estimation. Although the multitemporal mapping of crops has progressed considerably with the advent of deep learning in recent years, the classification models obtained still have limitations when exposed to unknown classes in the prediction phase, reducing their usefulness. In other words, these models are trained to identify a closed set of crops (e.g., soy and sugar cane) and are therefore unable to recognize other types of crops (e.g., maize). In this letter, we deal with the challenges of multitemporal crop recognition by proposing a new approach called OpenPCS++ that is not only able to learn known classes but is also capable of identifying new crops in the predicting phase. The proposed approach was evaluated in two challenging public datasets located in tropical climates in Brazil. Results showed that OpenPCS++ achieved increases of up to 0.19 in terms of area under the receiver-operating characteristic (ROC) curve in comparison with baselines. Code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/DiMorten/osss-mcr</uri> .

Image super-resolution with dense-sampling residual channel-spatial attention networks for multi-temporal remote sensing image classification

Image super-resolution (SR) techniques can benefit a wide range of applications in the remote sensing (RS) community, including image classification. This issue is particularly relevant for image classification on time series data, considering RS datasets that feature long temporal coverage generally have a limited spatial resolution. Recent advances in deep learning brought new opportunities for enhancing the spatial resolution of historic RS data. Numerous convolutional neural network (CNN)-based methods showed superior performance in terms of developing efficient end-to-end SR models for natural images. However, such models were rarely exploited for promoting image classification based on multispectral RS data. This paper proposes a novel CNN-based framework to enhance the spatial resolution of time series multispectral RS images. Thereby, the proposed SR model employs Residual Channel Attention Networks (RCAN) as a backbone structure, whereas based on this structure the proposed models uniquely integrate tailored channel-spatial attention and dense-sampling mechanisms for performance improvement. Subsequently, state-of-the-art CNN-based classifiers are incorporated to produce classification maps based on the enhanced time series data. The experiments proved that the proposed SR model can enable unambiguously better performance compared to RCAN and other (deep learning-based) SR techniques, especially in a domain adaptation context, i.e., leveraging Sentinel-2 images for generating SR Landsat images. Furthermore, the experimental results confirmed that the enhanced multi-temporal RS images can bring substantial improvement on fine-grained multi-temporal land use classification.

A Novel Change Detection Approach Based on Spectral Unmixing from Stacked Multitemporal Remote Sensing Images with a Variability of Endmembers

Due to the high temporal repetition rates, median/low spatial resolution remote sensing images are the main data source of change detection (CD). It is worth noting that they contain a large number of mixed pixels, which makes adequately capturing the details in the resulting thematic map challenging. The spectral unmixing (SU) method is a potential solution to this problem, as it decomposes mixed pixels into a set of fractions of the land covers. However, there are accumulated errors in the fractional difference images, which lead to a poor change detection results. Meanwhile, the spectra variation of the endmember and the heterogeneity of the land cover materials cannot be fully considered in the traditional framework. In order to solve this problem, a novel change detection approach with image stacking and dividing based on spectral unmixing while considering the variability of endmembers (CD_SDSUVE) was proposed in this paper. Firstly, the remote sensing images at different times were stacked into a unified framework. After that, several patch images were produced by dividing the stacked images so that the similar endmembers according to each land cover can be completely extracted and compared. Finally, the multiple endmember spectral mixture analysis (MESMA) is performed, and the abundant images were combined to produce the entire change detection thematic map. This proposed algorithm was implemented and compared to four relevant state-of-the-art methods on three experimental data, whereby the results confirmed that it effectively improved the accuracy. In the simulated data, the overall accuracy (OA) and Kappa coefficient values were 99.61% and 0.99. In the two real data, the maximum of OA were acquired with 93.26% and 80.85%, which gained 14.88% and 13.42% over the worst results at most. Meanwhile, the Kappa coefficient value was consistent with the OA.

Research on the Change of Coastline on the South Coast of Hangzhou Bay Based on Multi-temporal Remote Sensing Images

Coastline is an important geographic background for each coastal area, and dynamic monitoring of coastlines is beneficial to coastal natural protection and further coastal construction and development. Based on the Landsat8 data and Sentinel-2 data from 2014 to 2019, this paper uses a method combining regional growth and edge detection to analyze the Cixi section’s coastline changes on the south bank of Hangzhou Bay. Firstly, the multi-source remote sensing data is preprocessed, and then the MNDWI value of each image is calculated. Then, each image is processed into a binary image by a regional growth algorithm. Eventually, we realize the coastline segmentation by the Canny operator of edge detection. The research represents that the coastline of the south coast of Hangzhou Bay moves toward the ocean year by year, and the length of the coastline generally decreases. The analysis may be caused by human-made activities in Hangzhou Bay Area in recent years, such as river cut-offs and urban construction. The research in this paper provides a reference for the protection of natural coastlines and human-made coastlines.