Multi-source Remote Sensing Images Research Articles

Estimating Forest Canopy Height based on GEDI Lidar Data and Multi-source Remote Sensing Images

Abstract. Estimating forest canopy height is crucial for assessing aboveground biomass and carbon sequestration. Light detection and ranging (lidar) is an important technology for its ability in capturing vertical structural information. However, due to instrument limitations and cost constraints, acquiring large-scale and continuous forest data solely through lidar is challenging. To compensate this, remote sensing images can be used to cover wide regions. Therefore, leveraging multi-source data for constructing canopy height models (CHMs) holds great promise in this field. The objective of this study is to evaluate and compare the contributions of multi-source remote sensing data and methods in estimating forest canopy height. In constructing the CHM, the commonly used random forest (RF) and fully convolutional network (FCN) are assessed. The canopy height obtained from GEDI was used as the reference data, and Landsat 8 and Sentinel-2 data were used for prediction. Multiple CHMs were constructed for the Dabie Mountains, Central China, in 2019 based on different data sources and methods, respectively, which are then comparatively analysed. The results showed that (1) the accuracy of the CHM using Sentinel-2 as input is marginally better than that using Landsat 8 based on RF, where the difference is insignificant; and (2) FCN is less accurate than RF despite domain-specific fine-tuning, although further improvement in accuracy is expected by weighing in more FCN models.

Parcel-Level Crop Classification in Plain Fragmented Regions Based on Multi-Source Remote Sensing Images

Parcel-Level Crop Classification in Plain Fragmented Regions Based on Multi-Source Remote Sensing Images

Automatic Rice Early-Season Mapping Based on Simple Non-Iterative Clustering and Multi-Source Remote Sensing Images

Timely and accurate rice spatial distribution maps play a vital role in food security and social stability. Early-season rice mapping is of great significance for yield estimation, crop insurance, and national food policymaking. Taking Tongjiang City in Heilongjiang Province with strong spatial heterogeneity as study area, a hierarchical K-Means binary automatic rice classification method based on phenological feature optimization (PFO-HKMAR) is proposed, using Google Earth Engine platform and Sentinel-1/2, and Landsat 7/8 data. First, a SAR backscattering intensity time series is reconstructed and used to construct and optimize polarization characteristics. A new SAR index named VH-sum is built, which is defined as the summation of VH backscattering intensity for specific time periods based on the temporal changes in VH polarization characteristics of different land cover types. Then comes feature selection, optimization, and reconstruction of optical data. Finally, the PFO-HKMAR classification method is established based on Simple Non-Iterative Clustering. PFO-HKMAR can achieve early-season rice mapping one month before harvest, with overall accuracy, Kappa, and F1 score reaching 0.9114, 0.8240 and 0.9120, respectively (F1 score is greater than 0.9). Compared with the two crop distribution datasets in Northeast China and ARM-SARFS, overall accuracy, Kappa, and F1 scores of PFO-HKMAR are improved by 0.0507–0.1957, 0.1029–0.3945, and 0.0611–0.1791, respectively. The results show that PFO-HKMAR can be promoted in Northeast China to enable early-season rice mapping, and provide valuable and timely information to different stakeholders and decision makers.

Representation-Enhanced Status Replay Network for Multisource Remote-Sensing Image Classification.

Deep-learning-based methods are widely used in multisource remote-sensing image classification, and the improvement in their performance confirms the effectiveness of deep learning for classification tasks. However, the inherent underlying problems of deep-learning models still hinder the further improvement of classification accuracy. For example, after multiple rounds of optimization learning, representation bias and classifier bias are accumulated, which prevents the further optimization of network performance. In addition, the imbalance of fusion information among multisource images also leads to insufficient information interaction throughout the fusion process, thus making it difficult to fully utilize the complementary information of multisource data. To address these issues, a Representation-enhanced Status Replay Network (RSRNet) is proposed. First, a dual augmentation including modal augmentation and semantic augmentation is proposed to enhance the transferability and discreteness of feature representation, to reduce the impact of representation bias in the feature extractor. Then, to alleviate the classifier bias and maintain the stability of the decision boundary, a status replay strategy (SRS) is built to regulate the learning and optimization of the classifier. Finally, aiming to improve the interactivity of modal fusion, a novel cross-modal interactive fusion (CMIF) method is employed to jointly optimize the parameters of different branches by combining multisource information. Quantitative and qualitative results on three datasets demonstrate the superiority of RSRNet in multisource remote-sensing image classification, and its outperformance compared with other state-of-the-art methods.

Inversion of Forest Biomass Based on Multi-Source Remote Sensing Images

Ecological forests are an important part of terrestrial ecosystems, are an important carbon sink and play a pivotal role in the global carbon cycle. At present, the comprehensive utilization of optical and radar data has broad application prospects in forest parameter extraction and biomass estimation. In this study, tree and topographic data of 354 plots in key nature reserves of Liaoning Province were used for biomass analysis. Remote sensing parameters were extracted from Landsat 8 OLI and Sentinel-1A radar data. Based on the strong correlation factors obtained via Pearson correlation analysis, a linear model, BP neural network model and PSO neural network model were used to simulate the biomass of the study area. The advantages of the three models were compared and analyzed, and the optimal model was selected to invert the biomass of Liaoning province. The results showed that 44 factors were correlated with forest biomass (p < 0.05), and 21 factors were significantly correlated with forest biomass (p < 0.01). The comparison between the prediction results of the three models and the real results shows that the PSO-improved neural network simulation results are the best, and the coefficient of determination is 0.7657. Through analysis, it is found that there is a nonlinear relationship between actual biomass and remote sensing data. Particle swarm optimization (PSO) can effectively solve the problem of low accuracy in traditional BP neural network models while maintaining a good training speed. The improved particle swarm model has good accuracy and speed and has broad application prospects in forest biomass inversion.

S2EFT: Spectral-Spatial-Elevation Fusion Transformer for hyperspectral image and LiDAR classification

With the innovation of sensors, communication, computer and other technologies, remote sensing (RS) imaging methods show a diversification trend. At present, relying on a variety of earth observation platforms, more multi-source RS data can be obtained, which overcomes the inadequacy and incompleteness of the new earth observation information of single source RS image. In this paper, a classification method of hyperspectral images (HSI) and LiDAR data based on spectral-spatial-elevation fusion Transformer (S2EFT) framework is proposed. The Transformer framework is introduced into the task of multi-source RS image classification. Two simple but effective modules, spatial information recognition module and sliding group spectral embedding module, are added to the proposed framework, and the Patch form commonly used in traditional convolutional neural networks (CNNs) is used as the input. It effectively solves the shortcomings of Transformer’s insufficient attention to local information, reduces redundant spatial information, enhances the transmission of information, and fully integrates multidimensional information of pixels in same position. Experiments on three real data sets are compared with existing methods, and the best results are obtained by the proposed methods. The source code can be downloaded from https://github.com/SYFYN0317/S2EFT.

Retracted: Artificial Neural Network in Classification of Multisource Remote Sensing Images

Retracted: Artificial Neural Network in Classification of Multisource Remote Sensing Images

Semi-Supervised Tree Species Classification for Multi-Source Remote Sensing Images Based on a Graph Convolutional Neural Network

As a current research hotspot, graph convolution networks (GCNs) have provided new opportunities for tree species classification in multi-source remote sensing images. To solve the challenge of limited label information, a new tree species classification model was proposed by using the semi-supervised graph convolution fusion method for hyperspectral images (HSIs) and multispectral images (MSIs). In the model, the graph-based attribute features and pixel-based features are fused to deepen the correlation of multi-source images to improve accuracy. Firstly, the model employs the canonical correlation analysis (CCA) method to maximize the correlation of multi-source images, which explores the relationship between information from various sources further and offers more valuable insights. Secondly, convolution calculations were made to extract features and then map graph node fusion, which not only reduces redundancy features but also enhances compelling features. Finally, the relationship between representative descriptors is captured through the use of hyperedge convolution in the training process, and the dominant features on the graph are fully mined. The tree species are classified through two fusion feature operations, leading to improved classification performance compared to state-of-the-art methods. The fusion strategy can produce a complete classification map of the study areas.

Delineation Evaluation and Variation of Debris-Covered Glaciers Based on the Multi-Source Remote Sensing Images, Take Glaciers in the Eastern Tomur Peak Region for Example

As a particular type of alpine glacier, debris-covered glaciers are essential for local water resources and glacial disaster warnings. The Eastern Tomur Peak Region (EPTR) is the most concentrated glacier in Tien Shan Mountain, China, where the glaciers have not been studied in detail. This paper evaluates the delineation accuracy of Landsat8 OLI, Sentinel-1A, and GF images for debris-covered glaciers in the EPTR. Each image uses the most advanced delineation method for itself to minimize the error of inherent resolutions. The results show that the accuracy of these images for delineating debris-covered glaciers is very high, and the F1 scores are expressed as 96.73%, 93.55%, and 95.81%, respectively. Therefore, Landsat images were selected to analyze the area change of EPTR from 2000 to 2022 over a 5-year time scale. The results indicate that glaciers of the EPTR decreased by 19.05 km2 from 2000 to 2020, accounting for 1.9% (0.08% a−1), and debris increased by 10.8%, which validates the opinion that the presence of debris inhibits glacier melting. The most varied time was 2010–2022, but it was much less than other Tien Shan regions. The lower glacier ablation rate in this area results from the combined effect of decreased bare ice and increased debris. The main reason for the change in debris-covered glaciers is the increase in temperature.

Building Change Detection with Deep Learning by Fusing Spectral and Texture Features of Multisource Remote Sensing Images: A GF-1 and Sentinel 2B Data Case

Building change detection is an important task in the remote sensing field, and the powerful feature extraction ability of the deep neural network model shows strong advantages in this task. However, the datasets used for this study are mostly three-band high-resolution remote sensing images from a single data source, and few spectral features limit the development of building change detection from multisource remote sensing images. To investigate the influence of spectral and texture features on the effect of building change detection based on deep learning, a multisource building change detection dataset (MS-HS BCD dataset) is produced in this paper using GF-1 high-resolution remote sensing images and Sentinel-2B multispectral remote sensing images. According to the different resolutions of each Sentinel-2B band, eight different multisource spectral data combinations are designed, and six advanced network models are selected for the experiments. After adding multisource spectral and texture feature data, the results show that the detection effects of the six networks improve to different degrees. Taking the MSF-Net network as an example, the F1-score and IOU improved by 0.67% and 1.09%, respectively, compared with high-resolution images, and by 7.57% and 6.21% compared with multispectral images.

Tree Species Classification Based on Self-Supervised Learning with Multisource Remote Sensing Images

In order to solve the problem of manual labeling in semi-supervised tree species classification, this paper proposes a pixel-level self-supervised learning model named M-SSL (multisource self-supervised learning), which takes the advantage of the information of plenty multisource remote sensing images and self-supervised learning methods. Based on hyperspectral images (HSI) and multispectral images (MSI), the features were extracted by combining generative learning methods with contrastive learning methods. Two kinds of multisource encoders named MAAE (multisource AAE encoder) and MVAE (multisource VAE encoder) were proposed, respectively, which set up pretext tasks to extract multisource features as data augmentation. Then the features were discriminated by the depth-wise cross attention module (DCAM) to enhance effective ones. At last, joint self-supervised methods output the tress species classification map to find the trade-off between providing negative samples and reducing the amount of computation. The M-SSL model can learn more representative features in downstream tasks. By employing the feature cross-fusion process, the low-dimensional information of the data is simultaneously learned in a unified network. Through the validation of three tree species datasets, the classification accuracy reached 78%. The proposed method can obtain high-quality features and is more suitable for label-less tree species classification.

Extraction of Information on the Flooding Extent of Agricultural Land in Henan Province Based on Multi-Source Remote Sensing Images and Google Earth Engine

Sudden flood disasters cause serious damage to agricultural production. Rapidly extracting information such as the flooding extent of agricultural land and capturing the influence of flooding on crops provides important guidelines for estimating the flood-affected area, promoting post-disaster farmland restoration, and providing an auxiliary decision-making basis for flood prevention and disaster relief departments. Taking the flood event in Henan and Shanxi Provinces as example, based on the characteristics of the variations in radar data and optical data before and after the disaster, we propose an extent information extraction method for the flood inundation area and the flood-affected area of agricultural land. This method consists of change detection, threshold extraction, and superposition analysis, which weakens the negative impact of the radar data speckle noise and cloud contamination of the optical data on the extraction of the agricultural land flooding to a certain extent. The method was developed based on a flood event in Henan Province and validated in Shanxi Province. The results show that the production of this method have a clear boundary and accurate extent, and the overall precisions of the flood inundation area and flood-affected area extraction are 0.87 and 0.92, respectively. The proposed method combines the advantages of both radar and optical remote sensing data in extracting the specific extents of the flood inundation area and the flood-affected area in large spatial scale. Finally, the impact of time window size to the performance of the method is further analyzed. In the application of the proposed method, the Google Earth Engine (GEE) platform provides a low-cost, fast, and convenient way to extract flood information from remote sensing data. The proposed scheme provides a scientific data basis for restoring production of agricultural land after a flood disaster, as well as for national post-disaster damage assessment and disaster relief decision making.

Deep Multimodal Fusion Model for Building Structural Type Recognition Using Multisource Remote Sensing Images and Building-Related Knowledge

Deep Multimodal Fusion Model for Building Structural Type Recognition Using Multisource Remote Sensing Images and Building-Related Knowledge

Mapping Urban Functional Areas Using Multisource Remote Sensing Images and Open Big Data

Mapping Urban Functional Areas Using Multisource Remote Sensing Images and Open Big Data

SS-MAE: Spatial–Spectral Masked Autoencoder for Multisource Remote Sensing Image Classification

SS-MAE: Spatial–Spectral Masked Autoencoder for Multisource Remote Sensing Image Classification

Consistency Center-Based Deep Cross-Modal Hashing for Multisource Remote Sensing Image Retrieval

Consistency Center-Based Deep Cross-Modal Hashing for Multisource Remote Sensing Image Retrieval

A Local–Global Framework for Semantic Segmentation of Multisource Remote Sensing Images

Recently, deep learning has been widely used in the segmentation tasks of remote sensing images. However, the existing deep learning method most focus on local contextual information and has limited field of perception, which makes it difficult to capture the long-range contextual feature of objects at large scales form very-high-resolution (VHR) images. In this paper, we present a novel Local–global Framework consisting of the dual-source fusion network and local–global transformer modules, which efficiently utilize features extracted from multiple sources and fully capture features of local and global regions. The dual-source fusion network is an encoder designed to extract features from multiple sources such as spectra, synthetic aperture radar, and elevations, which selective fuse features from multiple sources and reduce the interference of redundant features. The local–global transformer module is proposed to capture fine-grained local features and coarse-grained global features, which enables the framework to focus on recognizing multiple-scale objects from the local and global regions. Moreover, we propose a pixelwise contrastive loss, which could encourage that the prediction is pulled closer to the ground truth. The Local–global Framework achieves state-of-the-art performance with 90.45% mean f1 score on the ISPRS Vaihingen dataset and 93.20% mean f1 score on the ISPRS Potsdam dataset.

Large-Scale Rice Mapping Based on Google Earth Engine and Multi-Source Remote Sensing Images

Large-Scale Rice Mapping Based on Google Earth Engine and Multi-Source Remote Sensing Images

Production Capacity Evaluation of Farmland Using Long Time Series of Remote Sensing Images

Farmland is a crucial resource for the survival and evolution of humans. The accurate evaluation of farmland production capacity (FPC) is of great significance for planting structure optimization, the improvement of low-yield farmland and sustainable utilization. The objective of this study is to quantitatively evaluate the FPC at the county scale using time series remote sensing (RS) images. Taking winter wheat as a benchmark crop, the relations between annual yield and the Normalized Difference Vegetation Index (NDVI) were established by a multiple linear regression algorithm. The mean and standard deviations (SD) of the multi-year yield of winter wheat were used to evaluate FPC and its instability using the farmland parcels as the basic unit. The results show that the estimation model for annual winter wheat yield performed best in 2011. The R2 of the modeling sample was 0.93, and the RMSE of the testing sample was 368.1 kg/ha. The FPC grades in the south and north of the study area were relatively high with a good stability, while those in the center were low with poor stability. There was a certain correlation between FPC and soil organic matter (SOM), and the correlation coefficient was 0.525 (p < 0.01). In this study, taking the farmland parcel as a basic unit instead of a pixel, long time series of multi-source RS images with medium resolution were used to monitor the per unit yield of benchmark crops and then evaluate the FPC. This can provide a method for the rapid evaluation of FPC at the county scale.

Artificial Neural Network in Classification of Multisource Remote Sensing Images

How to solve multi-category image recognition and meet a certain accuracy is a key issue in the research of high-resolution remote sensing images, and it is of great significance. This article mainly studies artificial neural network in the classification of multi-source remote sensing images. This paper improves the efficiency and accuracy of image segmentation by studying the principle and implementation process of image segmentation algorithm from the two aspects of initial segmentation and region merging; secondly, it studies the method of object feature quantization and the image of different object features on the classification results; and finally, it selects BP neural network. The network classification method classifies the image objects and realizes the extraction and classification of high-resolution remote sensing images. Experiments in this paper show that for multi-source remote sensing image data, the overall accuracy of the two parallel classification algorithms is very similar, and both are close to 85%, which has a good classification effect. When performing large-scale image classification, the terrain types in the image will be more complicated. In this case, the extraction accuracy relative to the artificial neural network classification method will decrease, and the classification time will also become longer. This paper proves through experiments that the classification method of multi-source remote sensing images based on artificial neural network is feasible and has certain advantages.