Optical Remote Sensing Research Articles

Corn Land Extraction Based on Integrating Optical and SAR Remote Sensing Images

Corn is an important food crop worldwide, and its yield is directly related to Chinese food security. Accurate remote sensing extraction of corn can realize the rational application of land resources, which is of great significance to the sustainable development of modern agriculture. In the field of large-scale crop remote sensing classification, single-period optical remote sensing images often cannot achieve high-precision classification. To improve classification accuracy, multiple time series image combinations have gradually been adopted. However, due to the influence of cloudy and rainy weather, it is often difficult to obtain complete time series optical images. Synthetic aperture radar (SAR) data are imaged by microwaves, which have strong penetrating power and are not affected by clouds. A critical way to solve this problem is to use SAR images to compensate for the lack of optical images and obtain a complete time series image in the corn-growing season. However, SAR images have limited wavelengths and cannot provide important wavelengths, such as visible light bands and near-infrared information. To solve this problem, this study took Zhaodong City, a vital corn-planting base in China, as the research area; took GF-6/GF-3 and Sentinel-1/Sentinel-2 as remote sensing data sources; designed12 classification scenarios; analyzed the best classification period and the best time series combination of corn classification; studied the influence of SAR images on the classification results of time series images; and compared the classification differences between GF-6/GF-3 and Sentinel-1/Sentinel-2. The results show that the classification accuracy of time series combinations is much higher than that of single-period images. The polarization characteristics of SAR images can improve the classification accuracy with time series images. The classification accuracy of GF series images from China is obviously higher than that of Sentinel series images. The research performed in this paper can provide a reference for agricultural classification by using remote sensing data.

Wavelet Integrated Convolutional Neural Network for Thin Cloud Removal in Remote Sensing Images

Cloud occlusion phenomena are widespread in optical remote sensing (RS) images, leading to information loss and image degradation and causing difficulties in subsequent applications such as land surface classification, object detection, and land change monitoring. Therefore, thin cloud removal is a key preprocessing procedure for optical RS images, and has great practical value. Recent deep learning-based thin cloud removal methods have achieved excellent results. However, these methods have a common problem in that they cannot obtain large receptive fields while preserving image detail. In this paper, we propose a novel wavelet-integrated convolutional neural network for thin cloud removal (WaveCNN-CR) in RS images that can obtain larger receptive fields without any information loss. WaveCNN-CR generates cloud-free images in an end-to-end manner based on an encoder–decoder-like architecture. In the encoding stage, WaveCNN-CR first extracts multi-scale and multi-frequency components via wavelet transform, then further performs feature extraction for each high-frequency component at different scales by multiple enhanced feature extraction modules (EFEM) separately. In the decoding stage, WaveCNN-CR recursively concatenates the processed low-frequency and high-frequency components at each scale, feeds them into EFEMs for feature extraction, then reconstructs the high-resolution low-frequency component by inverse wavelet transform. In addition, the designed EFEM consisting of an attentive residual block (ARB) and gated residual block (GRB) is used to emphasize the more informative features. ARB and GRB enhance features from the perspective of global and local context, respectively. Extensive experiments on the T-CLOUD, RICE1, and WHUS2-CR datasets demonstrate that our WaveCNN-CR significantly outperforms existing state-of-the-art methods.

Attention-Enhanced Region Proposal Networks for Multi-Scale Landslide and Mudslide Detection from Optical Remote Sensing Images

Detecting areas where a landslide or a mudslide might occur is critical for emergency response, disaster recovery, and disaster cost estimation. Previous works have reported that a variety of convolutional neural networks (CNNs) significantly outperform traditional approaches for landslide/mudslide detection. These approaches always consider features from the local window and neighborhood information. The CNNs mainly focus on the features derived at a local scale, which might be inefficient for recognizing complex landslide and mudslide scenes. To effectively identify landslide and mudslide risks at a local and global scale, this paper integrates attentions into the architecture of state-of-the-art CNNs—including Faster RCNN—to develop an attention-enhanced region proposal network for multi-scale landslide/mudslide detection. In detail, we employed the attentions to process the region proposals generated by a region proposal network and then combined the results obtained from the attentions and region proposal network to identify whether the object included in a region proposal was a landslide/mudslide. Based on our developed dataset and the Bijie dataset, the experimental results prove that: (1) although the state-of-the-art CNNs for object detection can precisely detect landslides and mudslides, they are inadequate in dealing with similarity to non-landslide/non-mudslide regions; and (2) the proposed method, which integrates global features from attention layers into local features derived from CNNs, outperforms the unmodified CNNs in detecting non-landslides and non-mudslides. Our findings prove that the representations at the local and global scale might be significant for precise landslide and mudslide detection.

Method of Building Detection in Optical Remote Sensing Images Based on SegFormer.

An appropriate detection network is required to extract building information in remote sensing images and to relieve the issue of poor detection effects resulting from the deficiency of detailed features. Firstly, we embed a transposed convolution sampling module fusing multiple normalization activation layers in the decoder based on the SegFormer network. This step alleviates the issue of missing feature semantics by adding holes and fillings, cascading multiple normalizations and activation layers to hold back over-fitting regularization expression and guarantee steady feature parameter classification. Secondly, the atrous spatial pyramid pooling decoding module is fused to explore multi-scale contextual information and to overcome issues such as the loss of detailed information on local buildings and the lack of long-distance information. Ablation experiments and comparison experiments are performed on the remote sensing image AISD, MBD, and WHU dataset. The robustness and validity of the improved mechanism are demonstrated by control groups of ablation experiments. In comparative experiments with the HRnet, PSPNet, U-Net, DeepLabv3+ networks, and the original detection algorithm, the mIoU of the AISD, the MBD, and the WHU dataset is enhanced by 17.68%, 30.44%, and 15.26%, respectively. The results of the experiments show that the method of this paper is superior to comparative methods such as U-Net. Furthermore, it is better for integrity detection of building edges and reduces the number of missing and false detections.

Few-shot incremental learning with continual prototype calibration for remote sensing image fine-grained classification

With the rapid acquisition of remote sensing (RS) data, new categories of objects continue to emerge, and some categories can only obtain a few training samples. Thus, few-shot class-incremental learning (FSCIL) has received intense attention in recent years. Nevertheless, the existing FSCIL methods are difficult to adapt to RS characteristics with high inter-class similarity, large intra-class differences, and complex backgrounds. In this paper, we propose the FSCIL method called continual prototype calibration (CPC), which applies the decoupled learning strategy to cope with the demand to retrain the model when new few-shot categories appear. It can also address the tedious problems in fine-grained RS images by calibrating inter-class and intra-class prototypes. Concretely, considering the severe issue of inter-class confusion, we introduce a prototype separability module (PSM) to update the distribution of inter-class prototypes. In this way, we can distinguish different categories more clearly by weakening their similarities, and the phenomenon of catastrophic forgetting can also be effectively mitigated. Furthermore, to generate more representative and accurate intra-class prototypes, we design a Meta-Network based on foreground enhancement mechanism (FEM). The Meta-Network can boost the learning ability of the model for few-shot data and suppress the occurrence of overfitting by training on task-based data. And the FEM enhances the object features by filtering out redundant background information, facilitating fine-grained classification. Our CPC method is the first FSCIL method applied to optical RS images, and we verify its effectiveness on three datasets. Extensive experiments illustrate that our method achieves state-of-the-art performance in FSCIL of fine-grained RS images. The code is available at https://github.com/ningerhhh/CPC.

Joint Use of Optical and Radar Remote Sensing Data for Characterizing the 2020 Aniangzhai Landslide Post-Failure Displacement

The ancient Aniangzhai (ANZ) landslide in Danba County, Sichuan Province of southwest China was reactivated after a series of complex hazard events that occurred in June 2020. Since then, and until June 2021, emergency engineering work was carried out to prevent the further failure of the reactivated landslide. This study investigates the potential of joint use of time series Interferometric Synthetic Aperture Radar (InSAR) and optical pixel offset tracking (POT) to assess deformation characteristic and spatial-temporal evolution of the reactivated ANZ landslide during the post-failure stage. The relationships between sun illumination differences, temporal baseline of correlation pairs and the uncertainties were deeply explored. Surface deformation along the line-of-sight (LoS) direction was retrieved by the time series InSAR processing with the two Sentinel-1 datasets, revealing a maximum deformation rate up to 190 mm/year. The large horizontal displacements were also detected from the POT processing using 11 optical images acquired by the PlanetScope satellite (3 m spatial resolution), showing a significant increase of about 24 m between 24 June 2020 and 11 June 2021. The time series analysis from the InSAR and optical POT results revealed that the reactivated ANZ landslide body is gradually slowing down to a steady deformation status since its occurrence in August 2020, indicating the effectiveness of engineering work on the prevention of further landslide. A slight acceleration was detected from both InSAR and optical POT time series analysis between May 2021 and June 2021, which could be caused by the increased rainfall in May 2021.

Surface Soil Moisture Estimation Using a Neural Network Model in Bare Land and Vegetated Areas

Most of the approaches to retrieve surface soil moisture (SSM) by optical and thermal infrared (TIR) spectroscopies are purposed to calculate various characteristic bands/indices and then to establish the regression relationship between them in combination with the measurement data. However, due to the combined impact of many factors, the regression relationship often shows nonlinearity. Moreover, the relationship between the single temporal image and the measured data are not transplantable in time and space, which makes it difficult to construct a more general model for the remote sensing (RS) estimation of SSM. In order to solve this problem, the back propagation (BP) neural network (NN) with an excellent nonlinear mapping ability is introduced to determine the relationship between the characteristic band/index and the measurement data. In the BPNN model, the optical and TIR RS data in different periods were taken as the input parameters, and the in situ soil moisture data were treated as the output parameter. There are 12 schemes designed to retrieve SSM. The key findings of study were as follows: (1) the BPNN model could retrieve SSM with a high accuracy that indicates the correlation coefficient between the estimated and measured soil moisture as 0.9001 and (2) the SSM retrieval model based on the BPNN can be applied to estimate the SSM with different spatial resolution values.

High‐resolution quantification of building stock using multi‐source remote sensing imagery and deep learning

AbstractIn recent decades, urbanization has led to an increase in building material stock. The high‐resolution quantification of building stock is essential to understand the spatial concentration of materials, urban mining potential, and sustainable urban development. Current approaches rely excessively on statistics or survey data, both of which are unavailable for most cities, particularly in underdeveloped areas. This study proposes an end‐to‐end deep‐learning model based on multi‐source remote sensing data, enabling the reliable estimation of building stock. Ground‐detail features extracted from optical remote sensing (ORS) and spatiotemporal features extracted from nighttime light (NTL) data are fused and incorporated into the model to improve accuracy. We also compare the performance of our feature‐fusion model with that of an ORS‐only regression model and traditional NTL regression for Beijing. The proposed model yields the best building‐stock estimation, with a Spearman's rank correlation coefficient of 0.69, weighted root mean square error of 0.58, and total error in the test set below 14%. Using gradient‐weighted class activation mapping, we further investigate the relationship between ORS features and building‐stock estimation. Our model exhibits reliable predictive capability and illustrates the tremendous value of the physical environment for estimating building stock. This research illustrates the significant potential of ORS and deep learning for stock estimation. Large‐scale, long‐term building‐stock investigations could also benefit from the end‐to‐end predictability and the data availability of the model.

Full Semantic Constructed Network for Urban Use Classification From Very High-Resolution Optical Remote Sensing Imagery

Recently, semantic segmentation technology has been a research hotspot in optical remote sensing urban use classification. However, since coupled semantic relations in very high resolution and complex urban scenes, a more effective semantic description for pixelwise urban use interpretation has become a challenge. Then, aiming to set up a more effective semantic description, the effective receptive field (ERF) is analyzed in general convolutional neural networks, and there is an unreasonable ERF distribution in the stacked convolutional layers of the encoder. This would lead to a large amount of small ERFs and fewer not large enough ERFs to form a naive semantic description at decoder. Therefore, in this article, a novel full semantic constructed network (FSCNet) is proposed to improve the naive semantic description and set up an effective semantic description. First, to avoid noise from shallow feature layers, a residual refinement convolution is designed to optimize the full scale skip connections based on the U-shaped encoder-decoder. Second, an interscale fusion module is newly designed for multiscale feature fusion, which can generate three initial semantic modalities that are prepared for redefining the full semantic description. Third, a multiscale local context spatial attention module and boundary supervision are designed for an initial shallow semantic modality to capture the pure boundary information, and then pyramid spatial pooling is employed for an initial deep semantic modality to further enlarge the ERF and obtain a more abstract global information. Next, a self-calibration convolution combined with the atrous spatial pyramid pooling is designed to rectify and enrich an initial middle semantic modality, which can improve the naive semantic description and bridge the semantic gap between the redefined shallow and deep semantic modalities to advance the full semantic feature fusion. Finally, extensive experiments are carried out on three benchmarks (e.g., ISPRS Vaihinge, Potsdam and DLRSD), and comparative results show that the proposed FSCNet can get remarkable performance compared to state-of-the-art (SOTA) methods. Besides, the code is available at https://github.com/DorisCV/FSCNet.

Topographic Effects on Optical Remote Sensing: Simulations by PLC Model

Topographic Effects on Optical Remote Sensing: Simulations by PLC Model

Cloud Detection in Optical Remote Sensing Images With Deep Semi-Supervised and Active Learning

Clouds hinder the surface observation by optical remote sensing sensors. It is of great significance to detect clouds and non-clouds in remote sensing images. Compared with traditional cloud detection methods, deep learning methods usually achieve promising detection results. Moreover, large-scale, high-quality labeled datasets can effectively improve the accuracy and generalization of deep learning models. However, this incurs a great deal of label effort and cost. In this paper, we proposes a cloud detection method based on deep semi-supervised learning(SSL) and active learning(AL) in optical remote sensing images named SSAL-CD. SSL part of SSAL-CD is implemented by the cross-validation between two-way neural networks, which can effectively train in the case of insufficient labeled data and reduce the cost of labeling. AL part proposes a suitable query strategy based on the uncertainty, degree of divergence and diversity information provided by the training model. The query strategy can screen out high-value samples for SSL, which further improves the detection effect. Repeat the process of SSL and AL until the accuracy meets requirements or the labeling budget is exhausted. Extensive experimental results in Landsat-8 Biome dataset demonstrate that SSAL-CD can achieve state-of-the-art segmentation performance with a small number of labels.

Design of Spaceborne Stepper Motor Driver Based on FPGA

In order to meet the specific application requirements of stepper motor in space optical remote sensor, such as large load current, low operating frequency, sufficient safety margin, anti radiation and other space environment indicators, a spaceborne stepper motor driver based on antifuse Field Programmable Gate Array (FPGA) is designed. The driver consists of power conversion module, control module, drive module, and position detection module. Using P-channel Metal-Oxide-Semiconductor Field-Effect Transistor (MOSFET) and N-channel MOSFET to design bipolar drive circuit, the driver hardware circuit is simple and the software resource occupancy rate is low. Through the experiments of the dual functions of logic protection circuit in hardware and dead zone protection in software, it is verified that the design of spaceborne stepper motor driver based on FPGA meets the requirements of space optical remote sensor mechanism control.

Boundary-Aware Network With Two-Stage Partial Decoders for Salient Object Detection in Remote Sensing Images

Salient object detection is a binary pixel-wise classification to distinguish objects in an image, and also have attracted many research interests in the optical Remote Sensing Images (RSIs). The existing state-of-the-art method exploits the full encoder-decoder architecture to predict salient objects in the optical RSIs, suffering from the problem of unsmooth edges and incomplete structures. To address these problems, in this paper, we propose a Boundary-Aware Network (BANet) with two-stage partial decoders sharing the same encoders for salient object detection in RSIs. Specifically, a Boundary-Aware Partial Decoder (BAD) is introduced at the first stage to focus on learning clear edges of salient objects. To solve the pixel-imbalance problem between boundary and background, an edge-aware loss is proposed to guide learning the BAD network. The resulting features are then employed in turn to enhance high-level features. Afterwards, the Structure-Aware Partial Decoder (SAD) is further introduced at the second stage to improve the structure integrity of salient objects. To alleviate the problem of incomplete structures, the structural similarity loss is further proposed to supervise learning the SAD network. In a consequence, our proposed BANet can predict salient objects with clear edges and complete structure, while reducing model parameters due to the discardment of low-level features. Besides, training a deep neural network requires a large amount of images, and the current benchmark datasets for optical remote sensing images are not large enough. Therefore, we also create a large-scale challenging dataset for salient object detection in RSIs. Extensive experiments demonstrate that our proposed BANet outperforms previous RSI SOD models on all existing benchmark datasets and our new presented dataset available at https://github.com/QingpingZheng/RSISOD.

Efficient Object Detection in Optical Remote Sensing Imagery via Attention-Based Feature Distillation

Efficient Object Detection in Optical Remote Sensing Imagery via Attention-Based Feature Distillation

An Effective Multimodel Fusion Method for SAR and Optical Remote Sensing Images

An Effective Multimodel Fusion Method for SAR and Optical Remote Sensing Images

A Copula-Based Method for Change Detection With Multisensor Optical Remote Sensing Images

A Copula-Based Method for Change Detection With Multisensor Optical Remote Sensing Images

Salient Object Detection in Optical Remote Sensing Images Driven by Transformer.

Existing methods for Salient Object Detection in Optical Remote Sensing Images (ORSI-SOD) mainly adopt Convolutional Neural Networks (CNNs) as the backbone, such as VGG and ResNet. Since CNNs can only extract features within certain receptive fields, most ORSI-SOD methods generally follow the local-to-contextual paradigm. In this paper, we propose a novel Global Extraction Local Exploration Network (GeleNet) for ORSI-SOD following the global-to-local paradigm. Specifically, GeleNet first adopts a transformer backbone to generate four-level feature embeddings with global long-range dependencies. Then, GeleNet employs a Direction-aware Shuffle Weighted Spatial Attention Module (D-SWSAM) and its simplified version (SWSAM) to enhance local interactions, and a Knowledge Transfer Module (KTM) to further enhance cross-level contextual interactions. D-SWSAM comprehensively perceives the orientation information in the lowest-level features through directional convolutions to adapt to various orientations of salient objects in ORSIs, and effectively enhances the details of salient objects with an improved attention mechanism. SWSAM discards the direction-aware part of D-SWSAM to focus on localizing salient objects in the highest-level features. KTM models the contextual correlation knowledge of two middle-level features of different scales based on the self-attention mechanism, and transfers the knowledge to the raw features to generate more discriminative features. Finally, a saliency predictor is used to generate the saliency map based on the outputs of the above three modules. Extensive experiments on three public datasets demonstrate that the proposed GeleNet outperforms relevant state-of-the-art methods. The code and results of our method are available at https://github.com/MathLee/GeleNet.

Lightweight Ship Detection Based on Optical Remote Sensing Images for Embedded Platform

Lightweight Ship Detection Based on Optical Remote Sensing Images for Embedded Platform

Contour Modeling Arbitrary-Oriented Ship Detection From Very High-Resolution Optical Remote Sensing Imagery

Under the multiscale distribution, due to dramatic aspect ratio variance leading to prominent arbitrary-oriented character of ships in very high-resolution (VHR) optical remote sensing imagery, how to generate accurate oriented bounding box (OBB) becomes a hot research topic for arbitrary-oriented ship detection. Consequently, in this letter, a concise and effective one-stage anchor-free contour modeling detector called CMDet is proposed for accurate arbitrary-oriented ship detection. Different from currently existed methods via carefully decoupling several independently characteristic parameters for OBB modeling and regression, we resolve the OBB modeling by jointly regressing the contour information. Specifically, the contour information is expressed as a series of Fourier transform coefficients, which are generated by setting up the mapping relation of 1-D Fourier contour coefficients and spatial OBB contour. In addition, a new inherent geometry loss is designed to make detector better learn the geometry information in training phase. After that, the proposed CMDet only needs to predict the correct center point of ships and regress the corresponding entire 1-D Fourier contour coefficients to generate accurate OBB for ship detection. Finally, extensive experiments are carried out on two public OBB ship detection datasets (e.g., HRSC2016 and DIOR-ship), and comparison results demonstrate that the proposed CMDet can obtain the competitive result than the state-of-the-art (SOTA) detectors.

Threatening Patch Attacks on Object Detection in Optical Remote Sensing Images

Advanced Patch Attacks (PAs) on object detection in natural images have pointed out the great safety vulnerability in methods based on deep neural networks. However, little attention has been paid to this topic in Optical Remote Sensing Images (O-RSIs). To this end, we focus on this research, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., PAs on object detection in O-RSIs, and propose a more Threatening PA without the scarification of the visual quality, dubbed TPA. Specifically, to address the problem of inconsistency between local and global landscapes in existing patch selection schemes, we propose leveraging the First-Order Difference (FOD) of the objective function before and after masking to select the sub-patches to be attacked. Further, considering the problem of gradient inundation when applying existing coordinate-based loss to PAs directly, we design an IoU-based objective function specific for PAs, dubbed Bounding box Drifting Loss (BDL), which pushes the detected bounding boxes far from the initial ones until there are no intersections between them. Finally, on two widely used benchmarks, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., DIOR and DOTA, comprehensive evaluations of our TPA with four typical detectors (Faster R-CNN, FCOS, RetinaNet, and YOLO-v4) witness its remarkable effectiveness. To the best of our knowledge, this is the first attempt to study the PAs on object detection in O-RSIs, and we hope this work can get our readers interested in studying this topic.