Remote Sensing Scene Research Articles

Knowledge Guided Evolutionary Transformer for Remote Sensing Scene Classification

Knowledge Guided Evolutionary Transformer for Remote Sensing Scene Classification

Improving remote sensing scene classification using dung Beetle optimization with enhanced deep learning approach

Remote sensing (RS) scene classification has received significant consideration because of its extensive use by the RS community. Scene classification in satellite images has widespread uses in remote surveillance, environmental observation, remote scene analysis, urban planning, and earth observations. Because of the immense benefits of the land scene classification task, various approaches have been presented recently for automatically classifying land scenes from remote sensing images (RSIs). Several approaches dependent upon convolutional neural networks (CNNs) are presented for classifying brutal RS scenes; however, they could only partially capture the context from RSIs due to the problematic texture, cluttered context, tiny size of objects, and considerable differences in object scale. This article designs a Remote Sensing Scene Classification using Dung Beetle Optimization with Enhanced Deep Learning (RSSC-DBOEDL) approach. The purpose of the RSSC-DBOEDL technique is to categorize different varieties of scenes that exist in the RSI. In the presented RSSC-DBOEDL technique, the enhanced MobileNet model is primarily deployed as a feature extractor. The DBO method could be implemented in this study for hyperparameter tuning of the enhanced MobileNet model. The RSSC-DBOEDL technique uses a multi-head attention-based long short-term memory (MHA-LSTM) technique to classify the scenes in the RSI. The simulation evaluation of the RSSC-DBOEDL approach has been examined under the benchmark RSI datasets. The simulation results of the RSSC-DBOEDL approach exhibited a more excellent accuracy outcome of 98.75 % and 95.07 % under UC Merced and EuroSAT datasets with other existing methods regarding distinct measures.

Spatio-Temporal Pruning for Training Ultra-Low-Latency Spiking Neural Networks in Remote Sensing Scene Classification

In remote sensing scene classification (RSSC), restrictions on real-time processing on power consumption, performance, and resources necessitate the compression of neural networks. Unlike artificial neural networks (ANNs), spiking neural networks (SNNs) convey information through spikes, offering superior energy efficiency and biological plausibility. However, the high latency of SNNs restricts their practical application in RSSC. Therefore, there is an urgent need to research ultra-low-latency SNNs. As latency decreases, the performance of the SNN significantly deteriorates. To address this challenge, we propose a novel spatio-temporal pruning method that enhances the feature capture capability of ultra-low-latency SNNs. Our approach integrates spatial fundamental structures during the training process, which are subsequently pruned. We conduct a comprehensive evaluation of the impacts of these structures across classic network architectures, such as VGG and ResNet, demonstrating the generalizability of our method. Furthermore, we develop an ultra-low-latency training framework for SNNs to validate the effectiveness of our approach. In this paper, we successfully achieve high-performance ultra-low-latency SNNs with a single time step for the first time in RSSC. Remarkably, our SNN with one time step achieves at least 200 times faster inference time while maintaining a performance comparable to those of other state-of-the-art methods.

Q-A2NN: Quantized All-Adder Neural Networks for Onboard Remote Sensing Scene Classification

Performing remote sensing scene classification (RSSC) directly on satellites can alleviate data downlink burdens and reduce latency. Compared to convolutional neural networks (CNNs), the all-adder neural network (A2NN) is a novel basic neural network that is more suitable for onboard RSSC, enabling lower computational overhead by eliminating multiplication operations in convolutional layers. However, the extensive floating-point data and operations in A2NNs still lead to significant storage overhead and power consumption during hardware deployment. In this article, a shared scaling factor-based de-biasing quantization (SSDQ) method tailored for the quantization of A2NNs is proposed to address this issue, including a powers-of-two (POT)-based shared scaling factor quantization scheme and a multi-dimensional de-biasing (MDD) quantization strategy. Specifically, the POT-based shared scaling factor quantization scheme converts the adder filters in A2NNs to quantized adder filters with hardware-friendly integer input activations, weights, and operations. Thus, quantized A2NNs (Q-A2NNs) composed of quantized adder filters have lower computational and memory overheads than A2NNs, increasing their utility in hardware deployment. Although low-bit-width Q-A2NNs exhibit significantly reduced RSSC accuracy compared to A2NNs, this issue can be alleviated by employing the proposed MDD quantization strategy, which combines a weight-debiasing (WD) strategy, which reduces performance degradation due to deviations in the quantized weights, with a feature-debiasing (FD) strategy, which enhances the classification performance of Q-A2NNs through minimizing deviations among the output features of each layer. Extensive experiments and analyses demonstrate that the proposed SSDQ method can efficiently quantize A2NNs to obtain Q-A2NNs with low computational and memory overheads while maintaining comparable performance to A2NNs, thus having high potential for onboard RSSC.

Remote Sensing Scene Classification using ConvNeXt-Tiny Model with Attention Mechanism and Label Smoothing

Remote Sensing Scene Classification (RSSC) is the discrete categorization of remote sensing images into various classes of scene categories based on their image content. RSSC plays an important role in many fields, such as agriculture, land mapping, and identification of disaster-prone areas. Therefore, a reliable and accurate RSSC algorithm is required to ensure the accuracy of land identification. Many existing studies in recent years have used deep learning methods, especially CNN combined with attention modules to solve this problem. This study focuses on solving the RSSC problem by proposing a deep learning-based method (CNN) with the ConvNeXt-Tiny model integrated with Efficient Channel Attention Module (ECANet) and label smoothing regularization (LSR). The ConvNeXt-Tiny model shows that a persistent superior outperforms the ‘large’ model in convinced metrics. The ConvNeXt-Tiny model also has a huge advantage in high-precision positioning and higher classification accuracy and localization precision in a variety of complicated scenarios of remote sensing scene recognition. The experiments in this study also aim to prove that the integration of the attention module and LSR into the basic CNN network can improve precision, because the attention module can strengthen important features and weaken features that are less useful for classification. The experimental results proved that the integration of ECANet and LSR in the ConvNeXt-Tiny base network obtained a higher precision of 0.38% in the UC-Merced dataset, 0.7% in the AID, and 0.4% in the WHU-RS19 dataset than the ConvNeXt-Tiny model without ECANet and LSR. The ConvNeXt-Tiny model with ECANet integration and LSR obtained an accuracy of 99.00±0.41% in the UC-Merced dataset, 95.08±0.20% in AID, and 99.50±0.31% in the WHU-RS19 dataset.

A Novel Depth-Wise Separable Convolutional Model for Remote Sensing Scene Classification

A Novel Depth-Wise Separable Convolutional Model for Remote Sensing Scene Classification

Federated Learning Approach for Remote Sensing Scene Classification

In classical machine learning algorithms, used in many analysis tasks, the data are centralized for training. That is, both the model and the data are housed within one device. Federated learning (FL), on the other hand, is a machine learning technique that breaks away from this traditional paradigm by allowing multiple devices to collaboratively train a model without each sharing their own data. In a typical FL setting, each device has a local dataset and trains a local model on that dataset. The local models are next aggregated at a central server to produce a global model. The global model is then distributed back to the devices, which update their local models accordingly. This process is repeated until the global model converges. In this article, a FL approach is applied for remote sensing scene classification for the first time. The adopted approach uses three different RS datasets while employing two types of CNN models and two types of Vision Transformer models, namely: EfficientNet-B1, EfficientNet-B3, ViT-Tiny, and ViT-Base. We compare the performance of FL in each model in terms of overall accuracy and undertake additional experiments to assess their robustness when faced with scenarios of dropped clients. Our classification results on test data show that the two considered Transformer models outperform the two models from the CNN family. Furthermore, employing FL with ViT-Base yields the highest accuracy levels even when the number of dropped clients is significant, indicating its high robustness. These promising results point to the notion that FL can be successfully used with ViT models in the classification of RS scenes, whereas CNN models may suffer from overfitting problems.

Advancements in Self-Supervised Learning for Remote Sensing Scene Classification: Present Innovations and Future Outlooks

Deep learning methodologies have significantly advanced the fields of computer vision and machine learning, enhancing performance across various tasks like classification, regression, and detection. In remote sensing for Earth observation, deep neural networks have propelled state-of-the-art results. However, a major drawback is their dependence on large annotated datasets, necessitating extensive human effort, especially in specialized domains like medical imaging or remote sensing. To mitigate this reliance on annotations, several self-supervised representation learning techniques have emerged, aiming to learn unsupervised image representations applicable to downstream tasks such as image classification, object detection, or semantic segmentation. Consequently, self-supervised learning approaches have gained traction in remote sensing. This article surveys the foundational principles of various self-supervised methods, focusing on scene classification tasks. We elucidate key contributions, analyze experimental setups, and synthesize findings from each study. Furthermore, we conduct comprehensive experiments on two public scene classification datasets to evaluate and benchmark different self-supervised models

Advancements in Self-Supervised Learning for Remote Sensing Scene Classification: Present Innovations and Future Outlooks

Deep learning methodologies have significantly advanced the fields of computer vision and machine learning, enhancing performance across various tasks like classification, regression, and detection. In remote sensing for Earth observation, deep neural networks have propelled state-of-the-art results. However, a major drawback is their dependence on large annotated datasets, necessitating extensive human effort, especially in specialized domains like medical imaging or remote sensing. To mitigate this reliance on annotations, several self-supervised representation learning techniques have emerged, aiming to learn unsupervised image representations applicable to downstream tasks such as image classification, object detection, or semantic segmentation. Consequently, self-supervised learning approaches have gained traction in remote sensing. This article surveys the foundational principles of various self-supervised methods, focusing on scene classification tasks. We elucidate key contributions, analyze experimental setups, and synthesize findings from each study. Furthermore, we conduct comprehensive experiments on two public scene classification datasets to evaluate and benchmark different self-supervised models

Frequency and spatial based multi-layer context network (FSCNet) for remote sensing scene classification

Remote Sensing Scene Classification (RSSC) is an important and challenging research topic due to the variety of land cover sizes and spatial combinations, as well as significant interclass similarity and intraclass variability. Currently, convolutional neural network (CNN)-based methods have been widely used in RSSC tasks with significant results. However, CNNs lack the ability to obtain long-term correlations. Transformer addressed this problem, thanks to the global receptive field of multi-head self-attention (MSA). Nevertheless, the vanilla transformer also needs further improvement to accommodate the diverse in type and scale of objects in RS scenes. In addition, the existing RSSC methods either use the last layer features, which is not conducive to process multi-scale remote sensing images, or directly fuse the multi-layer features, which will bring redundant or mutually exclusive information. To address the above issues, a novel RSSC framework, named frequency and spatial based multi-layer attention network (FSCNet) for remote sensing scene classification is proposed in this article. First, to fully extract the pyramid multi resolution features of CNN, a cross resolution injection model (CRIM) is proposed. Second, to generate better understand of the multilevel features, a frequency and spatial MLP (FS-MLP) is designed. Third, in order to aggregate contextual relations among multi-layer features, a multi-layer context align attention (MCAA) is adopted. The final classification is integration of top-layer feature and aggregated multi-layer feature. The experiment results on three well-known RS scene classification datasets (UCM, AID, and NWPU) prove the effectiveness of FSCNet and it outperforms many state-of-the-art methods.

Optimizing Few-Shot Remote Sensing Scene Classification Based on an Improved Data Augmentation Approach

In the realm of few-shot classification learning, the judicious application of data augmentation methods has a significantly positive impact on classification performance. In the context of few-shot classification tasks for remote sensing images, the augmentation of features and the efficient utilization of limited features are of paramount importance. To address the performance degradation caused by challenges such as high interclass overlap and large intraclass variance in remote sensing image features, we present a data augmentation-based classification optimization method for few-shot remote sensing image scene classification. First, we construct a distortion magnitude space using different types of features, and we perform distortion adjustments on the support set samples while introducing an optimal search for the distortion magnitude (ODS) method. Then, the augmented support set offers a wide array of feature distortions in terms of types and degrees, significantly enhancing the generalization of intrasample features. Subsequently, we devise a dual-path classification (DC) decision strategy, effectively leveraging the discriminative information provided by the postdistortion features to further reduce the likelihood of classification errors. Finally, we evaluate the proposed method using a widely used remote sensing dataset. Our experimental results demonstrate that our approach outperforms benchmark methods, achieving improved classification accuracy.

MSE-Net: A novel master–slave encoding network for remote sensing scene classification

Remote sensing scene (RSS) image classification plays a vital role in various fields such as urban planning and environmental protection. However, due to higher inter-class similarity and intra-class variability, achieving accurate classification for RSS images poses a considerable challenge for current convolutional neural networks (CNNs)-based and visual transformer (ViT)-based methods. To address these issues, this paper proposes a novel dual-encoding method named master–slave encoding network (MSE-Net) from two perspectives of feature extraction and fusion. The master encoder, based on ViT, extracts higher-level semantic features, while the slave encoder, based on CNN, captures relative lower-level spatial structure information. Secondly, to integrate feature information from the two encoders effectively, this paper further develop two fusion strategies. The first strategy involves the auxiliary enhancement units (AEUs), which eliminates semantic divergence between the two encoders, enhances spatial context awareness of the slave encoder and promotes effective feature learning. The interactive perception unit (IPU), as the second strategy, facilitates interaction and integration of the two encoders’ representations to extract more discriminative feature information. In addition, we conducted comparative experiments on four widely-used RSS datasets, including RSSCN7, SIRI-WHU, the aerial image dataset (AID) and NWPU-RESISC45 (NWPU45), to verify the effectiveness of MSE-Net. The experimental results demonstrate that MSE-Net achieved state-of-the-art (SOTA) performance across all the datasets.

Machine-to-Machine Visual Dialoguing with ChatGPT for Enriched Textual Image Description

Image captioning is a technique that enables the automatic extraction of natural language descriptions about the contents of an image. On the one hand, information in the form of natural language can enhance accessibility by reducing the expertise required to process, analyze, and exploit remote sensing images, while on the other, it provides a direct and general form of communication. However, image captioning is usually restricted to a single sentence, which barely describes the rich semantic information that typically characterizes remote sensing (RS) images. In this paper, we aim to move one step forward by proposing a captioning system that, mimicking human behavior, adopts dialogue as a tool to explore and dig for information, leading to more detailed and comprehensive descriptions of RS scenes. The system relies on a questions–answers scheme fed by a query image and summarizes the dialogue content with ChatGPT. Experiments carried out on two benchmark remote sensing datasets confirm the potential of such an approach in the context of semantic information mining. Strengths and weaknesses are highlighted and discussed, as well as some possible future developments.

Self-training guided disentangled adaptation for cross-domain remote sensing image semantic segmentation

Remote sensing (RS) image semantic segmentation using deep convolutional neural networks (DCNNs) has shown great success in various applications. However, the high dependence on annotated data makes it challenging for DCNNs to adapt to different RS scenes. To address this challenge, we propose a cross-domain RS image semantic segmentation task that considers ground sampling distance, remote sensing sensor variation, and different geographical landscapes as the main factors causing domain shifts between source and target images. To mitigate the negative impact of domain shift, we propose a self-training guided disentangled adaptation network (ST-DASegNet) that consists of source and target student backbones to extract source-style and target-style features. To align cross-domain single-style features, we adopt feature-level adversarial learning. We also propose a domain disentangled module (DDM) to extract universal and distinct features from single-domain cross-style features. Finally, we fuse these features and generate predictions using source and target student decoders. Moreover, we employ an exponential moving average (EMA) based cross-domain separated self-training mechanism to ease the instability and disadvantageous effect during adversarial optimization. Our experiments on several prominent RS datasets (Potsdam, Vaihingen, and LoveDA) demonstrate that ST-DASegNet outperforms previous methods and achieves new state-of-the-art results. Visualization and analysis also confirm the interpretability of ST-DASegNet. The code is publicly available at https://github.com/cv516Buaa/ST-DASegNet.

Digital-to-Physical Visual Consistency Optimization for Adversarial Patch Generation in Remote Sensing Scenes

Digital-to-Physical Visual Consistency Optimization for Adversarial Patch Generation in Remote Sensing Scenes

A Deeper Look Into Remote Sensing Scene Image Misclassification by CNNs

A Deeper Look Into Remote Sensing Scene Image Misclassification by CNNs

LGLFormer: Local–Global Lifting Transformer for Remote Sensing Scene Parsing

LGLFormer: Local–Global Lifting Transformer for Remote Sensing Scene Parsing

Gradient Guided Multiscale Feature Collaboration Networks for Few-Shot Class-Incremental Remote Sensing Scene Classification

Gradient Guided Multiscale Feature Collaboration Networks for Few-Shot Class-Incremental Remote Sensing Scene Classification

Weakly Supervised Semantic Segmentation With Consistency-Constrained Multiclass Attention for Remote Sensing Scenes

Weakly Supervised Semantic Segmentation With Consistency-Constrained Multiclass Attention for Remote Sensing Scenes

Gradient-Guided Multi-Scale Focal Attention Network for Remote Sensing Scene Classification

Gradient-Guided Multi-Scale Focal Attention Network for Remote Sensing Scene Classification