Remote Sensing Scene Classification Research Articles

Remote sensing scene classification based on contextual attention mechanism of lie group space

ABSTRACT High-resolution remote sensing (HRRS) scene classification is based on the semantic information in the image and its contextual spatial correlation, labelling different semantic categories for different HRRS. Convolutional neural networks (CNN) have been widely studied and applied in remote sensing scene classification in recent years. However, most of the existing CNN models focus on the global and high-level semantic features, ignoring the shallower feature information. In addition, complex background information and variable scales lead to a series of problems with large intra-class differences and high inter-class similarities, which also brings challenges to scene classification. To address the above problems and challenges, a scenario classification model based on the contextual spatial attention and channel attention mechanism of Lie Group manifold space learning is proposed in this study. In this model, we fully explore the multi-scale features of the scene (shallower-level and high-level) and propose a novel contextual spatial attention mechanism and channel attention mechanism. Extensive experimentation was carried out on the Union Remote Sensing Image Data Set (URSIS), which improved by 7.13% compared with the classical model. The experimental results showed that compared with other state-of-the-art remote sensing scene classification models, our proposed method has achieved significant improvement in classification accuracy and performance.

Teacher-Student Prototype Enhancement Network for a Few-Shot Remote Sensing Scene Classification

Few-shot remote sensing scene classification identifies new classes from limited labeled samples where the great challenges are intraclass diversity, interclass similarity, and limited supervision. To alleviate these problems, a teacher-student prototype enhancement network is proposed for a few-shot remote sensing scene classification. Instead of introducing an attentional mechanism in mainstream studies, a prototype enhancement module is recommended to adaptively select high-confidence query samples, which can enhance the support prototype representations to emphasize intraclass and interclass relationships. The construction of a few-shot teacher model generates more discriminative predictive representations with inputs from many labeled samples, thus providing a strong supervisory signal to the student model and encouraging the network to achieve accurate classification with a limited number of labeled samples. Extensive experiments of four public datasets, including NWPU-remote sens ing image scene classification (NWPU-RESISC45), aerial image dataset (AID), UC Merced, and WHU-RS19, demonstrate that this method achieves superior competitive performance than the state-of-the-art methods on five-way, one-shot, and five-shot classifications.

A multi-scale dense residual correlation network for remote sensing scene classification

Most existing scene classification methods based on remote sensing images tend to ignore important interactive information at different levels in the image. We propose an effective remote sensing scene classification method named multi-scale dense residual correlation network. The method is divided into three parts. First, the multi-stream feature extraction module is introduced which effectively utilizes features at different scales to extract different levels of information. Secondly, the dense residual connected feature fusion technology is proposed, which allows for a wide range of feature fusion. The Correlation Attention Module learn feature representations at multiple levels. This improves classification performance. The method outperforms existing algorithms in terms of effectiveness and accuracy, achieving state-of-the-art results on widely used remote sensing scene classification benchmarks.

Multi-label remote sensing scene classification using two-level double channel spatial attention residual blocks

Multi-label remote sensing scene classification using two-level double channel spatial attention residual blocks

SRL‐ProtoNet: Self‐supervised representation learning for few‐shot remote sensing scene classification

AbstractUsing a deep learning method to classify a large amount of labelled remote sensing scene data produces good performance. However, it is challenging for deep learning based methods to generalise to classification tasks with limited data. Few‐shot learning allows neural networks to classify unseen categories when confronted with a handful of labelled data. Currently, episodic tasks based on meta‐learning can effectively complete few‐shot classification, and training an encoder that can conduct representation learning has become an important component of few‐shot learning. An end‐to‐end few‐shot remote sensing scene classification model based on ProtoNet and self‐supervised learning is proposed. The authors design the Pre‐prototype for a more discrete feature space and better integration with self‐supervised learning, and also propose the ProtoMixer for higher quality prototypes with a global receptive field. The authors’ method outperforms the existing state‐of‐the‐art self‐supervised based methods on three widely used benchmark datasets: UC‐Merced, NWPU‐RESISC45, and AID. Compare with previous state‐of‐the‐art performance. For the one‐shot setting, this method improves by 1.21%, 2.36%, and 0.84% in AID, UC‐Merced, and NWPU‐RESISC45, respectively. For the five‐shot setting, this method surpasses by 0.85%, 2.79%, and 0.74% in the AID, UC‐Merced, and NWPU‐RESISC45, respectively.

Improving remote sensing scene classification using dung Beetle optimization with enhanced deep learning approach

Remote sensing (RS) scene classification has received significant consideration because of its extensive use by the RS community. Scene classification in satellite images has widespread uses in remote surveillance, environmental observation, remote scene analysis, urban planning, and earth observations. Because of the immense benefits of the land scene classification task, various approaches have been presented recently for automatically classifying land scenes from remote sensing images (RSIs). Several approaches dependent upon convolutional neural networks (CNNs) are presented for classifying brutal RS scenes; however, they could only partially capture the context from RSIs due to the problematic texture, cluttered context, tiny size of objects, and considerable differences in object scale. This article designs a Remote Sensing Scene Classification using Dung Beetle Optimization with Enhanced Deep Learning (RSSC-DBOEDL) approach. The purpose of the RSSC-DBOEDL technique is to categorize different varieties of scenes that exist in the RSI. In the presented RSSC-DBOEDL technique, the enhanced MobileNet model is primarily deployed as a feature extractor. The DBO method could be implemented in this study for hyperparameter tuning of the enhanced MobileNet model. The RSSC-DBOEDL technique uses a multi-head attention-based long short-term memory (MHA-LSTM) technique to classify the scenes in the RSI. The simulation evaluation of the RSSC-DBOEDL approach has been examined under the benchmark RSI datasets. The simulation results of the RSSC-DBOEDL approach exhibited a more excellent accuracy outcome of 98.75 % and 95.07 % under UC Merced and EuroSAT datasets with other existing methods regarding distinct measures.

Spatio-Temporal Pruning for Training Ultra-Low-Latency Spiking Neural Networks in Remote Sensing Scene Classification

In remote sensing scene classification (RSSC), restrictions on real-time processing on power consumption, performance, and resources necessitate the compression of neural networks. Unlike artificial neural networks (ANNs), spiking neural networks (SNNs) convey information through spikes, offering superior energy efficiency and biological plausibility. However, the high latency of SNNs restricts their practical application in RSSC. Therefore, there is an urgent need to research ultra-low-latency SNNs. As latency decreases, the performance of the SNN significantly deteriorates. To address this challenge, we propose a novel spatio-temporal pruning method that enhances the feature capture capability of ultra-low-latency SNNs. Our approach integrates spatial fundamental structures during the training process, which are subsequently pruned. We conduct a comprehensive evaluation of the impacts of these structures across classic network architectures, such as VGG and ResNet, demonstrating the generalizability of our method. Furthermore, we develop an ultra-low-latency training framework for SNNs to validate the effectiveness of our approach. In this paper, we successfully achieve high-performance ultra-low-latency SNNs with a single time step for the first time in RSSC. Remarkably, our SNN with one time step achieves at least 200 times faster inference time while maintaining a performance comparable to those of other state-of-the-art methods.

MSCAC: A Multi-Scale Swin–CNN Framework for Progressive Remote Sensing Scene Classification

MSCAC: A Multi-Scale Swin–CNN Framework for Progressive Remote Sensing Scene Classification

SCECNet: self-correction feature enhancement fusion network for remote sensing scene classification

SCECNet: self-correction feature enhancement fusion network for remote sensing scene classification

Q-A2NN: Quantized All-Adder Neural Networks for Onboard Remote Sensing Scene Classification

Performing remote sensing scene classification (RSSC) directly on satellites can alleviate data downlink burdens and reduce latency. Compared to convolutional neural networks (CNNs), the all-adder neural network (A2NN) is a novel basic neural network that is more suitable for onboard RSSC, enabling lower computational overhead by eliminating multiplication operations in convolutional layers. However, the extensive floating-point data and operations in A2NNs still lead to significant storage overhead and power consumption during hardware deployment. In this article, a shared scaling factor-based de-biasing quantization (SSDQ) method tailored for the quantization of A2NNs is proposed to address this issue, including a powers-of-two (POT)-based shared scaling factor quantization scheme and a multi-dimensional de-biasing (MDD) quantization strategy. Specifically, the POT-based shared scaling factor quantization scheme converts the adder filters in A2NNs to quantized adder filters with hardware-friendly integer input activations, weights, and operations. Thus, quantized A2NNs (Q-A2NNs) composed of quantized adder filters have lower computational and memory overheads than A2NNs, increasing their utility in hardware deployment. Although low-bit-width Q-A2NNs exhibit significantly reduced RSSC accuracy compared to A2NNs, this issue can be alleviated by employing the proposed MDD quantization strategy, which combines a weight-debiasing (WD) strategy, which reduces performance degradation due to deviations in the quantized weights, with a feature-debiasing (FD) strategy, which enhances the classification performance of Q-A2NNs through minimizing deviations among the output features of each layer. Extensive experiments and analyses demonstrate that the proposed SSDQ method can efficiently quantize A2NNs to obtain Q-A2NNs with low computational and memory overheads while maintaining comparable performance to A2NNs, thus having high potential for onboard RSSC.

A Scene Classification Model Based on Global-Local Features and Attention in Lie Group Space

The efficient fusion of global and local multi-scale features is quite important for remote sensing scene classification (RSSC). The scenes in high-resolution remote sensing images (HRRSI) contain many complex backgrounds, intra-class diversity, and inter-class similarities. Many studies have shown that global features and local features are helpful for RSSC. The receptive field of a traditional convolution kernel is small and fixed, and it is difficult to capture global features in the scene. The self-attention mechanism proposed in transformer effectively alleviates the above shortcomings. However, such models lack local inductive bias, and the calculation is complicated due to the large number of parameters. To address these problems, in this study, we propose a classification model of global-local features and attention based on Lie Group space. The model is mainly composed of three independent branches, which can effectively extract multi-scale features of the scene and fuse the above features through a fusion module. Channel attention and spatial attention are designed in the fusion module, which can effectively enhance the crucial features in the crucial regions, to improve the accuracy of scene classification. The advantage of our model is that it extracts richer features, and the global-local features of the scene can be effectively extracted at different scales. Our proposed model has been verified on publicly available and challenging datasets, taking the AID as an example, the classification accuracy reached 97.31%, and the number of parameters is 12.216 M. Compared with other state-of-the-art models, it has certain advantages in terms of classification accuracy and number of parameters.

Remote Sensing Scene Classification using ConvNeXt-Tiny Model with Attention Mechanism and Label Smoothing

Remote Sensing Scene Classification (RSSC) is the discrete categorization of remote sensing images into various classes of scene categories based on their image content. RSSC plays an important role in many fields, such as agriculture, land mapping, and identification of disaster-prone areas. Therefore, a reliable and accurate RSSC algorithm is required to ensure the accuracy of land identification. Many existing studies in recent years have used deep learning methods, especially CNN combined with attention modules to solve this problem. This study focuses on solving the RSSC problem by proposing a deep learning-based method (CNN) with the ConvNeXt-Tiny model integrated with Efficient Channel Attention Module (ECANet) and label smoothing regularization (LSR). The ConvNeXt-Tiny model shows that a persistent superior outperforms the ‘large’ model in convinced metrics. The ConvNeXt-Tiny model also has a huge advantage in high-precision positioning and higher classification accuracy and localization precision in a variety of complicated scenarios of remote sensing scene recognition. The experiments in this study also aim to prove that the integration of the attention module and LSR into the basic CNN network can improve precision, because the attention module can strengthen important features and weaken features that are less useful for classification. The experimental results proved that the integration of ECANet and LSR in the ConvNeXt-Tiny base network obtained a higher precision of 0.38% in the UC-Merced dataset, 0.7% in the AID, and 0.4% in the WHU-RS19 dataset than the ConvNeXt-Tiny model without ECANet and LSR. The ConvNeXt-Tiny model with ECANet integration and LSR obtained an accuracy of 99.00±0.41% in the UC-Merced dataset, 95.08±0.20% in AID, and 99.50±0.31% in the WHU-RS19 dataset.

A Novel Depth-Wise Separable Convolutional Model for Remote Sensing Scene Classification

A Novel Depth-Wise Separable Convolutional Model for Remote Sensing Scene Classification

Federated Learning Approach for Remote Sensing Scene Classification

In classical machine learning algorithms, used in many analysis tasks, the data are centralized for training. That is, both the model and the data are housed within one device. Federated learning (FL), on the other hand, is a machine learning technique that breaks away from this traditional paradigm by allowing multiple devices to collaboratively train a model without each sharing their own data. In a typical FL setting, each device has a local dataset and trains a local model on that dataset. The local models are next aggregated at a central server to produce a global model. The global model is then distributed back to the devices, which update their local models accordingly. This process is repeated until the global model converges. In this article, a FL approach is applied for remote sensing scene classification for the first time. The adopted approach uses three different RS datasets while employing two types of CNN models and two types of Vision Transformer models, namely: EfficientNet-B1, EfficientNet-B3, ViT-Tiny, and ViT-Base. We compare the performance of FL in each model in terms of overall accuracy and undertake additional experiments to assess their robustness when faced with scenarios of dropped clients. Our classification results on test data show that the two considered Transformer models outperform the two models from the CNN family. Furthermore, employing FL with ViT-Base yields the highest accuracy levels even when the number of dropped clients is significant, indicating its high robustness. These promising results point to the notion that FL can be successfully used with ViT models in the classification of RS scenes, whereas CNN models may suffer from overfitting problems.

A class distribution learning method for few-shot remote sensing scene classification

ABSTRACT In recent years, few-shot remote sensing scene classification has become an important foundational research topic, aiming to achieve better classification performance with limited labeled data. Due to the large intra-class variances and inter-class similarity of remote sensing scenes, the scene classification task for remote sensing is much more challenging than general few-shot image classification. In this letter, we propose a class distribution learning method under limited samples by fully using the support samples, which consists of intra-class feature aggregation (IFA) module, intra-class feature homogenization module (IFH) and inter-class cross calibration (ICC) module. IFA and IFH modules are proposed to enable the aggregation and even distribution of intra-class features around the prototypes generated by the model, reducing intra-class distances and mitigating the generation of extreme samples near decision boundaries. ICC module is designed to encourage the features generated by the model for inter-class to be further apart, thereby increasing the inter-class distance and reducing the probability of classification errors. To demonstrate the effectiveness of our proposed method, we conduct comprehensive experiments on three benchmark remote sensing scene datasets: NWPU-RESISC45, WHU-RS19, and UC Merced, achieving extremely advanced performance. Our source codes have been available at https://github.com/mitsuha97/CDLM.

Evaluation of Ten Deep-Learning-Based Out-of-Distribution Detection Methods for Remote Sensing Image Scene Classification

Although deep neural networks have made significant progress in tasks related to remote sensing image scene classification, most of these tasks assume that the training and test data are independently and identically distributed. However, when remote sensing scene classification models are deployed in the real world, the model will inevitably encounter situations where the distribution of the test set differs from that of the training set, leading to unpredictable errors during the inference and testing phase. For instance, in the context of large-scale remote sensing scene classification applications, it is difficult to obtain all the feature classes in the training phase. Consequently, during the inference and testing phases, the model will categorize images of unidentified unknown classes into known classes. Therefore, the deployment of out-of-distribution (OOD) detection within the realm of remote sensing scene classification is crucial for ensuring the reliability and safety of model application in real-world scenarios. Despite significant advancements in OOD detection methods in recent years, there remains a lack of a unified benchmark for evaluating various OOD methods specifically in remote sensing scene classification tasks. We designed different benchmarks on three classical remote sensing datasets to simulate scenes with different distributional shift. Ten different types of OOD detection methods were employed, and their performance was evaluated and compared using quantitative metrics. Numerous experiments were conducted to evaluate the overall performance of these state-of-the-art OOD detection methods under different test benchmarks. The comparative results show that the virtual-logit matching methods without additional training outperform the other types of methods on our benchmarks, suggesting that additional training methods are unnecessary for remote sensing image scene classification applications. Furthermore, we provide insights into OOD detection models and performance enhancement in real world. To the best of our knowledge, this study is the first evaluation and analysis of methods for detecting out-of-distribution data in remote sensing. We hope that this research will serve as a fundamental resource for future studies on out-of-distribution detection in remote sensing.

Advancements in Self-Supervised Learning for Remote Sensing Scene Classification: Present Innovations and Future Outlooks

Deep learning methodologies have significantly advanced the fields of computer vision and machine learning, enhancing performance across various tasks like classification, regression, and detection. In remote sensing for Earth observation, deep neural networks have propelled state-of-the-art results. However, a major drawback is their dependence on large annotated datasets, necessitating extensive human effort, especially in specialized domains like medical imaging or remote sensing. To mitigate this reliance on annotations, several self-supervised representation learning techniques have emerged, aiming to learn unsupervised image representations applicable to downstream tasks such as image classification, object detection, or semantic segmentation. Consequently, self-supervised learning approaches have gained traction in remote sensing. This article surveys the foundational principles of various self-supervised methods, focusing on scene classification tasks. We elucidate key contributions, analyze experimental setups, and synthesize findings from each study. Furthermore, we conduct comprehensive experiments on two public scene classification datasets to evaluate and benchmark different self-supervised models

Advancements in Self-Supervised Learning for Remote Sensing Scene Classification: Present Innovations and Future Outlooks

Deep learning methodologies have significantly advanced the fields of computer vision and machine learning, enhancing performance across various tasks like classification, regression, and detection. In remote sensing for Earth observation, deep neural networks have propelled state-of-the-art results. However, a major drawback is their dependence on large annotated datasets, necessitating extensive human effort, especially in specialized domains like medical imaging or remote sensing. To mitigate this reliance on annotations, several self-supervised representation learning techniques have emerged, aiming to learn unsupervised image representations applicable to downstream tasks such as image classification, object detection, or semantic segmentation. Consequently, self-supervised learning approaches have gained traction in remote sensing. This article surveys the foundational principles of various self-supervised methods, focusing on scene classification tasks. We elucidate key contributions, analyze experimental setups, and synthesize findings from each study. Furthermore, we conduct comprehensive experiments on two public scene classification datasets to evaluate and benchmark different self-supervised models

LPNet: A remote sensing scene classification method based on large kernel convolution and parameter fusion

LPNet: A remote sensing scene classification method based on large kernel convolution and parameter fusion

Frequency and spatial based multi-layer context network (FSCNet) for remote sensing scene classification

Remote Sensing Scene Classification (RSSC) is an important and challenging research topic due to the variety of land cover sizes and spatial combinations, as well as significant interclass similarity and intraclass variability. Currently, convolutional neural network (CNN)-based methods have been widely used in RSSC tasks with significant results. However, CNNs lack the ability to obtain long-term correlations. Transformer addressed this problem, thanks to the global receptive field of multi-head self-attention (MSA). Nevertheless, the vanilla transformer also needs further improvement to accommodate the diverse in type and scale of objects in RS scenes. In addition, the existing RSSC methods either use the last layer features, which is not conducive to process multi-scale remote sensing images, or directly fuse the multi-layer features, which will bring redundant or mutually exclusive information. To address the above issues, a novel RSSC framework, named frequency and spatial based multi-layer attention network (FSCNet) for remote sensing scene classification is proposed in this article. First, to fully extract the pyramid multi resolution features of CNN, a cross resolution injection model (CRIM) is proposed. Second, to generate better understand of the multilevel features, a frequency and spatial MLP (FS-MLP) is designed. Third, in order to aggregate contextual relations among multi-layer features, a multi-layer context align attention (MCAA) is adopted. The final classification is integration of top-layer feature and aggregated multi-layer feature. The experiment results on three well-known RS scene classification datasets (UCM, AID, and NWPU) prove the effectiveness of FSCNet and it outperforms many state-of-the-art methods.