Remote Sensing Scene Classification Research Articles

An Explainable Spatial–Frequency Multiscale Transformer for Remote Sensing Scene Classification

Deep convolutional neural networks (CNNs) are significant in remote sensing. Due to the strong local representation learning ability, CNNs have excellent performance in remote sensing scene classification. However, CNNs focus on location-sensitive representations in the spatial domain and lack contextual information mining capabilities. Meanwhile, remote sensing scene classification still faces challenges, such as complex scenes and significant differences in target sizes. To address the problems and challenges above, more robust feature representation learning networks are necessary. In this paper, a novel and explainable spatial-frequency multi-scale Transformer framework, SF-MSFormer, is proposed for remote sensing scene classification. It mainly comprises spatial-domain and frequency-domain multi-scale Transformer branches, which consider the spatial-frequency global multi-scale representation features. Besides, the texture-enhanced encoder is designed in the frequency-domain multi-scale Transformer branch, which is adaptive to capture the global texture features. In addition, an adaptive feature aggregation module is designed to integrate the spatial-frequency multi-scale feature for final recognition. The experimental results verify the effectiveness of SF-MSFormer and show better convergence. It achieves state-of-the-art results (98.72%, 98.6%, 99.72%, and 94.83% overall accuracies, respectively) on the AID, UCM, WHU-RS19, and NWPU-RESISC45 datasets. Besides, the feature visualizations evaluate the explainability of the texture-enhanced encoder. The code implementation of this article will be available at https://github.com/yutinyang/SF-MSFormer.

Resolution-Agnostic Remote Sensing Scene Classification With Implicit Neural Representations

Remote sensing scene classification is an important yet challenging task. In recent years, the excellent feature representation ability of convolutional neural networks (CNNs) has led to substantial improvements in scene classification accuracy. However, handling resolution variations of remote sensing images is still challenging because CNNs are not inherently capable of modeling multiresolution input images. In this letter, we propose a novel scene classification method with scale and resolution adaptation ability by leveraging the recent advances in implicit neural representations (INRs). Unlike previous CNN-based methods that make predictions based on rasterized image inputs, the proposed method converts the images as continuous functions with INRs optimization and then performs classification within the function space. When the image is represented as a function, the image resolution can be decoupled from the pixel values so that the resolution does not have much impact on the classification performance. Our method also shows great potential for multiresolution remote sensing scene classification. Using only a simple multilayer perceptron (MLP) classifier in the proposed function space, our method achieves classification accuracy comparable to deep CNNs but exhibits better adaptability to image scale and resolution changes.

Double Discriminative Constraint-Based Affine Nonnegative Representation for Few-Shot Remote Sensing Scene Classification

Remote sensing scene classification (RSSC) has recently attracted more attention. However, due to restrictions in the imaging environment and equipment, it is difficult to get a large number of labeled images in remote sensing. This has led to the emergence of few-shot learning for RSSC, which aims to achieve better performance with few labeled samples. Remote sensing images’ large interclass similarity may cause classification confusion. To overcome this issue, this study proposes a double discriminative constraints-based affine nonnegative representation for few-shot RSSC. To be specific, we devise a novel representation-based classifier with two discriminative constraint terms in the objective function and utilize affine nonnegative constraints to restrict the learned parameters. These constraints reduce the correlation between classes and strengthen the class specificity of the learned parameters. Experiments on benchmark datasets demonstrate the effectiveness of our method.

DBGA-Net: Dual-Branch Global–Local Attention Network for Remote Sensing Scene Classification

Remote sensing scene classification (RSSC) is a fundamental but challenging task in the discipline of remote sensing (RS). Significant recent advancements have been achieved in RSSC utilizing convolution neural network. The increasing amount of RS images has resulted in two tough problems: intra-class diversity and inter-class similarity. Existing approaches cannot effectively concentrate on both global and local key features simultaneously. Therefore, in this letter, we propose a dual-branch global-local attention network (DBGA-Net) to solve this problem. Specifically, effective complementary features are first extracted from two distinct models. Following that, it was designed that global-local attention module, which concurrently focuses on both the entire image global features and local pertinent parts within it, would be necessary to successfully extract valid information from RS images. In the end, the fusion module efficiently combines the extracted complementary features to yield more comprehensive scene information. To verify the effectiveness of DBGA-Net, experiments are conducted on three scene classification datasets, 99.86%, 97.60% and 96.06% experimental results are obtained, respectively, with significant classification performance of the proposed method compared with the state-of-the-art (SOTA) method. Our code and images are available at https://github.com/ZhouYao1020/DBGANet.

Multigranularity Decoupling Network With Pseudolabel Selection for Remote Sensing Image Scene Classification

The existing deep networks have shown excellent performance in remote sensing scene classification, which generally requires a large amount of class-balanced training samples. However, deep networks will result in underfitting with imbalanced training samples since they can easily bias toward the majority classes. To address these problems, a multi-granularity decoupling network (MGDNet) is proposed for remote sensing image scene classification. To begin with, we design a multi-granularity complementary feature representation (MGCFR) method to extract fine-grained features from remote sensing images, which utilizes region-level supervision to guide the attention of the decoupling network. Second, a class-imbalanced pseudo-label selection (CIPS) approach is proposed to evaluate the credibility of unlabeled samples. Finally, the diversity component feature (DCF) loss function is developed to force the local features to be more discriminative. Our model performs satisfactorily on three public datasets: UC Merced (UCM), NWPU-RESISC45, and Aerial Image Dataset (AID). Experimental results show that the proposed model yields superior performance compared with other state-of-the-art methods.

IDN: inner-class dense neighbours for semi-supervised learning-based remote sensing scene classification

ABSTRACT In remote sensing scene classification, manual annotation is time-consuming and expensive to collect, which limits the effective deployment of supervised classification networks. Semi-supervised learning (SSL), which can benefit from unlabelled and labelled samples simultaneously, attracts much attention. In this letter, we introduce a cluster group construction method for semi-supervised high-resolution remote sensing (HRRS) scene classification. The low-density separation assumption, which assumes that the decision boundary of a classification network should lie in the low-density region, is a vital assumption to guide the construction of clusters. Following this, we propose to construct inner-class dense neighbours (IDN) for the semi-supervised classification framework. The proposed IDN can increase the density of clusters for a better determination of high-density regions to help the network training in the SSL framework. The experiments on the NWPU-RESIS45 and RSSCN7 datasets demonstrate the effectiveness of our proposed method, which show that the performance of the classification network with our proposed IDN has a prominent classification accuracy improvement.

Local and Long-Range Collaborative Learning for Remote Sensing Scene Classification

With the development of high-resolution satellites, more and more attention has been paid to remote sensing (RS) scene classification. Convolutional neural networks (CNNs), which replace the traditional handcrafted features with a learning-based feature extraction mechanism, are widely used in scene classification. But CNNs are less effective in deriving long-range contextual relations, which limits the further improvement. Visual transformer (VT), an emerging image processing method, provides a new perspective for RS scene classification by directly acquiring long-range features. Although there have been limited works combining CNN and VT through simple concatenation, the collaborations between them are insufficient. To address these issues, we propose a local and long-range collaborative framework (L2RCF). First, we design a dual-stream structure to extract the local and long-range features. Second, a cross-feature calibration (CFC) module is designed for them to improve representation of the fusion features. Then, combining deep supervision (DS) and deep mutual learning (DML), a novel joint loss is proposed to enhance the dual-stream feature extractor and further improve the fused features. Finally, a two-stage semi-supervised training strategy is designed to improve performance with unlabeled samples. To demonstrate the effectiveness of L2RCF, we conducted experiments on three widely used RS scene classification data sets: RSSCN7, AID, and NWPU. The results show that L2RCF performs significantly better compared with some state-of-the-art scene classification methods.

Remote-Sensing Scene Classification via Multistage Self-Guided Separation Network

Remote-Sensing Scene Classification via Multistage Self-Guided Separation Network

Co-Compression via Superior Gene for Remote Sensing Scene Classification

Convolutional neural networks (CNNs) has been successfully employed in remote sensing image classification because of their robust feature representation for different visual tasks and powerful graphics processing units. The attendant problem is that high computational cost and high memory footprint hindering the application of CNNs for remote sensing applications in resource- and time-sensitive situations. Based on practical deployment requirements, we pioneer a pruning-quantization joint learning model compression method for remote sensing image classification, called Co-Compression via Superior Gene (CC-SG). An Enhanced Evolution Algorithm (EEA) is adopted as the agent to search a “superior gene”, and immediately following, a director receives the “superior gene” and gives a compression mask and a resource constraint feedback to the agent. The network is eventually compressed and fine-tuned according to the optimal compression mask. Specifically, we introduce gene age and progressive shrinkage mutation rate to EEA, and design a fitness function that balances accuracy and resource constraints. As validated using the UC Merced land-use and NWPU-RESISC45 datasets, the proposed CC-SG demonstrated superiority over other compared model compression approaches. For example, CC-SG attained substantial Bit operations (BOPs) compression ratio of 40.04 with 0.956% accuracy increase for VGG-16 on UC Merced land-use dataset and 40.00 with 0.203% accuracy increase for ResNet-56 on NWPU-RESISC45 dataset. The code is available at https://github.com/fanxxxxyi/CC-SG.

Multipretext-Task Prototypes Guided Dynamic Contrastive Learning Network for Few-Shot Remote Sensing Scene Classification

Multipretext-Task Prototypes Guided Dynamic Contrastive Learning Network for Few-Shot Remote Sensing Scene Classification

Contextual Spatial-Channel Attention Network for Remote Sensing Scene Classification

Contextual Spatial-Channel Attention Network for Remote Sensing Scene Classification

Reconstruction-Assisted and Distance-Optimized Adversarial Training: A Defense Framework for Remote Sensing Scene Classification

Reconstruction-Assisted and Distance-Optimized Adversarial Training: A Defense Framework for Remote Sensing Scene Classification

All Adder Neural Networks for On-Board Remote Sensing Scene Classification

All Adder Neural Networks for On-Board Remote Sensing Scene Classification

MFGNet: Multibranch Feature Generation Networks for Few-Shot Remote Sensing Scene Classification

Few-shot remote sensing scene classification aims to identify unseen classes using only a small number of labeled samples. Considering the large intra-class variances and inter-class similarity of remote sensing scenes, most existing methods focus on feature extraction, ignoring the overfitting problem caused by insufficient samples. To this end, we propose a novel few-shot learning framework, called multibranch feature generation networks (MFGNets), which solves the few-shot scene classification from the source by online sample generation at the representation space. Specifically, we first build a feature generation net to transform the few-shot classification into a regular classification problem, in which the generated samples are achieved by combining the class-specific features with the sampled intra-class features. Then, to ensure the quality of the generated samples, we introduce two novel regularization terms: the intra-class diversity loss (ID-Loss) and the inter-class consistency loss (IC-Loss), which aid the model in generating more diverse samples. Furthermore, we introduce a scale-angle aware self-supervised pretext to learn scale-invariant and rotation-invariant features, improving the model’s feature representation capability in remote sensing scenes. We evaluate the proposed method on three publicly available datasets, namely UC_Merced, NWPU-RESISC45, and AID. Our approach has achieved state-of-the-art performance, with an improvement of more than 3.31%, 2.64%, and 6.86% on the most challenging 1-shot tasks, respectively.

A Self-Supervised-Driven Open-Set Unsupervised Domain Adaptation Method for Optical Remote Sensing Image Scene Classification and Retrieval

Unsupervised domain adaptation (UDA) is an important solution to reduce the bias between the labeled source domain and the unlabeled target domain. It has attracted more attention for optical remote sensing image scene classification and retrieval. Currently, most of the previous work is devoted to closed-set UDA. In fact, the target domain often contains unknown classes. Moreover, some open UDA methods mine structural information of the target domain directly from the type knowledge of the source domain, and less directly from the unlabeled data of the target domain. In this paper, we propose a new self-supervised-driven open-set UDA method combining contrastive self-supervised learning with consistency self-training for optical remote sensing scene classification and retrieval. Specifically, a contrastive self-supervised learning network is introduced to learn discriminative features from the unlabeled target domain data. Moreover, a novel open-set class learning module is developed based on two-level confidence rules and the consistency self-training strategy, which can obtain reliable unknown class samples for co-training. Finally, an open-set dataset including six cross-domain scenarios is constructed based on three public datasets and several experiments are conducted with eleven state-of-the-art domain adaptation methods. Experimental results demonstrate that our proposed method achieves superior performances on the six open-set cross-domain scenarios in both scene classification and retrieval. Especially, our method improves the overall classification accuracies by 9.72% to 24.06% and improve mean average retrieval precisions by 8.06% to 16.21% on the complex UCMD (source domain) → NWPU (target domain) scenario, compared with the other eleven state-of-the-art methods.

Few-shot remote sensing scene classification based on multi subband deep feature fusion.

Recently, convolutional neural networks (CNNs) have performed well in object classification and object recognition. However, due to the particularity of geographic data, the labeled samples are seriously insufficient, which limits the practical application of CNN methods in remote sensing (RS) image processing. To address the problem of small sample RS image classification, a discrete wavelet-based multi-level deep feature fusion method is proposed. First, the deep features are extracted from the RS images using pre-trained deep CNNs and discrete wavelet transform (DWT) methods. Next, a modified discriminant correlation analysis (DCA) approach is proposed to distinguish easily confused categories effectively, which is based on the distance coefficient of between-class. The proposed approach can effectively integrate the deep feature information of various frequency bands. Thereby, the proposed method obtains the low-dimensional features with good discrimination, which is demonstrated through experiments on four benchmark datasets. Compared with several state-of-the-art methods, the proposed method achieves outstanding performance under limited training samples, especially one or two training samples per class.

Hierarchical Feature Fusion of Transformer With Patch Dilating for Remote Sensing Scene Classification

Hierarchical Feature Fusion of Transformer With Patch Dilating for Remote Sensing Scene Classification

Few-Shot Remote Sensing Scene Classification With Spatial Affinity Attention and Class Surrogate-Based Supervised Contrastive Learning

Few-Shot Remote Sensing Scene Classification With Spatial Affinity Attention and Class Surrogate-Based Supervised Contrastive Learning

Energy-Based CNN Pruning for Remote Sensing Scene Classification

Convolutional neural networks (CNNs) have been adopted to classify the remote sensing scene image. However, the application of these complicated networks on the satellite platform is difficult because of the limited computation resources. Therefore, we propose an energy-based filter pruning framework (EFPF) to reduce the size of the original model. The energy can be obtained through the eigenvalues of each weight tensor by singular value decomposition (SVD). Specifically, we calculate the energy of each layer by the ratio of eigenvalues that are lower than a specified truncation parameter and then remove filters from the original layer in light of the degree of energy. The EFPF is reliable because SVD techniques can capture the covariance among all filters from the original weight tensor, and therefore, the energy from the eigenvalues can reflect the redundancy of the filters. (i.e., if the distribution of eigenvalues is sharp, then the energy among filters will be lower, and the redundancy will be higher.) Surprisingly, the EFPF can reduce the FLOPs and parameters, as well as improve the top1 accuracy obviously when the VGG-16 and ResNet-50 are adopted to classify the AID, NWPU45, PatternNet, and WHU19 datasets. Additionally, the EFPF can achieve similar pruning results when the original model is fully-trained (converge) and under-trained (In-converge), which means we can save the training computation resources for the original model.

Exploring Hard Samples in Multiview for Few-Shot Remote Sensing Scene Classification

Exploring Hard Samples in Multiview for Few-Shot Remote Sensing Scene Classification