Remote Sensing Image Scene Classification Research Articles

Evaluation of Ten Deep-Learning-Based Out-of-Distribution Detection Methods for Remote Sensing Image Scene Classification

Although deep neural networks have made significant progress in tasks related to remote sensing image scene classification, most of these tasks assume that the training and test data are independently and identically distributed. However, when remote sensing scene classification models are deployed in the real world, the model will inevitably encounter situations where the distribution of the test set differs from that of the training set, leading to unpredictable errors during the inference and testing phase. For instance, in the context of large-scale remote sensing scene classification applications, it is difficult to obtain all the feature classes in the training phase. Consequently, during the inference and testing phases, the model will categorize images of unidentified unknown classes into known classes. Therefore, the deployment of out-of-distribution (OOD) detection within the realm of remote sensing scene classification is crucial for ensuring the reliability and safety of model application in real-world scenarios. Despite significant advancements in OOD detection methods in recent years, there remains a lack of a unified benchmark for evaluating various OOD methods specifically in remote sensing scene classification tasks. We designed different benchmarks on three classical remote sensing datasets to simulate scenes with different distributional shift. Ten different types of OOD detection methods were employed, and their performance was evaluated and compared using quantitative metrics. Numerous experiments were conducted to evaluate the overall performance of these state-of-the-art OOD detection methods under different test benchmarks. The comparative results show that the virtual-logit matching methods without additional training outperform the other types of methods on our benchmarks, suggesting that additional training methods are unnecessary for remote sensing image scene classification applications. Furthermore, we provide insights into OOD detection models and performance enhancement in real world. To the best of our knowledge, this study is the first evaluation and analysis of methods for detecting out-of-distribution data in remote sensing. We hope that this research will serve as a fundamental resource for future studies on out-of-distribution detection in remote sensing.

Learning scene-vectors for remote sensing image scene classification

Representing the scenes by learning the subtle variations in the spatial content of different classes is crucial for scene classification in remote sensing images. In this paper, we propose a scene attribute modeling to obtain a discriminative and compact representation for scene classification. First, we construct scene attribute model (SAM) by training a Gaussian mixture model (GMM) using convolutional features to capture the scene attributes implicitly. Then, we perform a maximum a posteriori (MAP) adaptation to enhance the contribution of significant attributes in each scene resulting in a high-dimensional feature vector which contains redundant attributes from all the scenes. Hence, we use factor analysis to obtain a compact representation of the high-dimensional feature vector termed scene-vector, which retains only the significant attributes specific to a scene. The proposed approach is demonstrated on three benchmark datasets, namely, UC Merced Land Use, AID, and NWPU-RESISC45 datasets. We further show that, being a compact representation, our scene-vector outperforms state-of-the-art methods for scene classification in remote sensing images.

On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)

Large pre-trained models, also known as foundation models (FMs), are trained in a task-agnostic manner on large-scale data and can be adapted to a wide range of downstream tasks by fine-tuning, few-shot, or even zero-shot learning. Despite their successes in language and vision tasks, we have yet seen an attempt to develop foundation models for geospatial artificial intelligence (GeoAI). In this work, we explore the promises and challenges of developing multimodal foundation models for GeoAI. We first investigate the potential of many existing FMs by testing their performances on seven tasks across multiple geospatial domains including Geospatial Semantics, Health Geography, Urban Geography, and Remote Sensing. Our results indicate that on several geospatial tasks that only involve text modality such as toponym recognition, location description recognition, and US state-level/county-level dementia time series forecasting, the task-agnostic LLMs can outperform task-specific fully-supervised models in a zero-shot or few-shot learning setting. However, on other geospatial tasks, especially tasks that involve multiple data modalities (e.g., POI-based urban function classification, street view image-based urban noise intensity classification, and remote sensing image scene classification), existing foundation models still underperform task-specific models. Based on these observations, we propose that one of the major challenges of developing a foundation model for GeoAI is to address the multimodality nature of geospatial tasks. After discussing the distinct challenges of each geospatial data modality, we suggest the possibility of a multimodal foundation model which can reason over various types of geospatial data through geospatial alignments. We conclude this paper by discussing the unique risks and challenges to develop such a model for GeoAI.

Integrating category-related key regions with a dual-stream network for remote sensing scene classification

Remote sensing image scene classification has made great progress with deep learning. Due to complex backgrounds and the large number of objects with inhomogeneous sizes, remote sensing image scene classification still remains challenging. In fact, only a few parts of regions are expected to be representative of the scene. In this paper, we propose a dual-stream framework for remote sensing classification with the help of category-related regions. Swin Transformer-based global feature extractor is used to produce coarse classification results. A category-related key region localization module is developed to extract the representative region from the whole image. The ResNet50 is employed as a local feature extractor to mine critical features from prominent regions. Finally, the classification results is obtained with the shared weighted classify head on global-local features. Extensive experiments show that our method provides a promising approach for key region locations and performs best on three public remote sensing datasets.

A Few-Shot Semi-Supervised Learning Method for Remote Sensing Image Scene Classification

Few-shot scene classification methods aim to obtain classification discriminative ability from few labeled samples and has recently seen substantial advancements. However, the current few-shot learning approaches still suffer from overfitting due to the scarcity of labeled samples. To this end, a few-shot semi-supervised method is proposed to address this issue. Specifically, semi-supervised learning method is used to increase target domain samples; then we train multiple classification models using the augmented samples. Finally, we perform decision fusion of the results obtained from the multiple models to accomplish the image classification task. According to the experiments conducted on two real few-shot remote sensing scene datasets, our proposed method achieves significantly higher accuracy (approximately 1.70% to 4.33%) compared to existing counterparts.

E2SCNet: Efficient Multiobjective Evolutionary Automatic Search for Remote Sensing Image Scene Classification Network Architecture.

Remote sensing image scene classification methods based on deep learning have been widely studied and discussed. However, most of the network architectures are directly reliant on natural image processing methods and are fixed. A few studies have focused on automatic search mechanisms, but they cannot weigh the interpretation accuracy and the parameter quantity for practical application. As a result, automatic global search methods based on multiobjective evolutionary computation have more advantages. However, in the ranking process, the network individuals with large parameter quantities are easy to eliminate, but a higher accuracy may be obtained after full training. In addition, evolutionary neural architecture search methods often take several days. In this article, in order to solve the above concerns, we propose an efficient multiobjective evolutionary automatic search framework for remote sensing image scene classification deep learning network architectures (E2SCNet). In E2SCNet, eight kinds of lightweight operators are used to build a diversified search space, and the coding connection mode is flexible. In the search process, a large model retention mechanism is implemented through two-step multiobjective modeling and evolutionary search, where one step involves the "parameter quantity and accuracy", and the other step involves the "parameter quantity and accuracy growth quantity." Moreover, a super network is constructed to share the weight in the process of individual network evaluation and promote the search speed. The effectiveness of E2SCNet is proven by comparison with several networks designed by human experts and networks obtained by gradient and evolutionary computing-based search methods.

Class-Aware Self-Distillation for Remote Sensing Image Scene Classification

Class-Aware Self-Distillation for Remote Sensing Image Scene Classification

In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification

In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification

Label Embedding based on Inter-class Correlation Mining for Remote Sensing Image Scene Classification

Label Embedding based on Inter-class Correlation Mining for Remote Sensing Image Scene Classification

Mining Contrastive Relations Between Cross-Modal Features for Zero-Shot Remote Sensing Image Scene Classification

Mining Contrastive Relations Between Cross-Modal Features for Zero-Shot Remote Sensing Image Scene Classification

MMPC-Net: Multigranularity and Multiscale Progressive Contrastive Learning Neural Network for Remote Sensing Image Scene Classification

MMPC-Net: Multigranularity and Multiscale Progressive Contrastive Learning Neural Network for Remote Sensing Image Scene Classification

Hierarchical Feature Progressive Alignment Network for Remote Sensing Image Scene Classification in Multitarget Domain Adaptation

Hierarchical Feature Progressive Alignment Network for Remote Sensing Image Scene Classification in Multitarget Domain Adaptation

Semi-supervised remote sensing image scene classification with prototype-based consistency

Deep learning significantly improves the accuracy of remote sensing image scene classification, benefiting from the large-scale datasets. However, annotating the remote sensing images is time-consuming and even tough for experts. Deep neural networks trained using a few labeled samples usually generalize less to new unseen images. In this paper, we propose a semi-supervised approach for remote sensing image scene classification based on the prototype-based consistency, by exploring massive unlabeled images. To this end, we, first, propose a feature enhancement module to extract discriminative features. This is achieved by focusing the model on the foreground areas. Then, the prototype-based classifier is introduced to the framework, which is used to acquire consistent feature representations. We conduct a series of experiments on NWPU-RESISC45 and Aerial Image Dataset (AID). Our method improves the State-Of-The-Art (SOTA) method on NWPU-RESISC45 from 92.03% to 93.08% and on AID from 94.25% to 95.24% in terms of accuracy.

Remote sensing image scene classification based on stackable attention structure

This work proposes a remote sensing image scene classification method based on stackable attention structure. This method utilizes a stackable attention structure to represent the feature information in remote sensing images as a linear function, changing the previous processing methods for global and local information in remote sensing images to avoid the need for memory intensive attention maps. Through experiments, it has been shown that the scene classification method we proposed based on stackable attention structure has higher accuracy and smaller model size than ResNet152. This provides a new approach for constructing remote sensing image scene classification models, especially for processing large-sized remote sensing images.

Deep Learning for Remote Sensing Image Scene Classification: A Review and Meta-Analysis

Remote sensing image scene classification with deep learning (DL) is a rapidly growing field that has gained significant attention in the past few years. While previous review papers in this domain have been confined to 2020, an up-to-date review to show the progression of research extending into the present phase is lacking. In this review, we explore the recent articles, providing a thorough classification of approaches into three main categories: Convolutional Neural Network (CNN)-based, Vision Transformer (ViT)-based, and Generative Adversarial Network (GAN)-based architectures. Notably, within the CNN-based category, we further refine the classification based on specific methodologies and techniques employed. In addition, a novel and rigorous meta-analysis is performed to synthesize and analyze the findings from 50 peer-reviewed journal articles to provide valuable insights in this domain, surpassing the scope of existing review articles. Our meta-analysis shows that the most adopted remote sensing scene datasets are AID (41 articles) and NWPU-RESISC45 (40). A notable paradigm shift is seen towards the use of transformer-based models (6) starting from 2021. Furthermore, we critically discuss the findings from the review and meta-analysis, identifying challenges and future opportunities for improvement in this domain. Our up-to-date study serves as an invaluable resource for researchers seeking to contribute to this growing area of research.

Remote Sensing Image Scene Classification in Hybrid Classical-Quantum Transferring CNN with Small Samples.

The scope of this research lies in the combination of pre-trained Convolutional Neural Networks (CNNs) and Quantum Convolutional Neural Networks (QCNN) in application to Remote Sensing Image Scene Classification(RSISC). Deep learning (RL) is improving by leaps and bounds pretrained CNNs in Remote Sensing Image (RSI) analysis, and pre-trained CNNs have shown remarkable performance in remote sensing image scene classification (RSISC). Nonetheless, CNNs training require massive, annotated data as samples. When labeled samples are not sufficient, the most common solution is using pre-trained CNNs with a great deal of natural image datasets (e.g., ImageNet). However, these pre-trained CNNs require a large quantity of labelled data for training, which is often not feasible in RSISC, especially when the target RSIs have different imaging mechanisms from RGB natural images. In this paper, we proposed an improved hybrid classical-quantum transfer learning CNNs composed of classical and quantum elements to classify open-source RSI dataset. The classical part of the model is made up of a ResNet network which extracts useful features from RSI datasets. To further refine the network performance, a tensor quantum circuit is subsequently employed by tuning parameters on near-term quantum processors. We tested our models on the open-source RSI dataset. In our comparative study, we have concluded that the hybrid classical-quantum transferring CNN has achieved better performance than other pre-trained CNNs based RSISC methods with small training samples. Moreover, it has been proven that the proposed algorithm improves the classification accuracy while greatly decreasing the amount of model parameters and the sum of training data.

A Novel Discriminative Enhancement Method for Few-Shot Remote Sensing Image Scene Classification

Remote sensing image scene classification (RSISC) has garnered significant attention in recent years. Numerous methods have been put forward in an attempt to tackle this issue, particularly leveraging deep learning methods that have shown promising performance in classifying remote sensing image (RSI). However, it is widely recognized that deep learning methods typically require a substantial amount of labeled data to effectively converge. Acquiring a sufficient quantity of labeled data often necessitates significant human and material resources. Hence, few-shot RSISC has become highly meaningful. Fortunately, the recently proposed deep nearest neighbor neural network based on the attention mechanism (DN4AM) model incorporates episodic training and class-related attention mechanisms, effectively reducing the impact of background noise regions on classification results. Nevertheless, the DN4AM model does not address the problem of significant intra-class variability and substantial inter-class similarities observed in RSI scenes. Therefore, the discriminative enhanced attention-based deep nearest neighbor neural network (DEADN4) is proposed to address the few-shot RSISC task. Our method makes three contributions. Firstly, we introduce center loss to enhance the intra-class feature compactness. Secondly, we utilize the deep local-global descriptor (DLGD) to increase inter-class feature differentiation. Lastly, we modify the Softmax loss by incorporating cosine margin to amplify the inter-class feature dissimilarity. Experiments are conducted on three diverse RSI datasets to gauge the efficacy of our approach. Through comparative analysis with various cutting-edge methods including MatchingNet, RelationNet, MAML, Meta-SGD, DN4, and DN4AM, our approach showcases promising outcomes in the few-shot RSISC task.

Adversarial Defense Method Based on Latent Representation Guidance for Remote Sensing Image Scene Classification.

Deep neural networks have made great achievements in remote sensing image analyses; however, previous studies have shown that deep neural networks exhibit incredible vulnerability to adversarial examples, which raises concerns about regional safety and production safety. In this paper, we propose an adversarial denoising method based on latent representation guidance for remote sensing image scene classification. In the training phase, we train a variational autoencoder to reconstruct the data using only the clean dataset. At test time, we first calculate the normalized mutual information between the reconstructed image using the variational autoencoder and the reference image as denoised by a discrete cosine transform. The reconstructed image is selectively utilized according to the result of the image quality assessment. Then, the latent representation of the current image is iteratively updated according to the reconstruction loss so as to gradually eliminate the influence of adversarial noise. Because the training of the denoiser only involves clean data, the proposed method is more robust against unknown adversarial noise. Experimental results on the scene classification dataset show the effectiveness of the proposed method. Furthermore, the method achieves better robust accuracy compared with state-of-the-art adversarial defense methods in image classification tasks.

HCPNet: Learning discriminative prototypes for few-shot remote sensing image scene classification

HCPNet: Learning discriminative prototypes for few-shot remote sensing image scene classification

HELViT: highly efficient lightweight vision transformer for remote sensing image scene classification

HELViT: highly efficient lightweight vision transformer for remote sensing image scene classification