Remote Sensing Scene Classification Methods Research Articles

A multi-scale dense residual correlation network for remote sensing scene classification

Most existing scene classification methods based on remote sensing images tend to ignore important interactive information at different levels in the image. We propose an effective remote sensing scene classification method named multi-scale dense residual correlation network. The method is divided into three parts. First, the multi-stream feature extraction module is introduced which effectively utilizes features at different scales to extract different levels of information. Secondly, the dense residual connected feature fusion technology is proposed, which allows for a wide range of feature fusion. The Correlation Attention Module learn feature representations at multiple levels. This improves classification performance. The method outperforms existing algorithms in terms of effectiveness and accuracy, achieving state-of-the-art results on widely used remote sensing scene classification benchmarks.

LPNet: A remote sensing scene classification method based on large kernel convolution and parameter fusion

LPNet: A remote sensing scene classification method based on large kernel convolution and parameter fusion

Zero-Shot Remote Sensing Scene Classification Method Based on Local-Global Feature Fusion and Weight Mapping Loss

Zero-Shot Remote Sensing Scene Classification Method Based on Local-Global Feature Fusion and Weight Mapping Loss

A Lightweight Dual-Branch Swin Transformer for Remote Sensing Scene Classification

The main challenge of scene classification is to understand the semantic context information of high-resolution remote sensing images. Although vision transformer (ViT)-based methods have been explored to boost the long-range dependencies of high-resolution remote sensing images, the connectivity between neighboring windows is still limited. Meanwhile, ViT-based methods commonly contain a large number of parameters, resulting in a huge computational consumption. In this paper, a novel lightweight dual-branch swin transformer (LDBST) method for remote sensing scene classification is proposed, and the discriminative ability of scene features is increased through combining a ViT branch and convolutional neural network (CNN) branch. First, based on the hierarchical swin transformer model, LDBST divides the input features of each stage into two parts, which are then separately fed into the two branches. For the ViT branch, a dual multilayer perceptron structure with a depthwise convolutional layer, termed Conv-MLP, is integrated into the branch to boost the connections with neighboring windows. Then, a simple-structured CNN branch with maximum pooling preserves the strong features of the scene feature map. Specifically, the CNN branch lightens the LDBST, by avoiding complex multi-head attention and multilayer perceptron computations. To obtain better feature representation, LDBST was pretrained on the large-scale remote scene classification images of the MLRSN and RSD46-WHU datasets. These two pretrained weights were fine-tuned on target scene classification datasets. The experimental results showed that the proposed LDBST method was more effective than some other advanced remote sensing scene classification methods.

Subspace prototype learning for few-Shot remote sensing scene classification

Recently, few-shot remote sensing scene classification (FSRSSC)has attracted more and more researchers’ attention. The FSRSSC aims to solve the problem of how to distinguish novel categories when there are limited labeled samples quickly. The FSRSSC framework mainly includes two stages: 1) pre-training (meta-training) stage, which uses training data to train the feature extractor. 2) meta-testing stage, the trained feature extractor is used to extract the features of testing data of different categories from the training data. A limited number of labeled samples are used to train the classifier and complete the classification task. In this paper, we proposed subspace prototype learning for few-shot remote sensing scene classification method (SPL). To improve the generalization performance of the feature extractor in the pre-training stage and the robustness of the classifier in the meta-testing stage, we improve the existing methods from two aspects. On the one hand, we introduce the pre-trained model on the natural images. Then we use remote sensing data to fine-tune the pre-trained model to solve the problem of negative transfer caused by the differences between natural and remote sensing images. On the other hand, in the meta-test phase, we use the subspace learning method to learn a subspace prototype for each type of sample and complete the classification task by measuring the distance between the samples and the prototype, which achieves performance classification performance.

Adaptive Discriminative Regions Learning Network for Remote Sensing Scene Classification.

As an auxiliary means of remote sensing (RS) intelligent interpretation, remote sensing scene classification (RSSC) attracts considerable attention and its performance has been improved significantly by the popular deep convolutional neural networks (DCNNs). However, there are still several challenges that hinder the practical applications of RSSC, such as complex composition of land cover, scale-variation of objects, and redundant and noisy areas for scene classification. In order to mitigate the impact of these issues, we propose an adaptive discriminative regions learning network for RSSC, referred as ADRL-Net briefly, which locates discriminative regions effectively for boosting the performance of RSSC by utilizing a novel self-supervision mechanism. Our proposed ADRL-Net consists of three main modules, including a discriminative region generator, a region discriminator, and a region scorer. Specifically, the discriminative region generator first generates some candidate regions which could be informative for RSSC. Then, the region discriminator evaluates the regions generated by region generator and provides feedback for the generator to update the informative regions. Finally, the region scorer makes prediction scores for the whole image by using the discriminative regions. In such a manner, the three modules of ADRL-Net can cooperate with each other and focus on the most informative regions of an image and reduce the interference of redundant regions for final classification, which is robust to the complex scene composition, object scales, and irrelevant information. In order to validate the efficacy of the proposed network, we conduct experiments on four widely used benchmark datasets, and the experimental results demonstrate that ADRL-Net consistently outperforms other state-of-the-art RSSC methods.

Semi-Supervised DEGAN for Optical High-Resolution Remote Sensing Image Scene Classification

Semi-supervised methods have made remarkable achievements via utilizing unlabeled samples for optical high-resolution remote sensing scene classification. However, the labeled data cannot be effectively combined with unlabeled data in the existing semi-supervised methods during model training. To address this issue, we present a semi-supervised optical high-resolution remote sensing scene classification method based on Diversity Enhanced Generative Adversarial Network (DEGAN), in which the supervised and unsupervised stages are deeply combined in the DEGAN training. Based on the unsupervised characteristic of the Generative Adversarial Network (GAN), a large number of unlabeled and labeled images are jointly employed to guide the generator to obtain a complete and accurate probability density space of fake images. The Diversity Enhanced Network (DEN) is designed to increase the diversity of generated images based on massive unlabeled data. Therefore, the discriminator is promoted to provide discriminative features by enhancing the generator given the game relationship between two models in DEGAN. Moreover, the conditional entropy is adopted to make full use of the information of unlabeled data during the discriminator training. Finally, the features extracted from the discriminator and VGGNet-16 are employed for scene classification. Experimental results on three large datasets demonstrate that the proposed scene classification method yields a superior classification performance compared with other semi-supervised methods.

Remote Sensing Image Scene Classification Based on Global Self-Attention Module

The complexity of scene images makes the research on remote-sensing image scene classification challenging. With the wide application of deep learning in recent years, many remote-sensing scene classification methods using a convolutional neural network (CNN) have emerged. Current CNN usually output global information by integrating the depth features extricated from the convolutional layer through the fully connected layer; however, the global information extracted is not comprehensive. This paper proposes an improved remote-sensing image scene classification method based on a global self-attention module to address this problem. The global information is derived from the depth characteristics extracted by the CNN. In order to better express the semantic information of the remote-sensing image, the multi-head self-attention module is introduced for global information augmentation. Meanwhile, the local perception unit is utilized to improve the self-attention module’s representation capabilities for local objects. The proposed method’s effectiveness is validated through comparative experiments with various training ratios and different scales on public datasets (UC Merced, AID, and NWPU-NESISC45). The precision of our proposed model is significantly improved compared to other methods for remote-sensing image scene classification.

TRS: Transformers for Remote Sensing Scene Classification

Remote sensing scene classification remains challenging due to the complexity and variety of scenes. With the development of attention-based methods, Convolutional Neural Networks (CNNs) have achieved competitive performance in remote sensing scene classification tasks. As an important method of the attention-based model, the Transformer has achieved great success in the field of natural language processing. Recently, the Transformer has been used for computer vision tasks. However, most existing methods divide the original image into multiple patches and encode the patches as the input of the Transformer, which limits the model’s ability to learn the overall features of the image. In this paper, we propose a new remote sensing scene classification method, Remote Sensing Transformer (TRS), a powerful “pure CNNs → Convolution + Transformer → pure Transformers” structure. First, we integrate self-attention into ResNet in a novel way, using our proposed Multi-Head Self-Attention layer instead of 3 × 3 spatial revolutions in the bottleneck. Then we connect multiple pure Transformer encoders to further improve the representation learning performance completely depending on attention. Finally, we use a linear classifier for classification. We train our model on four public remote sensing scene datasets: UC-Merced, AID, NWPU-RESISC45, and OPTIMAL-31. The experimental results show that TRS exceeds the state-of-the-art methods and achieves higher accuracy.

Few-Shot Scene Classification with attention mechanism in Remote Sensing

Remote sensing scene classification is a hot research topic in computer vision and it is of great significance to the semantic understanding of remote sensing images. At present, remote sensing scene classification methods based on deep learning occupy a dominant position in this field. However, it suffers from the lack of samples and poor model generalization ability in actual application scenarios. Therefore, this paper proposes a few-shot remote scene classification method based on attention mechanism, and designs a structure of dual-branches similarity measurement. This method is based on the meta-learning training strategy to divide the dataset into tasks. At the meantime, the input images are divided into blocks to preserve the feature distribution in the remote sensing image. Then the lightweight attention module is introduced into the feature extraction network to reduce the risk of overfitting and ensure the acquisition of discriminative features. Finally we add a dual-branches similarity measurement module on the basis of Earth Mover’s Distance to improve the discriminative ability of the classifier. The results show that compared with the classic small-sample learning method, the few-shot remote scene classification method proposed in this paper can significantly improve the classification performance.

Enhanced Feature Pyramid Network With Deep Semantic Embedding for Remote Sensing Scene Classification

Recent progress on remote sensing (RS) scene classification is substantial, benefiting mostly from the explosive development of convolutional neural networks (CNNs). However, different from the natural images in which the objects occupy most of the space, objects in RS images are usually small and separated. Therefore, there is still a large room for improvement of the vanilla CNNs that extract global image-level features for RS scene classification, ignoring local object-level features. In this article, we propose a novel RS scene classification method via enhanced feature pyramid network (EFPN) with deep semantic embedding (DSE). Our proposed framework extracts multiscale multilevel features using an EFPN. Then, to leverage the complementary advantages of the multilevel and multiscale features, we design a DSE module to generate discriminative features. Third, a feature fusion module, called two-branch deep feature fusion (TDFF), is introduced to aggregate the features at different levels in an effective way. Our method produces state-of-the-art results on two widely used RS scene classification benchmarks, with better effectiveness and accuracy than the existing algorithms. Beyond that, we conduct an exhaustive analysis on the role of each module in the proposed architecture, and the experimental results further verify the merits of the proposed method.

Remote sensing scene classification based on high-order graph convolutional network

ABSTRACT Remote sensing scene classification has gained increasing interest in remote sensing image understanding and feature representation is the crucial factor for classification methods. Convolutional Neural Network (CNN) generally uses hierarchical deep structure to automatically learn the feature representation from the whole images and thus has been widely applied in scene classification. However, it may fail to consider the discriminative components within the image during the learning process. Moreover, the potential relationships of scene semantics are likely to be ignored. In this paper, we present a novel remote sensing scene classification method based on high-order graph convolutional network (H-GCN). Our method uses the attention mechanism to focus on the key components inside the image during CNN feature learning. More importantly, high-order graph convolutional network is applied to investigate the class dependencies. The graph structure is built where each node is described by the mean of attentive CNN features from each semantic class. The semantic class dependencies are propagated with mixing neighbor information of nodes at different orders and thus the more informative representation of nodes can be gained. The node representations of H-GCN and attention CNN features are finally integrated as the discriminative feature representation for scene classification. Experimental results on benchmark datasets demonstrate the feasibility and effectiveness of our method for remote sensing scene classification.

Lightweight Channel Attention and Multiscale Feature Fusion Discrimination for Remote Sensing Scene Classification

High-resolution remote sensing image scene classification has attracted widespread attention as a basic earth observation task. Remote sensing scene classification aims to assign specific semantic labels to remote sensing scene images to serve specified applications. Convolutional neural networks are widely used for remote sensing image classification due to their powerful feature extraction capabilities. However, the existing methods have not overcome the difficulties of large-scene remote sensing images of large intraclass diversity and high interclass similarity, resulting in low performance. Therefore, we propose a new remote sensing scene classification method that combines lightweight channel attention and multiscale feature fusion discrimination, called LmNet. First, ResNeXt is used as the backbone; second, a new lightweight channel attention mechanism is constructed to quickly and adaptively learn the salient features of important channels. Furthermore, we designed a multiscale feature fusion discrimination framework, which fully integrates shallow edge feature information and deep semantic information to enhance feature representation capabilities and uses multiscale features for joint discrimination. Finally, a cross-entropy loss function based on label smoothing is built to reduce the influence of interclass similarity on feature representation. In particular, our lightweight channel attention and multiscale feature fusion mechanism can be flexibly embedded in any advanced backbone as a functional module. The experimental results on three large-scale remote sensing scene classification datasets show that compared with the existing advanced methods, our proposed high-efficiency end-to-end scene classification method has reached state-of-the-art. Moreover, our method has a weaker dependence on labeled data and provided better generalization performance.

Remote Sensing Image Scene Classification Based on an Enhanced Attention Module

Classifying different satellite remote sensing scenes is a very important subtask in the field of remote sensing image interpretation. With the recent development of convolutional neural networks (CNNs), remote sensing scene classification methods have continued to improve. However, the use of recognition methods based on CNNs is challenging because the background of remote sensing image scenes is complex and many small objects often appear in these scenes. In this letter, to improve the feature extraction and generalization abilities of deep neural networks so that they can learn more discriminative features, an enhanced attention module (EAM) was designed. Our proposed method achieved very competitive performance&#x2014;94.29&#x0025; accuracy on NWPU-RESISC45 and state-of-the-art performance on different remote sensing scene recognition data sets. The experimental results show that the proposed method can learn more discriminative features than state-of-the-art methods, and it can effectively improve the accuracy of scene classification for remote sensing images. Our code is available at <uri>https://github.com/williamzhao95/Pay-More-Attention</uri>.

Remote Sensing Scene Classification Using Heterogeneous Feature Extraction and Multi-Level Fusion

Understanding the scenes provided by remote sensing (RS) images has drawn increasing attention in the last decade. It is to automatically classify a RS scene image by feature extraction and label assignment. Although much effort has been dedicated to developing discriminative feature extraction as well as automatic classification techniques, it is still very challenging owing to the complex distributions of the ground objects in high spatial resolution scenes. To enhance the ability to represent the RS scenes, an integration of multiple types of features for remote sensing scenes is considered as an effective way. Nevertheless, different kinds of features possess different characteristics, and how to fuse them together is a critical problem. In this paper, to fuse three different but complementary types of features that are extracted to characterize global attributes of scenes together, a discriminant correlation analysis based feature-level fusion approach is proposed, which maximizes the correlation of corresponding features across the two feature sets and in addition de-correlates features that belong to different classes within each feature set. In addition, to further improve the scene classification performance, another kind of information fusion form, called decision-level fusion is adopted in the classification stage. In our proposed decision-level fusion technique, the final results of multiple classifiers are combined via majority voting. Extensive experiments results conducted on the well-known SIRI-WHU dataset, the WHU-RS dataset and the UC Merced Land Use dataset have demonstrated that the proposed remote sensing scene classification method based on heterogeneous feature extraction and multi-level fusion is superior to many state-of-the-art scene classification algorithms.

Scene classification from remote sensing images using mid-level deep feature learning

ABSTRACTTraditional remote sensing scene classification methods based on low-level local or global features easily lead to information loss, additionally, the influence of spatial correlation on scene images and the redundancy of feature representation are neglected. For overcoming these drawbacks, learnable multilayer energized locality constrained affine subspace coding (MELASC) – Convolutional Neural Network (CNN) framework (MELASC-CNN) which could generate orderless feature representation is proposed, and it considers both the diversity of local – global deep features and the redundancies of local geometric structure around visual words. Firstly, the energy of the basis is introduced to limit the number of neighbouring subspaces, moreover learnable locality-constrained affine subspace coding is presented for keeping the locality and sparsity of the corresponding coding vector, and otherwise, we utilize Gaussian Mixed Model (GMM) to improve the robustness of dictionary. Specifically, second-order coding based on information geometry is performed to further improve MELASC-CNN’s performance; additionally, three kinds of proximity measures are proposed for describing closeness between features and affine subspaces. Finally, MELASC-CNN is built on the combination of the convolutional and fully connected layers for considering the global and local features. Simultaneously, MELASC-CNN extracts the feature vector at different resolutions through Spatial Pyramid Matching (SPM), and it integrates the spatial information into the final representation vector. For validation and comparison purposes, we conduct extensive experiments on two challenging high-resolution remote sensing datasets and show better performance than other related works.

A novel two-stage scene classification model based on feature variable significance in high-resolution remote sensing

In this article, we focus on the high-resolution remote sensing scene classification, which aims to label the scene image according to its content. While the high similarity of the between-class and variety of the inner class in the scene images make it challenging. To address the issue, we propose a unified two-stage classification framework named variable-weighted multi-feature fusing (VWMF) method. In the first stage, we fuse multiple low-level features into a concatenated holistic histogram feature via pre-trained unsupervised models. Then, we use a kernel collaborative representation-based classification method to predict the two most possible categories. In the second stage, for the two possible categories, we calculate the variable-weighted feature through a pre-calculated feature variable weight set of the corresponding two categories, and then put the variable-weighted feature into the pre-trained SVM for one-versus-one classification. Comprehensive experiments on two publicly available data sets demonstrate the superiority of our VWMF method compared with state-of-the-art remote sensing scene classification methods.

Concentric Circle Pooling in Deep Convolutional Networks for Remote Sensing Scene Classification

Convolutional neural networks (CNNs) have been increasingly used in remote sensing scene classification/recognition. The conventional CNNs are sensitive to the rotation of the image scene, which will inevitably result in the misclassification of remote sensing scene images that belong to the same category. In this work, we equip the networks with a new pooling strategy, “concentric circle pooling”, to alleviate the above problem. The new network structure, called CCP-net can generate a concentric circle-based spatial-rotation-invariant representation of an image, hence improving the classification accuracy. The square kernel is adopted to approximate the circle kernels in concentric circle pooling, which is much more efficient and suitable for CNNs to propagate gradients. We implement the training of the proposed network structure with standard back-propagation, thus CCP-net is an end-to-end trainable CNNs. With these advantages, CCP-net should in general improve CNN-based remote sensing scene classification methods. Experiments using two publicly available remote sensing scene datasets demonstrate that using CCP-net can achieve competitive classification results compared with the state-of-art methods.