Remote Sensing Scene Research Articles

Remote-Sensing Cross-Domain Scene Classification: A Dataset and Benchmark

Domain adaptation for classification has achieved significant progress in natural images but not in remote-sensing images due to huge differences in data-imaging mechanisms between different modalities and inconsistencies in class labels among existing datasets. More importantly, the lack of cross-domain benchmark datasets has become a major obstacle to the development of scene classification in multimodal remote-sensing images. In this paper, we present a cross-domain dataset of multimodal remote-sensing scene classification (MRSSC). The proposed MRSSC dataset contains 26,710 images of 7 typical scene categories with 4 distinct domains that are collected from Tiangong-2, a Chinese manned spacecraft. Based on this dataset, we evaluate several representative domain adaptation algorithms on three cross-domain tasks to build baselines for future research. The results demonstrate that the domain adaptation algorithm can reduce the differences in data distribution between different domains and improve the accuracy of the three tasks to varying degrees. Furthermore, MRSSC also achieved fairly results in three applications: cross-domain data annotation, weakly supervised object detection and data retrieval. This dataset is believed to stimulate innovative research ideas and methods in remote-sensing cross-domain scene classification and remote-sensing intelligent interpretation.

A Lightweight Convolutional Neural Network Based on Hierarchical-Wise Convolution Fusion for Remote-Sensing Scene Image Classification

The large intra-class difference and inter-class similarity of scene images bring great challenges to the research of remote-sensing scene image classification. In recent years, many remote-sensing scene classification methods based on convolutional neural networks have been proposed. In order to improve the classification performance, many studies increase the width and depth of convolutional neural network to extract richer features, which increases the complexity of the model and reduces the running speed of the model. In order to solve this problem, a lightweight convolutional neural network based on hierarchical-wise convolution fusion (LCNN-HWCF) is proposed for remote-sensing scene image classification. Firstly, in the shallow layer of the neural network (groups 1–3), the proposed lightweight dimension-wise convolution (DWC) is utilized to extract the shallow features of remote-sensing images. Dimension-wise convolution is carried out in the three dimensions of width, depth and channel, and then, the convoluted features of the three dimensions are fused. Compared with traditional convolution, dimension-wise convolution has a lower number of parameters and computations. In the deep layer of the neural network (groups 4–7), the running speed of the network usually decreases due to the increase in the number of filters. Therefore, the hierarchical-wise convolution fusion module is designed to extract the deep features of remote-sensing images. Finally, the global average pooling layer, the fully connected layer and the Softmax function are used for classification. Using global average pooling before the fully connected layer can better preserve the spatial information of features. The proposed method achieves good classification results on UCM, RSSCN7, AID and NWPU datasets. The classification accuracy of the proposed LCNN-HWCF on the AID dataset (training:test = 2:8) and the NWPU dataset (training:test = 1:9), with great classification difficulty, reaches 95.76% and 94.53%, respectively. A series of experimental results show that compared with some state-of-the-art classification methods, the proposed method not only greatly reduces the number of network parameters but also ensures the classification accuracy and achieves a good trade-off between the model classification accuracy and running speed.

Self-Supervision and Self-Distillation with Multilayer Feature Contrast for Supervision Collapse in Few-Shot Remote Sensing Scene Classification

Although the means of catching remote sensing images are becoming more effective and more abundant, the samples that can be collected in some specific environments can be quite scarce. When there are limited labeled samples, the methods for analyzing remote sensing images for scene classification perform drastically worse. Methods that classify few-shot remote sensing image scenes are often based on meta-learning algorithms for the handling of sparse data. However, this research shows they will be affected by supervision collapse where features in remote sensing images that help with out-of-distribution classes are discarded, which is harmful for the generation of unseen classes and new tasks. In this work, we wish to remind readers of the existence of supervision collapse in scene classification of few-shot remote sensing images and propose a method named SSMR based on multi-layer feature contrast to overcome supervision collapse. First of all, the method makes use of the label information contained in a finite number of samples for supervision and guides self-supervised learning to train the embedding network with supervision generated by multilayer feature contrast. This can prevent features from losing intra-class variation. Intra-class variation is always useful in classifying unseen data. What is more, the multi-layer feature contrast is merged with self-distillation, and the modified self-distillation is used to encourage the embedding network to extract sufficiently general features that transfer better to unseen classes and new domains. We demonstrate that most of the existing few-shot scene classification methods suffer from supervision collapse and that SSMR overcomes supervision collapse well in the experiments on the new dataset we specially designed for examining the problem, with a 2.4–17.2% increase compared to the available methods. Furthermore, we performed a series of ablation experiments to demonstrate how effective and necessary each structure of the proposed method is and to show how different choices in training impact final performance.

Adaptive Unsupervised-Shadow-Detection Approach for Remote-Sensing Image Based on Multichannel Features

Shadow detection is an essential research topic in the remote-sensing domain, as the presence of shadow causes the loss of ground-object information in real areas. It is hard to define specific threshold values for the identification of shadow areas with the existing unsupervised approaches due to the complexity of remote-sensing scenes. In this study, an adaptive unsupervised-shadow-detection method based on multichannel features is proposed, which can adaptively distinguish shadow in different scenes. First, new multichannel features were designed in the hue, saturation, and intensity color space, and the shadow properties of high hue, high saturation, and low intensity were considered to solve the insufficient feature-extraction problem of shadows. Then, a dynamic local adaptive particle swarm optimization was proposed to calculate the segmentation thresholds for shadows in an adaptive manner. Finally, experiments performed on the Aerial Imagery dataset for Shadow Detection (AISD) demonstrated the superior performance of the proposed approach in comparison with traditional unsupervised shadow-detection and state-of-the-art deep-learning methods. The experimental results show that the proposed approach can detect the shadow areas in remote-sensing images more accurately and efficiently, with the F index being 82.70% on the testing images. Thus, the proposed approach has better application potential in scenarios without a large number of labeled samples.

An Attention Cascade Global–Local Network for Remote Sensing Scene Classification

Remote sensing image scene classification is an important task of remote sensing image interpretation, which has recently been well addressed by the convolutional neural network owing to its powerful learning ability. However, due to the multiple types of geographical information and redundant background information of the remote sensing images, most of the CNN-based methods, especially those based on a single CNN model and those ignoring the combination of global and local features, exhibit limited performance on accurate classification. To compensate for such insufficiency, we propose a new dual-model deep feature fusion method based on an attention cascade global–local network (ACGLNet). Specifically, we use two popular CNNs as the feature extractors to extract complementary multiscale features from the input image. Considering the characteristics of the global and local features, the proposed ACGLNet filters the redundant background information from the low-level features through the spatial attention mechanism, followed by which the locally attended features are fused with the high-level features. Then, bilinear fusion is employed to produce the fused representation of the dual model, which is finally fed to the classifier. Through extensive experiments on four public remote sensing scene datasets, including UCM, AID, PatternNet, and OPTIMAL-31, we demonstrate the feasibility of the proposed method and its superiority over the state-of-the-art scene classification methods.

Generative Adversarial Networks for Zero-Shot Remote Sensing Scene Classification

Deep learning-based methods succeed in remote sensing scene classification (RSSC). However, current methods require training on a large dataset, and if a class does not appear in the training set, it does not work well. Zero-shot classification methods are designed to address the classification for unseen category images in which the generative adversarial network (GAN) is a popular method. Thus, our approach aims to achieve the zero-shot RSSC based on GAN. We employed the conditional Wasserstein generative adversarial network (WGAN) to generate image features. Since remote sensing images have inter-class similarity and intra-class diversity, we introduced classification loss, semantic regression module, and class-prototype loss to constrain the generator. The classification loss was used to preserve inter-class discrimination. We used the semantic regression module to ensure that the image features generated by the generator can represent the semantic features. We introduced class-prototype loss to ensure the intra-class diversity of the synthesized image features and avoid generating too homogeneous image features. We studied the effect of different semantic embeddings for zero-shot RSSC. We performed experiments on three datasets, and the experimental results show that our method performs better than the state-of-the-art methods in zero-shot RSSC in most cases.

Multi-Output Network Combining GNN and CNN for Remote Sensing Scene Classification

Scene classification is an active research area in the remote sensing (RS) domain. Some categories of RS scenes, such as medium residential and dense residential scenes, would contain the same type of geographical objects but have various spatial distributions among these objects. The adjacency and disjointness relationships among geographical objects are normally neglected by existing RS scene classification methods using convolutional neural networks (CNNs). In this study, a multi-output network (MopNet) combining a graph neural network (GNN) and a CNN is proposed for RS scene classification with a joint loss. In a candidate RS image for scene classification, superpixel regions are constructed through image segmentation and are represented as graph nodes, while graph edges between nodes are created according to the spatial adjacency among corresponding superpixel regions. A training strategy of a jointly learning CNN and GNN is adopted in the MopNet. Through the message propagation mechanism of MopNet, spatial and topological relationships imbedded in the edges of graphs are employed. The parameters of the CNN and GNN in MopNet are updated simultaneously with the guidance of a joint loss via the backpropagation mechanism. Experimental results on the OPTIMAL-31 and aerial image dataset (AID) datasets show that the proposed MopNet combining a graph convolutional network (GCN) or graph attention network (GAT) and ResNet50 achieves state-of-the-art accuracy. The overall accuracy obtained on OPTIMAL-31 is 96.06% and those on AID are 95.53% and 97.11% under training ratios of 20% and 50%, respectively. Spatial and topological relationships imbedded in RS images are helpful for improving the performance of scene classification.

A Combination of Lie Group Machine Learning and Deep Learning for Remote Sensing Scene Classification Using Multi-Layer Heterogeneous Feature Extraction and Fusion

Discriminative feature learning is the key to remote sensing scene classification. Previous research has found that most of the existing convolutional neural networks (CNN) focus on the global semantic features and ignore shallower features (low-level and middle-level features). This study proposes a novel Lie Group deep learning model for remote sensing scene classification to solve the above-mentioned challenges. Firstly, we extract shallower and higher-level features from images based on Lie Group machine learning (LGML) and deep learning to improve the feature representation ability of the model. In addition, a parallel dilated convolution, a kernel decomposition, and a Lie Group kernel function are adopted to reduce the model’s parameters to prevent model degradation and over-fitting caused by the deepening of the model. Then, the spatial attention mechanism can enhance local semantic features and suppress irrelevant feature information. Finally, feature-level fusion is adopted to reduce redundant features and improve computational performance, and cross-entropy loss function based on label smoothing is used to improve the classification accuracy of the model. Comparative experiments on three public and challenging large-scale remote-sensing datasets show that our model improves the discriminative ability of features and achieves competitive accuracy against other state-of-the-art methods.

Identifying Urban Wetlands through Remote Sensing Scene Classification Using Deep Learning: A Case Study of Shenzhen, China

Urban wetlands provide cities with unique and valuable ecosystem services but are under great degradation pressure. Correctly identifying urban wetlands from remote sensing images is fundamental for developing appropriate management and protection plans. To overcome the semantic limitations of traditional pixel-level urban wetland classification techniques, we proposed an urban wetland identification framework based on an advanced scene-level classification scheme. First, the Sentinel-2 high-resolution multispectral image of Shenzhen was segmented into 320 m × 320 m square patches to generate sample datasets for classification. Next, twelve typical convolutional neural network (CNN) models were transformed for the comparison experiments. Finally, the model with the best performance was used to classify the wetland scenes in Shenzhen, and pattern and composition analyses were also implemented in the classification results. We found that the DenseNet121 model performed best in classifying urban wetland scenes, with overall accuracy (OA) and kappa values reaching 0.89 and 0.86, respectively. The analysis results revealed that the wetland scene in Shenzhen is generally balanced in the east–west direction. Among the wetland scenes, coastal open waters accounted for a relatively high proportion and showed an obvious southward pattern. The remaining swamp, marsh, tidal flat, and pond areas were scattered, accounting for only 4.64% of the total area of Shenzhen. For scattered and dynamic urban wetlands, we are the first to achieve scene-level classification with satisfactory results, thus providing a clearer and easier-to-understand reference for management and protection, which is of great significance for promoting harmony between humanity and ecosystems in cities.

Synthetic aircraft RS image modelling based on improved conditional GAN joint embedding network

Developing an efficient and quality remote sensing (RS) technology using volume and efficient modelling in different aircraft RS images is challenging. Generative models serve as a natural and convenient simulation method. Because aircraft types belong to the fine class under the rough class, the issue of feature entanglement may occur while modelling multiple aircraft classes. Our solution to this issue was a novel first-generation realistic aircraft type simulation system (ATSS-1) based on the RS images. It realised fine modelling of the seven aircraft types based on a real scene by establishing an adaptive weighted conditional attention generative adversarial network and joint geospatial embedding (GE) network. An adaptive weighted conditional batch normalisation attention block solved the subclass entanglement by reassigning the intra-class-wise characteristic responses. Subsequently, an asymmetric residual self-attention module was developed by establishing a remote region asymmetric relationship for mining the finer potential spatial representation. The mapping relationship between the input RS scene and the potential space of the generated samples was explored through the GE network construction that used the selected prior distribution z, as an intermediate representation. A public RS dataset (OPT-Aircraft_V1.0) and two public datasets (MNIST and Fashion-MNIST) were used for simulation model testing. The results demonstrated the effectiveness of ATSS-1, promoting further development of realistic automatic RS simulation.

Homo–Heterogenous Transformer Learning Framework for RS Scene Classification

Remote sensing (RS) scene classification plays an essential role in the RS community and has attracted increasing attention due to its wide applications. Recently, benefiting from the powerful feature learning capabilities of convolutional neural networks (CNNs), the accuracy of the RS scene classification has significantly been improved. Although the existing CNN-based methods achieve excellent results, there is still room for improvement. First, the CNN-based methods are adept at capturing the global information from RS scenes. Still, the context relationships hidden in RS scenes cannot be thoroughly mined. Second, due to the specific structure, it is easy for normal CNNs to exploit the heterogenous information from RS scenes. Nevertheless, the homogenous information, which is also crucial to comprehensively understand complex contents within RS scenes, does not get the attention it deserves. Third, most CNNs focus on establishing the relationships between RS scenes and semantic labels. However, the similarities between them are not considered deeply, which are helpful to distinguish the intra-/interclass samples. To overcome the limitations mentioned previously, we propose a homo–heterogenous transformer learning (HHTL) framework for the RS scene classification in this article. First, a patch generation module is designed to generate homogenous and heterogenous patches. Then, a dual-branch feature learning module (FLM) is proposed to mine homogenous and heterogenous information within RS scenes simultaneously. In the FLM, based on vision transformer, not only the global information but also the local areas and their context information can be captured. Finally, we design a classification module, which consists of a fusion submodule and a metric-learning module. It can integrate homo–heterogenous information and compact/separate samples from the same/different RS scene categories. Extensive experiments are conducted on four public RS scene datasets. The encouraging results demonstrate that our HHTL framework can outperform many state-of-the-art methods. Our source codes are available at the below website.

EMTCAL: Efficient Multiscale Transformer and Cross-Level Attention Learning for Remote Sensing Scene Classification

In recent years, convolutional neural network (CNN)-based methods have been widely used for remote sensing (RS) scene classification tasks and achieved excellent results. However, CNNs are not good at exploring contextual information, which is essential for fully understanding RS scenes. A new model named transformer attracts researchers’ attention to address this problem, which is skilled in mining the latent contextual information in RS scenes. Nevertheless, since the contents of RS scenes are diverse in type and various in scale, the performance of the original transformer in RS scene classification cannot reach what we expect. In addition, due to the specific self-attention mechanism, the time costs of the transformer are high, which hinders its practicability in the RS community. To overcome the above limitations, we propose a new model named efficient multi-scale transformer and cross-level attention learning (EMTCAL) for RS scene classification in this paper. EMTCAL combines the advantages of CNN and transformer to mine information within RS scenes fully. First, it uses a multi-layer feature extraction module (MFEM) to acquire global visual features and multi-level convolutional features from RS scenes. Second, a contextual information extraction module (CIEM) is proposed to capture rich contextual information from multi-level features. In CIEM, taking the characteristics of RS scenes and the computational complexity into account, we propose an efficient multi-scale transformer (EMST). EMST can mine the abundant knowledge with various scales hidden in RS scenes and model their inherent relations at small-time costs. Third, a cross-level attention module (CLAM) is developed to aggregate and explore correlations of multi-level features. Finally, a class score fusion module (CSFM) is designed to integrate the contributions of global and aggregated multi-level features for the discriminative scene representations. Extensive experiments are conducted on three public RS scene data sets. The positive results demonstrate that our EMTCAL can achieve superior classification performance and outperform many state-of-the-art methods.

Double FCOS: A Two-Stage Model Utilizing FCOS for Vehicle Detection in Various Remote Sensing Scenes

Vehicle detection in various remote sensing scenes is a challenging task. Various remote sensing scenes are mixed up with images of multi-scene, multi-quality, multi-scale and multi-class. Vehicle detection models suffer from inadequate candidate boxes, weak positive proposal sampling and poor classification performance, resulting in a detection performance degradation when they are applied in various scenes. What is worse, there is no such a dataset covering various scenes which is for vehicle detection. This paper proposes a vehicle detection model called Double FCOS and a vehicle dataset called 4MVD for vehicle detection in various remote sensing scenes. Double FCOS is a two-stage detection model based on fully convolution one-stage object detection (FCOS). FCOS is exploited in the RPN stage to generate candidate boxes in various scenes. The two-stage positive and negative sample model (TPNSM) is carefully designed to enhance the positive proposal sampling effects, particularly the tiny or weak vehicles, which are ignored in FCOS. A two-step classification model (TSCM) has been designed in the RCNN stage with a proposal classification branch and point classification branch to enhance the classification performance between the various types of vehicles. 4MVD (multi-scene, multi-quality, multi-scale and multi-class vehicle dataset) is collected from various remote sensing scenes to evaluate the performance of Double FCOS. A mean average accuracy of 78.3 percentage for vehicle detection on 5 categories has been received by Double FCOS on 4MVD. Extensive experiments demonstrate that Double FCOS significantly improves the performance of vehicle detection in various remote sensing scenes.

RDPN: Tackling the Data Shift Problem in Remote Sensing Scene Classification

Currently, remote-sensing (RS) scene classification has played an important role in many practical applications. However, traditional methods that are based on deep convolutional neural networks (DCNNs) have several difficulties when faced with data shift problems: novel classes, varied orientations, and large intraclass variations of RS scene images. In this letter, we propose the rotation-invariant and discriminative-learning prototypical networks (RDPNs) for RS scene classification. RDPN uses A-ORConv32 basic blocks and attention mechanisms to obtain rotation-invariant and discriminative features. In addition, adaptive cosine center loss is proposed to constrain the features to mitigate the large intraclass variations and penalize the hard samples adaptively. We conduct extensive experiments on publicly available datasets and achieve 1.69%–19.38% higher accuracy than existing methods. The experimental results verify that the proposed RDPN can solve the data shift problems well in RS scene classification.

WRMatch: Improving FixMatch With Weighted Nuclear-Norm Regularization for Few-Shot Remote Sensing Scene Classification

Semisupervised learning (SSL), such as FixMatch, has been successfully applied to remote sensing scene classification to relieve the burden of data annotation. However, in some extreme settings, only very few samples available, e.g., one to ten labels per remote sensing scene, can be used. When meeting this “few-shot” scenario, the deep model may be overfitting and prone to generate confusing predictions due to the lack of labels and strong augmentation-based perturbations. Thus, the prediction’s diversity may collapse, and the discriminability exceeds the reasonable interval. How to improve the performance of few-shot learning is underexplored for the remote sensing scene classification in previous studies. In this article, we present a novel framework for the task by utilizing the improved FixMatch and the weighted nuclear-norm regularization (WNNR). Specifically, we regularize the prediction matrix by exploiting the nuclear-norm, which is an approximation of the matrix rank and a relaxed boundary for the Shannon entropy. We further provide two weighting schemes to improve the nuclear-norm-based regularization. First, the random-weighting scheme for nuclear-norm (RWNNR) is proposed based on the Dirichlet distribution to improve the model’s generalization. Second, we present the self-weighting scheme (SWNNR) to weight the singular values according to singular values themselves and adjust the relaxed degree for the boundary between the nuclear-norm and the Shannon entropy. Maximizing the weighted nuclear-norm can improve the prediction diversity and optimize the prediction discriminability simultaneously. Combining the advantages of SSL and the aforementioned improvements, we can reliably classify the remote sensing scene image with very limited annotated datasets. To empirically demonstrate the proposed method’s effectiveness, we comprehensively evaluate the method on three publicly available benchmark datasets. The results show that the proposed method outperforms the baseline methods by a large margin and achieves superior performance on all three datasets. Our method can be an effective alternative to metalearning in few-shot scene classification, with the advantage lying in the competitive performance and the absence of metatraining stage associated with a large number of labels.

Class-Level Prototype Guided Multiscale Feature Learning for Remote Sensing Scene Classification With Limited Labels

Remote sensing scene classification (RSSC) is an open and challenging research topic in the remote sensing (RS) community. It aims to define semantic labels to RS scenes according to their contents. Recently, with the development of deep convolutional neural networks (DCNNs), the results of RSSC have been enhanced to a large extent. However, the cracking performance of these DCNN-based models depends on a large number of labeled data. Once the volume of the labeled data is decreased, their behavior would be weakened dramatically. In this paper, we propose a new training algorithm that can work smoothly with a few labeled samples to address this limitation. Along with the introduced DCNN, the presented methods perform satisfactorily. In particular, we first construct a dual-branch network (DBNet) to mine the multi-scale and multi-angle information from RS scenes. Thus, the abundant land covers with diverse sizes, directions, and shapes can be captured simultaneously. Then, to train DBNet using scarce semantic labels, a class-level prototype guided learning (CPGL) algorithm is developed based on the mate-learning paradigm. Besides the usual episode training manner, a prototype refinement module (PRM) and a prototype discrimination module (PDM) are designed with the help of metric learning theory to ensure the effectiveness of our CPGL. The comprehensive experiments are conducted on four public RS scene data sets, and the encouraging results imply that our DBNet and CPGL can copy with RSSC tasks with small labeled data. Our source codes are available at https://github.com/TangXu-Group/Remote-Sensing-Images-Classification/tree/main/CPGL.

Weakly Supervised Semantic Segmentation in Aerial Imagery via Explicit Pixel-Level Constraints

In recent years, image-level weakly supervised semantic segmentation (WSSS) has developed rapidly in natural scenes due to the easy availability of classification tags. However, limited to complex backgrounds, multi-category scenes, and dense small targets in remote sensing (RS) images, relatively little research has been conducted in this field. To alleviate the impact of the above problems in RS scenes, a self-supervised Siamese network based on an explicit pixel-level constraints framework is proposed, which greatly improves the quality of class activation maps and the positioning accuracy in multi-category RS scenes. Specifically, there are three novel devices in this paper to promote performance to a new level: (a) A pixel-soft classification loss is proposed, which realizes explicit constraints on pixels during the image-level training; (b) A pixel global awareness module, which captures high-level semantic context and low-level pixel spatial information, is constructed to improve the consistency and accuracy of RS object segmentation; (c) A dynamic multi-scale fusion module with a gating mechanism is devised, which enhances feature representation and improves the positioning accuracy of RS objects, particularly on small and dense objects. Experiments on two RS challenge datasets demonstrate that these proposed modules achieve new state-of-the-art results by only using image-level labels, which improve mIoU to 36.79% on iSAID and 45.43% on ISPRS in the WSSS task. To the best of our knowledge, this is the first work to perform image-level WSSS on multi-class RS scenes.

Learning to Cooperate: Decision Fusion Method for Few-Shot Remote-Sensing Scene Classification

Recently, remote-sensing scene classification has become an essential primary research topic. Nowadays, scholars have proposed various few-shot remote-sensing scene classification methods to achieve superior performance with few labeled data. Most of the prior work utilized a meta-learning strategy, which suffered from too little data affecting performance. In this letter, we apply the pre-trained feature extractor for image embedding. Meanwhile, because of the negative transfer problem caused by the inadaptability of the pre-trained feature extractor to remote-sensing data, we propose to exploit two pre-trained models to classify the remote-sensing scene, respectively. Then we fuse the decision to obtain the final classification category. We design a decision attention module to automatically update combination weights for each decision. It comprehensively considers the contribution of various decisions and further improves the discrimination of features. We conduct comprehensive experiments to validate the method and achieve state-of-the-art performance on two benchmark remote-sensing scene datasets, namely NWPU-RESISC45 and UC Merced.

Open Set Few-Shot Remote Sensing Scene Classification Based on a Multiorder Graph Convolutional Network and Domain Adaptation

Few-shot scene classification aims to recognize task data, which consists of unlabeled data and a few annotated samples, given some labeled auxiliary data. The task and the auxiliary data often come from the same source with non-overlapping categories, such tasks are defined as closed set few-shot scene classification. However, most remote sensing applications involve open set few-shot scene classification, where the categories of the task and auxiliary data are often partly overlapped and sources of data are usually different (i.e., different domains). We propose an open set few-shot scene classification method called the multi-order graph convolutional network (MGCN) specifically for such tasks. MGCN solves the open set few-shot scene classification task in two ways: reducing the inter-domain discrepancy by a feature dispersion degree weighting domain adaptation method, and reducing the dispersion degree of node features by a proposed multi-order graph convolutional network. Based on a shallow backbone and an improved graph convolutional network, MGCN improves the classification performance of the open set few-shot classification task by utilizing information from common categories and reducing domain discrepancies. The experimental results, based on three public remote sensing image datasets and a Tibetan Plateau remote sensing scene dataset we collected, demonstrate the effectiveness of the proposed method on the open set few-shot scene classification task.

Relation-Attention Networks for Remote Sensing Scene Classification

Remote sensing (RS) scene classification plays an important role in a wide range of RS applications. Recently, convolutional neural networks (CNNs) have been applied to the field of scene classification in RS images and achieved impressive performance. However, to classify RS scenes, most of the existing CNN methods either utilize the high-level features from the last convolutional layer of CNNs, missing much important information existing at the other levels, or directly fuse the features at different levels, bringing redundant and/or mutually exclusive information. Inspired by the attention mechanism of the human visual system, in this article, we explore a novel relation-attention model and design an end-to-end relation-attention network (RANet) to learn powerful feature representations of multiple levels to further improve the classification performance. First, we propose to extract convolutional features at different levels by pretrained CNNs. Second, a multiscale feature computation module is constructed to connect features at different levels and generate multiscale semantic features. Third, a novel relation-attention model is designed to focus on the critical features whilst avoiding the use of redundant and even distractive ones by exploiting the scale contextual information. Finally, the resulting relation-attention features are concatenated and fed into a softmax layer for the final classification. Experiments on four well-known RS scene classification datasets (UC-Merced, WHU-RS19, AID, and OPTIMAL-31) show that our method outperforms some state-of-the-art algorithms.