High-resolution Remote Sensing Scene Classification Research Articles

SAGN: Semantic-Aware Graph Network for Remote Sensing Scene Classification.

The scene classification of remote sensing (RS) images plays an essential role in the RS community, aiming to assign the semantics to different RS scenes. With the increase of spatial resolution of RS images, high-resolution RS (HRRS) image scene classification becomes a challenging task because the contents within HRRS images are diverse in type, various in scale, and massive in volume. Recently, deep convolution neural networks (DCNNs) provide the promising results of the HRRS scene classification. Most of them regard HRRS scene classification tasks as single-label problems. In this way, the semantics represented by the manual annotation decide the final classification results directly. Although it is feasible, the various semantics hidden in HRRS images are ignored, thus resulting in inaccurate decision. To overcome this limitation, we propose a semantic-aware graph network (SAGN) for HRRS images. SAGN consists of a dense feature pyramid network (DFPN), an adaptive semantic analysis module (ASAM), a dynamic graph feature update module, and a scene decision module (SDM). Their function is to extract the multi-scale information, mine the various semantics, exploit the unstructured relations between diverse semantics, and make the decision for HRRS scenes, respectively. Instead of transforming single-label problems into multi-label issues, our SAGN elaborates the proper methods to make full use of diverse semantics hidden in HRRS images to accomplish scene classification tasks. The extensive experiments are conducted on three popular HRRS scene data sets. Experimental results show the effectiveness of the proposed SAGN.

Remote Sensing Scene Classification Via Multigranularity Alternating Feature Mining

Models based on convolutional neural networks (CNNs) have achieved remarkable advances in high-resolution remote sensing (HRRS) images scene classification, but there are still challenges due to the high similarity among different categories and loss of local information. To address this issue, a multigranularity alternating feature mining (MGA-FM) framework is proposed in this article to learn and fuse both global and local information for HRRS scene classification. First, a region confusion mechanism is adopted to guide network's shallow layers to adaptively learn the salient features of distinguishing regions. Second, an alternating comprehensive training strategy is designed to capture and fuse shallow local feature information and deep semantic information to enhance feature representation capabilities. In particular, the MGA-FM framework can be flexibly embedded in various CNN backbone networks as a training mechanism. Extensive experimental results and visualization analysis on three remote sensing scene datasets indicated that the proposed method can achieve competitive classification performance.

Semi-Supervised DEGAN for Optical High-Resolution Remote Sensing Image Scene Classification

Semi-supervised methods have made remarkable achievements via utilizing unlabeled samples for optical high-resolution remote sensing scene classification. However, the labeled data cannot be effectively combined with unlabeled data in the existing semi-supervised methods during model training. To address this issue, we present a semi-supervised optical high-resolution remote sensing scene classification method based on Diversity Enhanced Generative Adversarial Network (DEGAN), in which the supervised and unsupervised stages are deeply combined in the DEGAN training. Based on the unsupervised characteristic of the Generative Adversarial Network (GAN), a large number of unlabeled and labeled images are jointly employed to guide the generator to obtain a complete and accurate probability density space of fake images. The Diversity Enhanced Network (DEN) is designed to increase the diversity of generated images based on massive unlabeled data. Therefore, the discriminator is promoted to provide discriminative features by enhancing the generator given the game relationship between two models in DEGAN. Moreover, the conditional entropy is adopted to make full use of the information of unlabeled data during the discriminator training. Finally, the features extracted from the discriminator and VGGNet-16 are employed for scene classification. Experimental results on three large datasets demonstrate that the proposed scene classification method yields a superior classification performance compared with other semi-supervised methods.

DO WE STILL NEED IMAGENET PRE-TRAINING IN REMOTE SENSING SCENE CLASSIFICATION?

Abstract. Due to the scarcity of labeled data, using supervised models pre-trained on ImageNet is a de facto standard in remote sensing scene classification. Recently, the availability of larger high resolution remote sensing (HRRS) image datasets and progress in self-supervised learning have brought up the questions of whether supervised ImageNet pre-training is still necessary for remote sensing scene classification and would supervised pre-training on HRRS image datasets or self-supervised pre-training on ImageNet achieve better results on target remote sensing scene classification tasks. To answer these questions, in this paper we both train models from scratch and fine-tune supervised and self-supervised ImageNet models on several HRRS image datasets. We also evaluate the transferability of learned representations to HRRS scene classification tasks and show that self-supervised pre-training outperforms the supervised one, while the performance of HRRS pre-training is similar to self-supervised pre-training or slightly lower. Finally, we propose using an ImageNet pre-trained model combined with a second round of pre-training using in-domain HRRS images, i.e. domain-adaptive pre-training. The experimental results show that domain-adaptive pre-training results in models that achieve state-of-the-art results on HRRS scene classification benchmarks. The source code and pre-trained models are available at https://github.com/risojevicv/RSSC-transfer.

When CNNs Meet Vision Transformer: A Joint Framework for Remote Sensing Scene Classification

Scene classification is an indispensable part of remote sensing image interpretation, and various convolutional neural network (CNN)-based methods have been explored to improve classification accuracy. Although they have shown good classification performance on high-resolution remote sensing (HRRS) images, discriminative ability of extracted features is still limited. In this letter, a high-performance joint framework combined CNNs and vision transformer (ViT) (CTNet) is proposed to further boost the discriminative ability of features for HRRS scene classification. The CTNet method contains two modules, including the stream of ViT (T-stream) and the stream of CNNs (C-stream). For the T-stream, flattened image patches are sent into pretrained ViT model to mine semantic features in HRRS images. To complement with T-stream, pretrained CNN is transferred to extract local structural features in the C-stream. Then, semantic features and structural features are concatenated to predict labels of unknown samples. Finally, a joint loss function is developed to optimize the joint model and increase the intraclass aggregation. The highest accuracies on the aerial image dataset (AID) and Northwestern Polytechnical University (NWPU)-RESISC45 datasets obtained by the CTNet method are 97.70% and 95.49%, respectively. The classification results reveal that the proposed method achieves high classification performance compared with other state-of-the-art (SOTA) methods.

High-Resolution Remote Sensing Scene Classification Based on Salient Features and DCNN

High-Resolution Remote Sensing Scene Classification Based on Salient Features and DCNN

Mining Deep Semantic Representations for Scene Classification of High-Resolution Remote Sensing Imagery

Scene classification is one of the most fundamental task in interpretation of high-resolution remote sensing (HRRS) images. Many recent works show that the probabilistic topic models which are capable of mining latent semantics of images can be effectively applied to HRRS scene classification. However, the existing approaches based on topic models simply utilize low-level hand-crafted features to form semantic features, which severely limit the representative capability of the semantic features derived from topic models. To alleviate this problem, this paper propose to build powerful semantic features using the probabilistic latent semantic analysis (pLSA) model, by employing the pre-trained deep convolutional neural networks (CNNs) as feature extractors rather than relying on the hand-crafted features. Specifically, we develop two methods to generate semantic features, called multi-scale deep semantic representation (MSDS) and multi-level deep semantic representation (MLDS), by extracting CNN features from different layers: (1) in MSDS, the final semantic features are learned by the pLSA with multi-scale features extracted from the convolutional layer of a pre-trained CNN; (2) in MLDS, we extract CNN features for densely sampled image patches at different size level from the fully-connected layer of a pre-trained CNN, and concatenate the sematic features learned by the pLSA at each level. We comprehensively evaluate the two methods on two public HRRS scene datasets, and achieve significant performance improvement over the state-of-the-art. The outstanding results demonstrate that the pLSA model is capable of discovering considerably discriminative semantic features from the deep CNN features.

A novel unsupervised adversarial domain adaptation network for remotely sensed scene classification

ABSTRACT High-resolution remote sensing scene classification is a widely applicable task. Due to the diversity of natural scenes and acquisition methods, in different satellite images, scenes of the same class are often variable in texture, background, illumination and spatial resolution. Thus, it is hard for a remote sensing sense database to contain enough representative images. In this case, the generalization error of traditional supervised classification methods might be too large to generate ideal results. Domain adaptation (DA) has been applied to image classification by reducing feature distribution discrepancy between the source domain (where labels are available) and the target domain (where images need to be classified). In this paper, we propose an unsupervised adversarial domain adaptation method boosted by a domain confusion network (ADA-BDC) which aims at adapting the images from different domains to appear as if drawn from the same domain and improve the transferability of our classifier. For this purpose, the feature extractor in ADA-BDC makes the source and target distributions closer by training a Generative Adversarial Nets (GAN) model. After that, a transferred classifier trained by transferred source domain features is able to acquire a better classification accuracy on the target domain than a non-transferred classifier. In this paper, the experiments are conducted on four remote sensing scene benchmark datasets which are different in spatial scale, resolution, land-cover pattern, etc. Experimental results show that our proposed method is able to improve the transferability across different datasets and improve the classification overall accuracy by 17.18% on average. The comparative experiments demonstrate that the proposed DA network outperforms the compared state-of-the-art domain adaptation methods on remote sensing image scene classification.

Remote Sensing Scene Classification Using Convolutional Features and Deep Forest Classifier

High-resolution remote sensing scene classification (HR-RSSC) plays an increasingly important role since it aims to enhance the scene semantic understanding. Recently, convolutional neural networks (CNNs) proved their effectiveness in learning powerful feature representations for various visual recognition tasks. However, in the RS domain, the performance of CNN is still limited due to the lack of sufficient labeled data. In this letter, we propose an HR-RSSC method based on CNN transfer learning (TL) for feature extraction (FE) and deep forest (DF) for classification. In fact, we extract deep features from the last convolutional layer in order to avoid the use of the fully connected layers (FCLs) which need many parameters to tune. Moreover, we train a DF model that is based on ensemble learning that can achieve better performances than single classifiers and is easy to train with few parameters. We evaluate the proposed method on two RS image. Compared to full-training, fine-tuning, and state-of-the-art CNN TL methods, the results demonstrate the effectiveness of the DF model for HR-RSSC based on CNN TL in terms of overall accuracy and training time.

A Multi-Scale Approach for Remote Sensing Scene Classification Based on Feature Maps Selection and Region Representation

Scene classification is one of the bases for automatic remote sensing image interpretation. Recently, deep convolutional neural networks have presented promising performance in high-resolution remote sensing scene classification research. In general, most researchers directly use raw deep features extracted from the convolutional networks to classify scenes. However, this strategy only considers single scale features, which cannot describe both the local and global features of images. In fact, the dissimilarity of scene targets in the same category may result in convolutional features being unable to classify them into the same category. Besides, the similarity of the global features in different categories may also lead to failure of fully connected layer features to distinguish them. To address these issues, we propose a scene classification method based on multi-scale deep feature representation (MDFR), which mainly includes two contributions: (1) region-based features selection and representation; and (2) multi-scale features fusion. Initially, the proposed method filters the multi-scale deep features extracted from pre-trained convolutional networks. Subsequently, these features are fused via two efficient fusion methods. Our method utilizes the complementarity between local features and global features by effectively exploiting the features of different scales and discarding the redundant information in features. Experimental results on three benchmark high-resolution remote sensing image datasets indicate that the proposed method is comparable to some state-of-the-art algorithms.

An Improved Pretraining Strategy-Based Scene Classification With Deep Learning

High-resolution remote sensing (HRRS) image scene classification takes an important role in many applications and has attracted much attention. Recently, notable efforts have been made to present massive methods for HRRS scene classification, wherein deep-learning-based methods demonstrate remarkable performance compared with state-of-the-art methods. However, HRRS images contain complex contextual relationships and large differences of object scale, which are significantly different from natural images. The existing deep-learning-based scene classification methods are originally designed for natural image processing and have not been optimized to adapt to the characteristics of HRRS images, which significantly affects the efficiency of the feature extraction and recognition accuracy. In addition, when designing a model for remote sensing tasks, the pretraining of the model is time-consuming. The enormous amount of pretraining time and computation resources necessarily increase the difficulty of producing an excellent model. In this letter, focusing on the problems above, we proposed a new convolutional neural network (CNN)-based scene classification method. The CNN-based scene classification method is constructed by spatial-scale-aware blocks and is efficient in extracting the abundant spatial features, but can also adaptively adjust feature responses to maximize the function of informative features in the classification results. In addition, an HRRS imagery-based learning strategy is utilized to obtain an initial model for fine-tuning the model parameters, which drastically reduces the pretraining time. The proposed method has been demonstrated using two HRRS data sets, and experimental results have proven the superiority of the proposed method.

A novel two-stage scene classification model based on feature variable significance in high-resolution remote sensing

In this article, we focus on the high-resolution remote sensing scene classification, which aims to label the scene image according to its content. While the high similarity of the between-class and variety of the inner class in the scene images make it challenging. To address the issue, we propose a unified two-stage classification framework named variable-weighted multi-feature fusing (VWMF) method. In the first stage, we fuse multiple low-level features into a concatenated holistic histogram feature via pre-trained unsupervised models. Then, we use a kernel collaborative representation-based classification method to predict the two most possible categories. In the second stage, for the two possible categories, we calculate the variable-weighted feature through a pre-calculated feature variable weight set of the corresponding two categories, and then put the variable-weighted feature into the pre-trained SVM for one-versus-one classification. Comprehensive experiments on two publicly available data sets demonstrate the superiority of our VWMF method compared with state-of-the-art remote sensing scene classification methods.

A NOVEL FRAMEWORK FOR REMOTE SENSING IMAGE SCENE CLASSIFICATION

Abstract. High resolution remote sensing (HRRS) images scene classification aims to label an image with a specific semantic category. HRRS images contain more details of the ground objects and their spatial distribution patterns than low spatial resolution images. Scene classification can bridge the gap between low-level features and high-level semantics. It can be applied in urban planning, target detection and other fields. This paper proposes a novel framework for HRRS images scene classification. This framework combines the convolutional neural network (CNN) and XGBoost, which utilizes CNN as feature extractor and XGBoost as a classifier. Then, this framework is evaluated on two different HRRS images datasets: UC-Merced dataset and NWPU-RESISC45 dataset. Our framework achieved satisfying accuracies on two datasets, which is 95.57 % and 83.35 % respectively. From the experiments result, our framework has been proven to be effective for remote sensing images classification. Furthermore, we believe this framework will be more practical for further HRRS scene classification, since it costs less time on training stage.

A semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification

High resolution remote sensing (HRRS) image scene classification plays a crucial role in a wide range of applications and has been receiving significant attention. Recently, remarkable efforts have been made to develop a variety of approaches for HRRS scene classification, wherein deep-learning-based methods have achieved considerable performance in comparison with state-of-the-art methods. However, the deep-learning-based methods have faced a severe limitation that a great number of manually-annotated HRRS samples are needed to obtain a reliable model. However, there are still not sufficient annotation datasets in the field of remote sensing. In addition, it is a challenge to get a large scale HRRS image dataset due to the abundant diversities and variations in HRRS images. In order to address the problem, we propose a semi-supervised generative framework (SSGF), which combines the deep learning features, a self-label technique, and a discriminative evaluation method to complete the task of scene classification and annotating datasets. On this basis, we further develop an extended algorithm (SSGA-E) and evaluate it by exclusive experiments. The experimental results show that the SSGA-E outperforms most of the fully-supervised methods and semi-supervised methods. It has achieved the third best accuracy on the UCM dataset, the second best accuracy on the WHU-RS, the NWPU-RESISC45, and the AID datasets. The impressive results demonstrate that the proposed SSGF and the extended method is effective to solve the problem of lacking an annotated HRRS dataset, which can learn valuable information from unlabeled samples to improve classification ability and obtain a reliable annotation dataset for supervised learning.

Remote Sensing Scene Classification Using a Preclassification Strategy and an Improved Structural Feature

Due to the high intraclass variability and the low interclass disparity in high-resolution remote sensing (RS) image scenes, high-resolution RS scene classification is a challenging task. The performance of scene classification not only relies on discriminative feature representation but also needs appropriate classification strategies. In this paper, a scene preclassification strategy based on unsupervised learning is proposed, which divides scenes into two groups. The division method called dynamic-sphere division method (DSDM) is based on a dynamic-sphere division and an inside-sphere membership assessment. For the group with lower membership, after introducing the spatial location and scale of the scale invariant feature transformation (SIFT) descriptor to form a transaction, frequent itemset mining and an improved feature selection criterion are implemented to reduce the redundancy from the aspects of feature quantity and feature dimension, and a more discriminative structural feature histogram FMS-hist is finally obtained. Both the radius of the dynamic-sphere and the final optimal feature dimension are automatically selected according to the inflection point of the corresponding curves. Experimental results based on two representative data sets show that the proposed DSDM can select the suitable group, the proposed FMS-hist is superior to the bag-of-SIFT-based models. The holistic procedure can further enhance the scene classification accuracy.

Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery

Learning efficient image representations is at the core of the scene classification task of remote sensing imagery. The existing methods for solving the scene classification task, based on either feature coding approaches with low-level hand-engineered features or unsupervised feature learning, can only generate mid-level image features with limited representative ability, which essentially prevents them from achieving better performance. Recently, the deep convolutional neural networks (CNNs), which are hierarchical architectures trained on large-scale datasets, have shown astounding performance in object recognition and detection. However, it is still not clear how to use these deep convolutional neural networks for high-resolution remote sensing (HRRS) scene classification. In this paper, we investigate how to transfer features from these successfully pre-trained CNNs for HRRS scene classification. We propose two scenarios for generating image features via extracting CNN features from different layers. In the first scenario, the activation vectors extracted from fully-connected layers are regarded as the final image features; in the second scenario, we extract dense features from the last convolutional layer at multiple scales and then encode the dense features into global image features through commonly used feature coding approaches. Extensive experiments on two public scene classification datasets demonstrate that the image features obtained by the two proposed scenarios, even with a simple linear classifier, can result in remarkable performance and improve the state-of-the-art by a significant margin. The results reveal that the features from pre-trained CNNs generalize well to HRRS datasets and are more expressive than the low- and mid-level features. Moreover, we tentatively combine features extracted from different CNN models for better performance.