High-resolution Remote Sensing Image Scene Research Articles

Study on High-resolution Remote Sensing Image Scene Classification Using Transfer Learning

Remote sensing image classification (RSIC) has been increasingly concerned and becoming a challenging task. Recently, deep convolutional neural networks (DCNN) offer the effective classification method include the capacity to handle high-dimensional data and to distinguish classes with very complex characteristics on the remote sensing community. However, the focus of these methods is on publicly available data sets in the field of remote sensing, there are few studies on RSIC composed of different benchmark datasets, which the complexity, diversity, and similarity of data greatly increase the difficulty of classification. In this paper, we reconstructed and selected one new dataset from two standard benchmark remote sensing datasets: UC Merged Land-Use and NWPU-RESISC45. We utilize three transfer learning frameworks to extract the high-level feature map and feed feature information into the proposed model for partial and full fine-tuning. Data augmentation technology is used to increase the number of training samples and dropout strategies to prevent overfitting. The experimental results demonstrate that the proposed methodology achieved remarkable performance in scene classification of overall accuracy: 90.1%,91.0%,93.3 with VggNet, DesNet, InceptionNet, respectively.

Semi-Supervised DEGAN for Optical High-Resolution Remote Sensing Image Scene Classification

Semi-supervised methods have made remarkable achievements via utilizing unlabeled samples for optical high-resolution remote sensing scene classification. However, the labeled data cannot be effectively combined with unlabeled data in the existing semi-supervised methods during model training. To address this issue, we present a semi-supervised optical high-resolution remote sensing scene classification method based on Diversity Enhanced Generative Adversarial Network (DEGAN), in which the supervised and unsupervised stages are deeply combined in the DEGAN training. Based on the unsupervised characteristic of the Generative Adversarial Network (GAN), a large number of unlabeled and labeled images are jointly employed to guide the generator to obtain a complete and accurate probability density space of fake images. The Diversity Enhanced Network (DEN) is designed to increase the diversity of generated images based on massive unlabeled data. Therefore, the discriminator is promoted to provide discriminative features by enhancing the generator given the game relationship between two models in DEGAN. Moreover, the conditional entropy is adopted to make full use of the information of unlabeled data during the discriminator training. Finally, the features extracted from the discriminator and VGGNet-16 are employed for scene classification. Experimental results on three large datasets demonstrate that the proposed scene classification method yields a superior classification performance compared with other semi-supervised methods.

LPIN: A Lightweight Progressive Inpainting Network for Improving the Robustness of Remote Sensing Images Scene Classification

At present, the classification accuracy of high-resolution Remote Sensing Image Scene Classification (RSISC) has reached a quite high level on standard datasets. However, when coming to practical application, the intrinsic noise of satellite sensors and the disturbance of atmospheric environment often degrade real Remote Sensing (RS) images. It introduces defects to them, which affects the performance and reduces the robustness of RSISC methods. Moreover, due to the restriction of memory and power consumption, the methods also need a small number of parameters and fast computing speed to be implemented on small portable systems such as unmanned aerial vehicles. In this paper, a Lightweight Progressive Inpainting Network (LPIN) and a novel combined approach of LPIN and the existing RSISC methods are proposed to improve the robustness of RSISC tasks and satisfy the requirement of methods on portable systems. The defects in real RS images are inpainted by LPIN to provide a purified input for classification. With the combined approach, the classification accuracy on RS images with defects can be improved to the original level of those without defects. The LPIN is designed on the consideration of lightweight model. Measures are adopted to ensure a high gradient transmission efficiency while reducing the number of network parameters. Multiple loss functions are used to get reasonable and realistic inpainting results. Extensive tests of image inpainting of LPIN and classification tests with the combined approach on NWPU-RESISC45, UC Merced Land-Use and AID datasets are carried out which indicate that the LPIN achieves a state-of-the-art inpainting quality with less parameters and a faster inpainting speed. Furthermore, the combined approach keeps the comparable classification accuracy level on RS images with defects as that without defects, which will improve the robustness of high-resolution RSISC tasks.

Semantic Multigranularity Feature Learning for High-Resolution Remote Sensing Image Scene Classification

High-resolution remote sensing image scene classification is a challenging visual task due to the large intravariance and small intervariance between the categories. To accurately recognize the scene categories, it is essential to learn discriminative features from both global and local critical regions. Recent efforts focus on how to encourage the network to learn multigranularity features with the destruction of the spatial information on the input image at different scales, which leads to meaningless edges that are harmful to training. In this study, we propose a novel method named Semantic Multigranularity Feature Learning Network (SMGFL-Net) for remote sensing image scene classification. The core idea is to learn both global and multigranularity local features from rearranged intermediate feature maps, thus, eliminating the meaningless edges. These features are then fused for the final prediction. Our proposed framework is compared with a collection of state-of-the-art (SOTA) methods on two fine-grained remote sensing image scene datasets, including the NWPU-RESISC45 and Aerial Image Datasets (AID). We justify several design choices, including the branch granularities, fusion strategies, pooling operations, and necessity of feature map rearrangement through a comparative study. Moreover, the overall performance results show that SMGFL-Net consistently outperforms other peer methods in classification accuracy, and the superiority is more apparent with less training data, demonstrating the efficacy of feature learning of our approach.

Unsupervised Representation High-Resolution Remote Sensing Image Scene Classification via Contrastive Learning Convolutional Neural Network

Inspired by the outstanding achievement of deep learning, supervised deep learning representation methods for high-spatial-resolution remote sensing image scene classification obtained state-of-the-art performance. However, supervised deep learning representation methods need a considerable amount of labeled data to capture class-specific features, limiting the application of deep learning-based methods while there are a few labeled training samples. An unsupervised deep learning representation, high-resolution remote sensing image scene classification method is proposed in this work to address this issue. The proposed method, called contrastive learning, narrows the distance between positive views: color channels belonging to the same images widens the gaps between negative view pairs consisting of color channels from different images to obtain class-specific data representations of the input data without any supervised information. The classifier uses extracted features by the convolutional neural network (CNN)-based feature extractor with labeled information of training data to set space of each category and then, using linear regression, makes predictions in the testing procedure. Comparing with existing unsupervised deep learning representation high-resolution remote sensing image scene classification methods, contrastive learning CNN achieves state-of-the-art performance on three different scale benchmark data sets: small scale RSSCN7 data set, midscale aerial image data set, and large-scale NWPU-RESISC45 data set.

High-Resolution Remote Sensing Image Scene Classification by Merging Multilevel Features of Convolutional Neural Networks

The rapid development of remote sensing technologies has yielded a large amount of high-resolution remote sensing (HRRS) data. However, effectively extracting information from these large datasets is a significant challenge. In this study, a state-of-the-art convolutional neural network was extended for HRRS image classification by establishing an integrated structure including three branches of a pruned DenseNet, a decoder, and an encoder (PDDE-Net). PDDE-Net initially combines deep-level features extracted from the pruned DenseNet with middle-level features extracted from the designed encoder. Subsequently, the designed decoder branch extracts the merged features to obtain more feature expressions. In addition, the earlystop strategy is used to prevent overfitting during the training process. Experiments performed on an aerial image dataset demonstrated that the proposed network can achieve favorable overall accuracy and Kappa values of 90% and 0.897, respectively. Further, under the condition of imbalanced data, two merging methods—concatenation and summation—were applied to test the sensitivity of the integrated PDDE-Net to the feature size. The results on the remote sensing image classification benchmark datasets revealed little difference between the two merging methods, both of which exhibited accuracies greater than 95%. Moreover, the integrated PDDE-Net was tested on hyperspectral remote sensing data, and it achieved a high classification accuracy comparable to those of other advanced methods, which also demonstrated the generalizability of the proposed model.

Message-Passing-Driven Triplet Representation for Geo-Object Relational Inference in HRSI

A high-resolution remote sensing image (HRSI) scene typically contains multiple geo-objects, and geospatial relations among these geo-objects are obvious. As the important information conveyed by HRSI, the intelligent expression of geospatial relation is helpful in understanding HRSI scenes. Previous HRSI semantic understanding was mainly based on image captions that only generate one sentence to describe image content, thereby resulting in insufficient understanding of the scene. Thus, the present letter proposes an approach to represent geospatial relations in an HRSI scene with structured form of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\langle $ </tex-math></inline-formula> subject, geospatial relation, object <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\rangle $ </tex-math></inline-formula> . A geospatial relation triplet representation data set that contains visual and semantic information, such as category, location, and geospatial relations of the geo-objects, is constructed first. An “object-relation” message-passing mechanism is adopted to enhance the information exchange between the geo-objects and geospatial relations to predict triplets accurately. The experimental results show that the proposed method can effectively predict the geospatial relation in a HRSI scene.

Lightweight Channel Attention and Multiscale Feature Fusion Discrimination for Remote Sensing Scene Classification

High-resolution remote sensing image scene classification has attracted widespread attention as a basic earth observation task. Remote sensing scene classification aims to assign specific semantic labels to remote sensing scene images to serve specified applications. Convolutional neural networks are widely used for remote sensing image classification due to their powerful feature extraction capabilities. However, the existing methods have not overcome the difficulties of large-scene remote sensing images of large intraclass diversity and high interclass similarity, resulting in low performance. Therefore, we propose a new remote sensing scene classification method that combines lightweight channel attention and multiscale feature fusion discrimination, called LmNet. First, ResNeXt is used as the backbone; second, a new lightweight channel attention mechanism is constructed to quickly and adaptively learn the salient features of important channels. Furthermore, we designed a multiscale feature fusion discrimination framework, which fully integrates shallow edge feature information and deep semantic information to enhance feature representation capabilities and uses multiscale features for joint discrimination. Finally, a cross-entropy loss function based on label smoothing is built to reduce the influence of interclass similarity on feature representation. In particular, our lightweight channel attention and multiscale feature fusion mechanism can be flexibly embedded in any advanced backbone as a functional module. The experimental results on three large-scale remote sensing scene classification datasets show that compared with the existing advanced methods, our proposed high-efficiency end-to-end scene classification method has reached state-of-the-art. Moreover, our method has a weaker dependence on labeled data and provided better generalization performance.

Ensemble model with cascade attention mechanism for high-resolution remote sensing image scene classification.

Scene classification of high-resolution remote sensing images is a fundamental task of earth observation. And numerous methods have been proposed to achieve this. However, these models are inadequate as the number of labelled training data limits them. Most of the existing methods entirely rely on global information, while regions with class-specific ground objects determine the categories of high-resolution remote sensing images. An ensemble model with a cascade attention mechanism, which consists of two kinds of the convolutional neural network, is proposed to address these issues. To improve the generality of the feature extractor, each branch is trained on different large datasets to enrich the prior knowledge. Moreover, to force the model to focus on the most class-specific region in each high-resolution remote sensing image, a cascade attention mechanism is proposed to combine the branches and capture the most discriminative information. By experiments on four benchmark datasets, OPTIMAL-31, UC Merced Land-Use Dataset, Aerial Image Dataset and NWPU-RESISC45, the proposed end-to-end model cascade attention-based double branches model in this paper achieves state-of-the-art performance on each benchmark dataset.

Multi-deep features fusion for high-resolution remote sensing image scene classification

In view of the small number of categories and the relatively little amount of labeled data, it is challenging to apply the fusion of deep convolution features directly to remote sensing images. To address this issue, we propose a pyramid multi-subset feature fusion method, which can effectively fuse the deep features extracted from different pre-trained convolutional neural networks and integrate the global and local information of the deep features, thereby obtaining stronger discriminative and low-dimensional features. By introducing the idea of weighting the difference between different categories, the weight discriminant correlation analysis method is designed to make it pay more attention to those categories that are not easy to distinguish. In order to mine global and local feature information, the pyramid method is employed to divide feature fusion into several layers. Each layer divides the features into several subsets and then performs feature fusion on the corresponding feature subsets, and the number of subsets from top to bottom gradually increases. Feature fusion at the top of the pyramid obtains a global representation, while feature fusion at the bottom obtains a local detail representation. Our experiment results on three public remote sensing image data sets demonstrate that the proposed multi-deep features fusion method produces improvements over other state-of-the-art deep learning methods.

High-Resolution Remote Sensing Image Scene Classification via Key Filter Bank Based on Convolutional Neural Network

High-resolution remote sensing (HRRS) image scene classification has attracted an enormous amount of attention due to its wide application in a range of tasks. Due to the rapid development of deep learning (DL), models based on convolutional neural network (CNN) have made competitive achievements on HRRS image scene classification because of the excellent representation capacity of DL. The scene labels of HRRS images extremely depend on the combination of global information and information from key regions or locations. However, most existing models based on CNN tend only to represent the global features of images or overstate local information capturing from key regions or locations, which may confuse different categories. To address this issue, a key region or location capturing method called key filter bank (KFB) is proposed in this article, and KFB can retain global information at the same time. This method can combine with different CNN models to improve the performance of HRRS imagery scene classification. Moreover, for the convenience of practical tasks, an end-to-end model called KFBNet where KFB combined with DenseNet-121 is proposed to compare the performance with existing models. This model is evaluated on public benchmark data sets, and the proposed model makes better performance on benchmarks than the state-of-the-art methods.

Semisupervised Center Loss for Remote Sensing Image Scene Classification

High-resolution remote sensing image scene classification is a scene-level classification task. Driven by a wide range of applications, accurate scene annotation has become a hot and challenging research topic. In recent years, convolutional neural networks (ConvNets) have achieved promising performance among a variety of supervised classification methods. However, due to the lack of clearly labeled remote sensing images, it may be difficult to further improve the performance of scene classification. To address this issue, we propose a novel semisupervised center loss for scene classification. The main innovation of our method is to develop a cooperative framework of supervised and unsupervised branches in an end-to-end way. Specifically, we consider the class centers as guiding factors between the supervised and unsupervised branches. The supervised branch relies on a small number of labeled samples to generate class centers, which serve as initialization centers for the unsupervised branch. Meanwhile, the unsupervised branch utilizes the easily available remote sensing images to correct the class centers for enhancing the discriminative power of supervised ConvNets. Experimental results on three public benchmarks have indicated that the proposed method is superior to supervised center loss based methods.

A Combined Deep Learning Model for the Scene Classification of High-Resolution Remote Sensing Image

Deep learning now plays an important role in solving complex problems in computer vision fields. The highly challenging high-resolution remote sensing image scene classification problem can also be solved using deep learning methods. The most commonly used method of deep learning is the convolutional neural network model. In this letter, based on deep learning, a combined model named Inception-long short-term memory (LSTM) is proposed. First, we combine the deep learning feature extracted from the pretrained Inception-V3 model with a hand-crafted feature: the GIST feature. The different features are then combined and input into the batch normalization (BN) layer. Second, the BN layer plays the role of the bridge to combine the InceptionV3 model with the LSTM model, which features a softmax classifier. The LSTM model is used to analyze the features and classify the different high-resolution remote sensing scene images. The proposed model, as a whole, can be uniformly trained. Three different datasets-the NWPU-RESISC45 dataset, the UC Merced dataset, and the SIRI-WHU dataset-were used to verify the effectiveness of the proposed model. The results show that the proposed Inception-LSTM model shows an outstanding performance in the scene classification task.

Unsupervised Scene Change Detection via Latent Dirichlet Allocation and Multivariate Alteration Detection

Scene change detection is the process of identifying the differences between the multitemporal image scenes, which has significant potential in the application of urban development and land management at the semantic level. Traditional scene change detection methods are based on the supervised scene classification, and then directly compare the independent classification results without considering the temporal correlation between the unchanged regions. However, few studies have focused on detecting the semantic changes of multitemporal image scenes with unsupervised methods. In this paper, we propose a novel unsupervised scene change detection method by using latent Dirichlet allocation (LDA) and multivariate alteration detection (MAD). First, the scene is represented by the bag-of-visual-words model, and adopt the union dictionary to ensure the consistency of dictionary space. Then, LDA is used to achieve the middle-level feature dimension reduction, and generate the common topic space of the two multitemporal image scene datasets. And finally, the MAD method was applied to detect the semantic changes of corresponding multitemporal image scenes. Two experiments with high-resolution remote sensing image scene datasets demonstrated that our proposed approach can get a better performance in unsupervised scene change detection without prior knowledge.

Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification

Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification

A semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification

High resolution remote sensing (HRRS) image scene classification plays a crucial role in a wide range of applications and has been receiving significant attention. Recently, remarkable efforts have been made to develop a variety of approaches for HRRS scene classification, wherein deep-learning-based methods have achieved considerable performance in comparison with state-of-the-art methods. However, the deep-learning-based methods have faced a severe limitation that a great number of manually-annotated HRRS samples are needed to obtain a reliable model. However, there are still not sufficient annotation datasets in the field of remote sensing. In addition, it is a challenge to get a large scale HRRS image dataset due to the abundant diversities and variations in HRRS images. In order to address the problem, we propose a semi-supervised generative framework (SSGF), which combines the deep learning features, a self-label technique, and a discriminative evaluation method to complete the task of scene classification and annotating datasets. On this basis, we further develop an extended algorithm (SSGA-E) and evaluate it by exclusive experiments. The experimental results show that the SSGA-E outperforms most of the fully-supervised methods and semi-supervised methods. It has achieved the third best accuracy on the UCM dataset, the second best accuracy on the WHU-RS, the NWPU-RESISC45, and the AID datasets. The impressive results demonstrate that the proposed SSGF and the extended method is effective to solve the problem of lacking an annotated HRRS dataset, which can learn valuable information from unlabeled samples to improve classification ability and obtain a reliable annotation dataset for supervised learning.

Kernel Slow Feature Analysis for Scene Change Detection

Scene change detection between multitemporal image scenes can be used to interpret the variation of regional land use, and has significant potential in the application of urban development monitoring at the semantic level. The traditional methods directly comparing the independent semantic classes neglect the temporal correlation, and thus suffer from accumulated classification errors. In this paper, we propose a novel scene change detection method via kernel slow feature analysis (KSFA) and postclassification fusion, which integrates independent scene classification with scene change detection to accurately determine scene changes and identify the “from-to” transition type. After representation with the bag-of-visual-words model, KSFA is proposed to extract the nonlinear temporally invariant features, to better measure the change probability between corresponding multitemporal image scenes. Two postclassification fusion methods, which are based on Bayesian theory and predefined rules, respectively, are then employed to identify the optimal coupled class combinations of multitemporal scene pairs. Furthermore, in addition to identifying semantic changes, the proposed method can also improve the performance of scene classification, since the unchanged scenes are more likely to belong to the same class. Two experiments with high-resolution remote sensing image scene data sets confirm that the proposed method can increase the accuracy of scene change detection, scene transition identification, and scene classification.