Remote Sensing Image Classification Research Articles

Contrastive Learning for Fine-Grained Ship Classification in Remote Sensing Images

Fine-grained image classification can be considered as a discriminative learning process where images of different subclasses are separated from each other while the same subclass images are clustered. Most existing methods perform synchronous discriminative learning in their approaches. Although achieving promising results in fine-grained visual classification (FGVC) in natural images, these methods may fail in fine-grained ship classification (FGSC) problem in remote sensing (RS) images due to the highly “imbalanced fineness" and “imbalanced appearances" of ships among subclasses. To tackle the issue, we propose an asynchronous contrastive learning-based method for effective FGSC. The proposed method, which we refer to as “Push-and-Pull Network (P2Net)", includes a “push-out stage” and a “pull-in stage”, where the first stage forces all the instances to be de-correlated and then the second one groups them into each subclass. A dual-branch network is designed to separate/de-correlate the images with each other, while an Integration Module is designed to aggregate the de-correlated images into their corresponding subclass together with a Proxy-based Module designed for acceleration. In this way, the correlation between subclasses can be decoupled, which in turn makes the final classification much easier. Our method can be trained end-to-end and requires no additional annotations other than category information. Extensive experiments are conducted on two large-scale FGSC datasets (FGSC-23 and FGSCR-42). Our method outperforms other state-of-the-art approaches. Ablation experiments also suggest the effectiveness of our design. Our code is available at https://github.com/WindVChen/Push-and-Pull-Network.

Separable Attention Network in Single- and Mixed-Precision Floating Point for Land-Cover Classification of Remote Sensing Images

Land-cover information is of paramount importance in a wide range of environmental and socioeconomic applications. Deep learning (DL) provides a large variety of potential models for extracting useful information from raw images. However, remote sensing image (RSI) classification remains a challenging goal due to the intrinsic features of the data, such as the high sample variability and lack of labeled data. This provides a challenge to the reliability of deep classifiers. In particular, convolution-based models are greatly affected by overfitting and vanishing gradient problems. To overcome these drawbacks, this letter presents a new attention-based architecture, including attention modular blocks. These blocks divide their input feature maps into several groups and split them along the channel dimension and then combine them to create an attention mask encoding global contextual information. The mask is applied to obtain a refined feature representation, strengthening those features that affect most significantly the classification and attenuating the rest. Our new method reduces significantly the number of trainable parameters. Our results, obtained using several widely used RSIs, demonstrate that the new method exhibits higher classification performance when compared to several state-of-the-art methods.

ECA-MobileNetV3(Large)+SegNet Model for Binary Sugarcane Classification of Remotely Sensed Images

In recent years, several models based on fully convolutional neural networks have been proposed. These models mainly focused on improving accuracy but ignored computational efficiency. For this reason, this research proposes an innovative deep learning model, entitled “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ECA-MobileNetV3(large)+Seg-Net model</i> ” for simultaneously concerning both. In the encoder, the structure-reduced MobileNetV3(large) is selected as the backbone network, which uses 11 network layers to replace 20 network layers of MobileNetV3(large). The hybrid dilated convolution with dilation rates of 1, 2, and 4 is introduced in the depthwise separable convolution to expand the local receptive field and to enhance the connection of contextual information. Finally, the ECA module is fused with the bneck structure. In the decoder, the 18 network layers of SegNet’s decoder are replaced with 9 layers to achieve a lightweight network with small parameters. When compared with the original SegNet, the overall accuracy (OA) of the model proposed in this research is averagely improved by 5.97%, 5.56%, and 4.13%, and the sugarcane identification accuracy is averagely improved by 7.16%, 4.01%, and 9.13%, respectively in the three tested areas. Additionally, the memory size, the number of parameters, and FLOPs are all reduced by 6/7.

Transfer Representation Learning Meets Multimodal Fusion Classification for Remote Sensing Images

To maximize the complementary advantages of synergistic multimodal, a transfer representation learning fusion network (TRLF-Net) is proposed for multisource remote sensing images collaborative classification in this article. First, with respect to the feature encoding, we design a dual-branch attention sparse transfer module (DAST-Module), which combines the spatial and channel attention (CA) masks to migrate the advantage attributes of the panchromatic (PAN) and the MS images mutually. This not only enhances their respective image advantages but also facilitates the sparse fusion of low-level features. Second, for the separation of multiscale information, a deep dual-scale decomposition module (DDSD-Module) is designed, which allows the decompose of high-frequency and low-frequency components. Then it uses the decomposed information to make the essential difference as small as possible, and the surrounding contour difference is as large as possible of the complementary multimodal image through the design of the loss function. Finally, to address the problem of large intraclass and small interclass differences, we develop a representation fusion of the global and local features’ module (RFGAL-Module). It mainly adopts global features to sort local features within classes, and then outputs them in a cascade. Thus, the characterization ability of features is improved, and the global and local features are used in a coordinated manner to accomplish the sample classification tasks. In particular, the experimental results demonstrate that TRLF-Net can obtain much improved accuracy and efficiency. The code is accessible in: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ru-willow/SRLF-Net</uri> .

Complementary Learning-Based Scene Classification of Remote Sensing Images With Noisy Labels

Recently, many deep convolutional neural network (DCNN)-based methods have been proposed for remote sensing (RS) image scene classification (SC). In general, DCNNs obtain good generalization capabilities under the condition of correct labels. Unfortunately, the given samples are sometimes mislabeled. In this letter, the classification of RS images with noisy labels is investigated. First, complementary learning (CL), which learns from complementary labels rather than the original labels, is introduced for RS image classification with noisy labels. CL can decrease the probability of learning from incorrect information, and therefore, it is robust to noisy labels. Then, soft CL, which randomly disturbs the complementary labels of the training samples, is proposed to prevent the overfitting issue in training a DCNN. Moreover, an RS image scene classification framework combining ordinary learning (OL) and CL (RS-COCL) is proposed, which uses CL to obtain a good model and OL to fine-tune the deep model. Additionally, noisy labels filtering is used in RS-COCL (RS-COCL-NLF) to detected and corrected noisy samples. At last, soft CL is used in RS-COCL-NLF to obtain better classification performance. The proposed methods are tested on two widely used datasets (i.e., Northwestern Polytechnical University (NWPU)-RESISC45 and PatternNet) and the obtained results show that the proposed methods provide competitive classification accuracy compared to the state-of-the-art methods.

Relation-Attention Networks for Remote Sensing Scene Classification

Remote sensing (RS) scene classification plays an important role in a wide range of RS applications. Recently, convolutional neural networks (CNNs) have been applied to the field of scene classification in RS images and achieved impressive performance. However, to classify RS scenes, most of the existing CNN methods either utilize the high-level features from the last convolutional layer of CNNs, missing much important information existing at the other levels, or directly fuse the features at different levels, bringing redundant and/or mutually exclusive information. Inspired by the attention mechanism of the human visual system, in this article, we explore a novel relation-attention model and design an end-to-end relation-attention network (RANet) to learn powerful feature representations of multiple levels to further improve the classification performance. First, we propose to extract convolutional features at different levels by pretrained CNNs. Second, a multiscale feature computation module is constructed to connect features at different levels and generate multiscale semantic features. Third, a novel relation-attention model is designed to focus on the critical features whilst avoiding the use of redundant and even distractive ones by exploiting the scale contextual information. Finally, the resulting relation-attention features are concatenated and fed into a softmax layer for the final classification. Experiments on four well-known RS scene classification datasets (UC-Merced, WHU-RS19, AID, and OPTIMAL-31) show that our method outperforms some state-of-the-art algorithms.

Remote Sensing Image Classification Using CNNs With Balanced Gradient for Distributed Heterogeneous Computing

Remote Sensing Image Classification Using CNNs With Balanced Gradient for Distributed Heterogeneous Computing

A Two-Stage Adaptation Network (TSAN) for Remote Sensing Scene Classification in Single-Source-Mixed-Multiple-Target Domain Adaptation (S²M²T DA) Scenarios

Over the past decade, domain adaptation (DA) algorithms have been proposed to address domain gap problems as they do not need any interpretation in the target domain. However, most existing efforts focus on scenarios with only one source domain and one target domain. In this article, we explore the scenario with one source domain and mixed multiple target domains for remote sensing applications and propose a new algorithm, named the two-stage adaptation network (TSAN). First, we utilize the adversarial learning approach to confuse the classifier to discriminate between the source domain and the whole mixed-multiple-target domain. Second, we adopt self-supervised learning to divide the mixed-multiple-target domain with automated generation of “pseudo”-domain labels, which guides our network to learn intrinsic features of multiple target domains. Finally, these two steps are combined as an iterative procedure. We integrate a test dataset that includes five remote sensing datasets and ten classes. Our method achieves an average accuracy of 63.25% and 73.68% with two typical backbones, considerably outperforming other DA methods with an average accuracy improvement of 4.84%–20.19% and 9.06%–17.04%, respectively. Furthermore, we identify the negative transfer effect in existing mainstream DA methods in remote sensing image classification with multiple different domains.

Grouped Subspace Linear Semantic Alignment for Hyperspectral Image Transfer Learning

Transfer learning (TL) offers an effective way to reduce the demand for labeled samples in remote sensing image classification. However, existing TL methods have some limitations. Simple linear TL methods cannot align the source and target domains well when the data shift is complicated, while nonlinear methods consist of many learnable parameters and often need many labeled samples. To overcome these issues, we design a novel grouped subspace linear semantic alignment (G-SLSA) algorithm, which consists of four main ingredients. First, inspired by the linear supervised transfer learning (LSTL) approach, we propose subspace linear semantic alignment (SLSA) aiming to reduce the demand for labeled samples. Second, considering the heterogeneity of class-level data shift, we extend SLSA to G-SLSA through a grouped alignment strategy, which can reduce the data shift by decomposing a multiclass transfer learning task into a set of binary subtasks. Third, considering the demand of subtasks fusion on posterior probabilities, we propose a robust posterior probability estimation method for the binary generalized learning vector quantization (GLVQ) that is used in G-SLSA. Finally, a pairwise coupling (PWC) method is applied to fuse the results of each subtask. Experimental results conducted on three popular hyperspectral datasets demonstrate that G-SLSA outperforms other traditional and state-of-the-art deep learning (DL) methods, with an OA of 80.78&#x00B1;4.28% for PC-UP transfer learning scenario when 5 samples per class are available in the target domain.

Network Pruning for Remote Sensing Images Classification Based on Interpretable CNNs

Convolutional neural network (CNN)-based research has been successfully applied in remote sensing image classification due to its powerful feature representation ability. However, these high-capacity networks bring heavy inference costs and are easily overparameterized, especially for the deep CNNs pretrained on natural image datasets. Network pruning is regarded as a prevalent approach for compressing networks, but most existing research ignores model interpretability while formulating pruning criterion. To address these issues, a network pruning method for remote sensing image classification based on interpretable CNNs is proposed. More specifically, an original interpretable CNN with a predefined pruning ratio is trained at first. The filters, namely channels in the high convolutional layer, are able to learn specific semantic meanings in proportion to the predefined pruning ratio. The filters without interpretability are supposed to be removed. As for other convolutional layers, a sensitivity function is designed to assess the risk of pruning channels for each layer, and furthermore, the pruning ratio for each layer is corrected adaptively. The pruning method based on the proposed sensitivity function is effective and requires little computational costs to search abandoned channels without damaging classification performance. To demonstrate the effectiveness, the proposed method is implemented on different scales of modern CNN models, including VGG-VD and AlexNet. The experimental results, obtained on the UC Merced dataset and NWPU-RESISC45 dataset, prove that our method significantly reduces the inference costs and improves the interpretability of networks.

Self-Supervised Feature Representation and Few-Shot Land Cover Classification of Multimodal Remote Sensing Images

Although deep learning-based approaches have made significant progress in remote sensing image classification, the supervised learning paradigm has shortcomings under a limited number of labeled samples, which restricts the classification performance to a great extent. In this article, we investigate an effective self-supervised feature representation architecture (SSFR) for multimodal remote sensing images few-shot land cover classification. Specifically, we exploit multiview learning strategy to construct multiple views from multimodal remote sensing images. This method builds several complementary views of the same observed scenes from hyperspectral images or different modalities of remote sensing data. Then we build the deep feature extractor to learn high-level feature representations from each view via contrastive learning. The contrastive learning aggregates the samples of the same scene while separating samples of different scenes in the latent space, and this process does not require any labeled information. What’s more, to learn more robust features from different views, we utilize multitask learning strategy to train the feature extraction network. Finally, a lightweight machine learning method is employed to classify the learned features using a few annotated samples. To further demonstrate the self-supervised feature learning capability of the proposed model, we train the feature representation network in multiple source datasets. Comprehensive feature learning and classification experiments have certified the effectiveness and superiority of the proposed method.

Multiple Embeddings Contrastive Pretraining for Remote Sensing Image Classification

This letter focuses on remote sensing image interpretation and aims to promote the use of contrastive self-supervised learning in varied applications of remote sensing image classification. The proposed method is a contrastive self-supervised pre-training framework that encourages the network to learn image representations by comparing image embeddings extracted by different encoders and predictors. Experiments were carried out on a variety of remote sensing image datasets to determine the efficacy of the proposed method for classification tasks. Results show that the proposed framework exploits the capabilities of encoders and outperforms the supervised learning method in terms of classification accuracy. Besides, it takes a few pre-training epochs to find a suboptimal initialization of network weights, and the pre-trained encoders use a little training data to get outstanding classification results, which shows the time and data efficiency of the proposed framework. Code is available at https://github.com/yinxu98/MECo.

Few-Shot SAR Target Classification via Metalearning

The state-of-the-art deep neural networks have made a great breakthrough in remote sensing image classification. However, the heavy dependence on large-scale data sets limits the application of the deep learning to synthetic aperture radar (SAR) automatic target recognition (ATR) field where the target sample set is generally small. In this work, a metalearning framework named MSAR, consisting of a metalearner and a base-learner, is proposed to solve the sample restriction problem, which can learn a good initialization as well as a proper update strategy. After training, MSAR can implement fast adaptation with a few training images on new tasks. To the best of our knowledge, this is the first study to solve a few-shot SAR target classification via metalearning. In particular, the few-task problem is defined by analyzing the effect of available training classes on the performance of metalearning models. In order to reduce the metalearning difficulties caused by the few-task problem, three transfer-learning methods are employed, which can leverage the prior knowledge from the pretraining phase. Besides, we design a hard task mining method for effective metalearning. Based on the Moving and Stationary Target Acquisition and Recognition (MSTAR) data set, a specialized data set named NIST-SAR is devised to train and evaluate the proposed method. The experiments on NIST-SAR have shown that the proposed method yields better performances with the largest absolute improvements of 1.7% and 2.3% for 1-shot and 5-shot, respectively, over the next best, which indicates that the proposed method is promising and metalearning is a feasible solution for few-shot SAR ATR.

Ship Classification and Detection Method for Optical Remote Sensing Images Based on Improved YOLOv5s

Ship Classification and Detection Method for Optical Remote Sensing Images Based on Improved YOLOv5s

Mono- and Dual-Regulated Contractive-Expansive-Contractive Deep Convolutional Networks for Classification of Multispectral Remote Sensing Images

Deep Convolutional Neural Networks are state-of-the-art methods in the domain of classification of remote sensing (RS) data. However, traditional CNN models suffer from huge computational costs in learning land-use and land-cover features, particularly in large scale RS problems. To address this issue, we propose a reliable mono and dual Regulated contractive-expansive-contractive (CEC) convolutional neural network for scene based multispectral (MS) image classification. The proposed technique increases the accuracy of learning and minimizes the loss in the feature maps by incorporating the CEC approach in the classification. Extensive experiments conducted on the Sentinel-2 EuroSATallbands dataset pointed out that the proposed model outperforms state-of-the-art models such as EfficientNet-B0, RESNet-50 and EfficientNet-B7.

Novel Cross-Resolution Feature-Level Fusion for Joint Classification of Multispectral and Panchromatic Remote Sensing Images

With the increasing availability and resolution of satellite sensor data, multispectral (MS) and panchromatic (PAN) images are the most popular data that are used in remote sensing among applications. This article proposes a novel cross-resolution hidden layer feature fusion (CRHFF) approach for joint classification of multiresolution MS and PAN images. In particular, shallow spectral and spatial features at a global scale are first extracted from an MS image. Then, deep cross-resolution hidden layer features extracted from MS and PAN are fused from patches at a local scale according to an autoencoder (AE)-like deep network. Finally, the selected multiresolution hidden layer features are classified in a supervised manner. By taking advantage of integrated shallow-to-deep and global-to-local features from the high-resolution MS and PAN images, the cross-resolution latent information can be extracted and fused in order to better model imaged objects from the multimodal representation and finally increase the classification accuracy. Experimental results obtained on three real multiresolution datasets covering complex urban scenarios confirm the effectiveness of the proposed approach in terms of higher accuracy and robustness with respect to literature methods.

Few-Shot Scene Classification of Optical Remote Sensing Images Leveraging Calibrated Pretext Tasks

Small data holds big AI potential. As one of the promising small data AI approaches, few-shot learning has the goal to learn a model efficiently that can recognize novel classes with extremely limited training samples. Therefore it is critical to accumulate useful prior knowledge obtained from large-scale base class dataset. To realize few-shot scene classification of optical remote sensing images, we start from a baseline model that trains all base classes using a standard cross-entropy loss leveraging two auxiliary objectives to capture intrinsical characteristics across the semantic classes. Specifically, rotation prediction learns to recognize the 2D rotation of an input to guide the learning of class-transferable knowledge, and contrastive learning aims to pull together the positive pairs while pushing apart the negative pairs to promote intra-class consistency and inter-class inconsistency. We jointly optimize such two pretext tasks and semantic class prediction task in an end-to-end manner. To further overcome the overfitting issue, we introduce a regularization technique, adversarial model perturbation, to calibrate the pretext tasks so as to enhance the generalization ability. Extensive experiments on public remote sensing benchmarks including NWPU-RESISC45, AID, and WHU-RS-19 demonstrate that our method works effectively and achieves best performance that significantly outperforms many state-of-the-art approaches.

Land Cover Classification of Resources Survey Remote Sensing Images Based on Segmentation Model

Land type survey is an important task of land resources survey and the basis of scientific management of land resources. With the increasingly prominent problems of population, resources, and environment, there is an urgent need for a fast and accurate classification method of large-scale land use and land cover based on remote sensing data. Traditional machine learning classification methods based on pixel classification achieved sufficient results and are widely used, such as maximum likelihood classification and random forests method. However, with the development of the novel technology of deep learning, in practical application, for multi-classified land resources, how to use the fast and effective classification method of low and medium resolution RS images needs further research. This paper takes the land resource classification of the Tonghe medium resolution RS dataset of the third land survey in China as an example to screen and compare traditional machine learning classification methods and semantic segmentation models FC-DenseNet56, GCN, BiSeNet, U-Net, DeepLabV3, AdapNet, and PSPNet, which aim to select the optimal feature extraction model. The results show that the classification accuracy of the U-Net model can reach 93.62%, which is more accurate and effective than traditional machine learning methods and other semantic segmentation models. It is suitable for multi-classification tasks of land cover resources in low and medium resolution RS images and shows a superior effect in practical application. Besides, the conclusion of this study can provide a demonstration for large-scale land cover resources investigation using low and medium resolution RS images.

A Multicenter Soft Supervised Classification Method for Modeling Spectral Diversity in Multispectral Remote Sensing Data

Due to the spectral diversity of objects within the same classes and the spectral similarity between different classes, classical fuzzy classification methods perform poorly in multispectral remote sensing image classification. The basic reason is that they often use a single spectral curve to represent a land cover type while ignoring spectral diversity characteristics. To solve this issue, this article proposes a novel multicenter supervised fuzzy classification (MCSFC) method for modeling spectral diversity in multispectral remote sensing data. Images are first split into clusters with similar sizes or volumes, namely, granularities, by a hierarchical clustering process. Second, the granularities are labeled by training samples. As a result, the centers of the granularities corresponding to one type of sample can represent the spectral diversity within one class and, thus, are treated as the multiple centers of a land cover type. The membership degree of each unlabeled sample belonging to any land cover type is determined by the shortest distance to the centers of each type, and the spectral diversity is considered in this stage. A comparison reveals that the proposed method clearly improves the classification performance of various single-center fuzzy semisupervised clustering (FSSC) methods. Two case studies indicate that the granularity volume parameter notably affects the classification performance, and the overall accuracy (OA) decreases with increasing granularity volume.

A Collaborative Correlation-Matching Network for Multimodality Remote Sensing Image Classification

Recently, with the increasing availability of the high-quality panchromatic (PAN) and multi-spectral (MS) remote sensing images, the inherent complementarity between PAN and MS images provides a wide development prospect for the multi-modality remote sensing image classification task. However, how to cleverly relieve the modal differences and effectively integrate the single-modality PAN and MS features is still a challenge. In this paper, we design a Collaborative Correlation-Matching Network (CCM-Net) for multi-modality remote sensing image classification. Concretely, we first propose a Bi-directional Dominant Feature Supervision (Bi-DFS) learning, it utilizes single-modality dominant features as supplementary supervision information to establish the joint optimization loss function, thereby adaptively narrowing the differences between modalities before the feature extraction. In the feature extraction stage, the Interactive Correlation Feature Matching (ICFM) learning, composing the Spa-FM and Spe-FM strategies, is proposed to establish interactive matching and enhancement between multi-modality strong correlation features from the perspective of spatial and spectral, respectively, thereby effectively alleviating the semantic deviation of multi-modality features. Finally, we aggregate finer multi-level multi-modality features to obtain top-level features with high discrimination. The effectiveness of the proposed algorithm has been verified on multiple data sets. Our code is available at: https://github.com/Momuli/CCM-Net.git.