Performance Of Scene Classification Research Articles

Scene Classification of Remote Sensing Image Based on Multi-Path Reconfigurable Neural Network

Land image recognition and classification and land environment detection are important research fields in remote sensing applications. Because of the diversity and complexity of different tasks of land environment recognition and classification, it is difficult for researchers to use a single model to achieve the best performance in scene classification of multiple remote sensing land images. Therefore, to determine which model is the best for the current recognition classification tasks, it is often necessary to select and experiment with many different models. However, finding the optimal model is accompanied by an increase in trial-and-error costs and is a waste of researchers’ time, and it is often impossible to find the right model quickly. To address the issue of existing models being too large for easy selection, this paper proposes a multi-path reconfigurable network structure and takes the multi-path reconfigurable residual network (MR-ResNet) model as an example. The reconfigurable neural network model allows researchers to selectively choose the required modules and reassemble them to generate customized models by splitting the trained models and connecting them through modules with different properties. At the same time, by introducing the concept of a multi-path input network, the optimal path is selected by inputting different modules, which shortens the training time of the model and allows researchers to easily find the network model suitable for the current application scenario. A lot of training data, computational resources, and model parameter experience are saved. Three public datasets, NWPU-RESISC45, RSSCN7, and SIRI-WHU datasets, were used for the experiments. The experimental results demonstrate that the proposed model surpasses the classic residual network (ResNet) in terms of both parameters and performance.

When complementarity meets consistency: weighted collaboration fusion constrained by consistency between views for multi-view remote sensing scene classification

ABSTRACTRemote sensing scene classification is the basis of advanced smart urban planning tasks such as urban functional zone division and land use type identification. In recent years, a wide range of emerging data sources is receiving progressive attention for urban features extraction, such as satellites, unmanned aerial vehicles (UAVs), and ground sensors. How to effectively utilize these multi-view data jointly to improve scene classification performance has become a hot topic of remote sensing challenge. Existing feature fusion methods tend to map data from different views into a common feature space, which is often difficult to find when the data between views differ greatly. Furthermore, because these methods require data from all views as input, they are not flexible enough to handle situations where there is only one view input when inferring. To address the aforementioned issues, a novel Coupled Parallel Architecture (CPA) using Weighted Collaboration Fusion Constrained by Consistency Between Views (CBV-WCF) is proposed in this paper. In the training phase, the CBV module reduces the impact of the heterogeneous gap across views by capturing the consistency information between views. Well the WCF module is used to fully mine and effectively fuse the complementary information between views to improve the performance of downstream tasks. In the inference phase, the proposed architecture can effectively improve the classification performance in both cases of multi-view and single-view input. Our method is evaluated on air-ground dual-view scene classification, which is a typical multi-view task with large image differences between views. Experimental results on two publicly available air-ground dual-view datasets demonstrate that the proposed framework significantly improves classification performance while bringing some inspiration and new solutions to multi-view tasks. The code of this paper will be published at: https://github.com/Forest-repo/CBV-WCF.

Study on High-resolution Remote Sensing Image Scene Classification Using Transfer Learning

Remote sensing image classification (RSIC) has been increasingly concerned and becoming a challenging task. Recently, deep convolutional neural networks (DCNN) offer the effective classification method include the capacity to handle high-dimensional data and to distinguish classes with very complex characteristics on the remote sensing community. However, the focus of these methods is on publicly available data sets in the field of remote sensing, there are few studies on RSIC composed of different benchmark datasets, which the complexity, diversity, and similarity of data greatly increase the difficulty of classification. In this paper, we reconstructed and selected one new dataset from two standard benchmark remote sensing datasets: UC Merged Land-Use and NWPU-RESISC45. We utilize three transfer learning frameworks to extract the high-level feature map and feed feature information into the proposed model for partial and full fine-tuning. Data augmentation technology is used to increase the number of training samples and dropout strategies to prevent overfitting. The experimental results demonstrate that the proposed methodology achieved remarkable performance in scene classification of overall accuracy: 90.1%,91.0%,93.3 with VggNet, DesNet, InceptionNet, respectively.

Multispectral-to-RGB Knowledge Distillation for Remote Sensing Image Scene Classification

Scene classification is a fundamental task in the remote sensing (RS) field, assigning semantic labels to RS images. Multispectral (MS) images play an essential role in scene classification as they contain richer spectral information than RGB images. However, MS images are not always available due to the higher cost and complexity of MS sensors compared to RGB sensors. To improve scene classification performance using only RGB images, in this letter, we propose a novel MS-to-RGB knowledge distillation framework (MS2RGB-KD) that transfers MS knowledge from a teacher model to a student model. Specifically, our MS2RGB-KD drives a student model that requires only an RGB image as input to mimic the feature representations of different modalities extracted by the teacher model. Moreover, we introduce novel loss functions that encourage the student model to preserve intra-modal and inter-modal relationships of the feature representations in the teacher model. Experiments on the EuroSAT dataset demonstrate the effectiveness of MS2RGB-KD compared with other KD baselines.

ASYMMETRIC FUZZY CLASSIFICATION NETWORKS FOR CONSTRUCTION LAND DETECTION IN HIGH RESOLUTION REMOTE SENSING IMAGES

Abstract. Urbanization is an essential phase of a nation's economic development. A very effective way to examine urban growth is to look at how impervious surface changes over time, however impervious surface can only show the current situation in terms of urban development. Compared to the existing methods, the use of construction bare land to monitor urban growth has the following benefits over currently used techniques. First, it is possible to track the progress of structures being constructed as part of urban expansion. The second is to assess the city's development intensity and identify the inward expansion. Therefore, the detection of construction bare land is of great significance for the development of a more sophisticated urban dynamic perception technology. This paper proposes an asymmetric fuzzy classification network (AFCNet) for detection of construction bare land scenes. The generation of fuzzy sample sets, the backbone network, and the proposed fuzzy classification module make up the method's three primary components. The deep features of the scene are then extracted using the fuzzy classification network and converted into ambiguity. Finally, the ambiguity is converted into predicted probability using the fuzzy weight vector. The fuzzy sample set is generated to introduce more prior information into the network. High-level features are extracted using the backbone network. Fuzzy classification methods based on spectral features are used to improve the performance of scene classification. The results demonstrate that the OA of our method is higher than all other comparison methods.

Task-specific contrastive learning for few-shot remote sensing image scene classification

Deep neural network has been successfully applied to remote sensing image scene classification, which requires a large amount of annotated data for training. However, it is time-consuming and labor-intensive to obtain abundant labeled samples in various applications. Therefore, it is of great importance to conduct scene classification with only a few annotated images. In order to address the issue, we propose a task-specific contrastive learning (TSC) model for few-shot scene classification of remote sensing images, which aims to enhance the scene classification performance with fewer labeled samples. Specifically, a self-attention and mutual-attention module (SMAM) is developed to learn feature correlations and reduce the background interference. Moreover, a task-specific contrastive loss function is proposed to optimize the deep network, which generates task-specific paired data based on different views of original images. This strategy has a contribution to enhance the discrimination of features between intra-class and inter-class images. Experimental results on NWPU-RESISC45, WHU-RS19 and UCM datasets demonstrate that the proposed method produces superior accuracies compared with other related few-shot learning methods.

Dance Art Scene Classification Based on Convolutional Neural Networks

Digital multimedia resources have become an important part of people’s daily cultural life. Automatic scene classification of a large number of dance art videos is the basis for scene semantic based video content retrieval. In order to improve the accuracy of scene classification, the videos are identified using a deep convolutional neural network based on differential evolution for dance art videos. First, the Canny operator is used in YCbCr colour space to detect the human silhouette in the key frames of the video. Then, the AdaBoost algorithm based on cascade structure is used to implement human target tracking and labelling, and the construction and updating of weak classifiers are analysed. Next, a differential evolution algorithm is used to optimise the structural parameters of the convolutional neural network, and an adaptive strategy is adopted for the scaling factor of the differential evolution algorithm to improve the optimisation solution accuracy. Finally, the improved deep convolutional neural network is used to train the classification of the labelled videos in order to obtain stable scene classification results. The experimental results show that by reasonably setting the crossover rate of differential evolution and the convolutional kernel size of the convolutional neural network, high scene classification performance can be obtained. The high accuracy and low root-mean-square error validate the applicability of the proposed method in dance art scene classification.

Robust Object Categorization and Scene Classification over Remote Sensing Images via Features Fusion and Fully Convolutional Network

The latest visionary technologies have made an evident impact on remote sensing scene classification. Scene classification is one of the most challenging yet important tasks in understanding high-resolution aerial and remote sensing scenes. In this discipline, deep learning models, particularly convolutional neural networks (CNNs), have made outstanding accomplishments. Deep feature extraction from a CNN model is a frequently utilized technique in these approaches. Although CNN-based techniques have achieved considerable success, there is indeed ample space for improvement in terms of their classification accuracies. Certainly, fusion with other features has the potential to extensively improve the performance of distant imaging scene classification. This paper, thus, offers an effective hybrid model that is based on the concept of feature-level fusion. We use the fuzzy C-means segmentation technique to appropriately classify various objects in the remote sensing images. The segmented regions of the image are then labeled using a Markov random field (MRF). After the segmentation and labeling of the objects, classical and CNN features are extracted and combined to classify the objects. After categorizing the objects, object-to-object relations are studied. Finally, these objects are transmitted to a fully convolutional network (FCN) for scene classification along with their relationship triplets. The experimental evaluation of three publicly available standard datasets reveals the phenomenal performance of the proposed system.

Multi-Output Network Combining GNN and CNN for Remote Sensing Scene Classification

Scene classification is an active research area in the remote sensing (RS) domain. Some categories of RS scenes, such as medium residential and dense residential scenes, would contain the same type of geographical objects but have various spatial distributions among these objects. The adjacency and disjointness relationships among geographical objects are normally neglected by existing RS scene classification methods using convolutional neural networks (CNNs). In this study, a multi-output network (MopNet) combining a graph neural network (GNN) and a CNN is proposed for RS scene classification with a joint loss. In a candidate RS image for scene classification, superpixel regions are constructed through image segmentation and are represented as graph nodes, while graph edges between nodes are created according to the spatial adjacency among corresponding superpixel regions. A training strategy of a jointly learning CNN and GNN is adopted in the MopNet. Through the message propagation mechanism of MopNet, spatial and topological relationships imbedded in the edges of graphs are employed. The parameters of the CNN and GNN in MopNet are updated simultaneously with the guidance of a joint loss via the backpropagation mechanism. Experimental results on the OPTIMAL-31 and aerial image dataset (AID) datasets show that the proposed MopNet combining a graph convolutional network (GCN) or graph attention network (GAT) and ResNet50 achieves state-of-the-art accuracy. The overall accuracy obtained on OPTIMAL-31 is 96.06% and those on AID are 95.53% and 97.11% under training ratios of 20% and 50%, respectively. Spatial and topological relationships imbedded in RS images are helpful for improving the performance of scene classification.

Graph-Based Embedding Smoothing Network for Few-Shot Scene Classification of Remote Sensing Images

As a fundamental task in the field of remote sensing, scene classification is increasingly attracting attention. The most popular way to solve scene classification is to train a deep neural network with a large-scale remote sensing dataset. However, given a small amount of data, how to train a deep neural network with outstanding performance remains a challenge. Existing methods seek to take advantage of transfer knowledge or meta-knowledge to resolve the scene classification issue of remote sensing images with a handful of labeled samples while ignoring various class-irrelevant noises existing in scene features and the specificity of different tasks. For this reason, in this paper, an end-to-end graph neural network is presented to enhance the performance of scene classification in few-shot scenarios, referred to as the graph-based embedding smoothing network (GES-Net). Specifically, GES-Net adopts an unsupervised non-parametric regularizer, called embedding smoothing, to regularize embedding features. Embedding smoothing can capture high-order feature interactions in an unsupervised manner, which is adopted to remove undesired noises from embedding features and yields smoother embedding features. Moreover, instead of the traditional sample-level relation representation, GES-Net introduces a new task-level relation representation to construct the graph. The task-level relation representation can capture the relations between nodes from the perspective of the whole task rather than only between samples, which can highlight subtle differences between nodes and enhance the discrimination of the relations between nodes. Experimental results on three public remote sensing datasets, UC Merced, WHU-RS19, and NWPU-RESISC45, showed that the proposed GES-Net approach obtained state-of-the-art results in the settings of limited labeled samples.

GSCCTL: a general semi-supervised scene classification method for remote sensing images based on clustering and transfer learning

ABSTRACT Recently, much research has shown that deep learning methods are superior in scene classification for remote sensing images (HSIs). However, the lack of labelled samples and computational resources present serious obstacles for HSI scene classification. To address these problems, we consider the pragmatic deep learning problem with only a few or no labelled samples. A general semi-supervised scene classification method based on clustering and transfer learning named GSCCTL is proposed to transfer outstanding supervised learning models to semi-supervised scene classification. Specifically, the GSCCTL model iterates between (1) scene clustering and (2) transfer learning to improve scene classification performance for the HSI data set. Since the clustering results may contain noise, we propose a new method to calculate the distance between the samples and the clustering center and add a reliable sample selection operation to the data after clustering. Initially, the model’s scene classification performance is weak, and transfer learning is carried out on a few samples close to the clustering center. In subsequent iterations, the model is constantly optimized, more and more samples are selected as reliable samples for training, and the model’s scene classification performance is strengthened. Through iterative clustering and transfer learning, the model is progressively optimized and gradually converged until the end condition is reached. Extensive experiments on the UCMerced, AID, and NWPU-RESISC45 remote sensing data sets demonstrate that the GSCCTL model based on clustering and transfer learning improved scene classification accuracy with only a few labelled samples and is suitable for semi-supervised scene classification.

Multispectral Scene Classification via Cross-Modal Knowledge Distillation

Scene classification is a fundamental task for numeral remote sensing applications, which aims to assign semantic labels to image patches. Although deep neural networks (DNN) demonstrated unique strength in scene classification, their performances are still limited due to the lack of training samples in the remote sensing field. Recent studies show that the performance of scene classification can be improved by taking advantage of the knowledge transferred from models pre-trained on RGB images. However, the modalities differences between input images hinder the knowledge transfer across models, especially when the input of the models has distinct spectral bands. To tackle the challenges, we propose a cross-modal knowledge distillation framework to improve the performance of multispectral scene classification by transferring the prior knowledge from teacher models pre-trained on RGB images to the student network with limited samples. Moreover, a teacher assistant (TA) network is introduced to further improve the classification performance by bridging the gap between the teacher and student networks. The proposed strategy is evaluated on models with multimodality inputs with distinct spectral bands and demonstrates superior performance as compared to the state-of-the-art methods.

Semi-Supervised Remote-Sensing Image Scene Classification Using Representation Consistency Siamese Network

Deep learning has achieved excellent performance in remote-sensing image scene classification, since a large number of datasets with annotations can be applied for training. However, in actual applications, there is just a few annotated samples and a large number of unannotated samples in remote-sensing images, which leads to overfitting of the deep model and affects the performance of scene classification. In order to address these problems, a semi-supervised representation consistency Siamese network (SS-RCSN) is proposed for remote-sensing image scene classification. First, considering intraclass diversity and interclass similarity of remote-sensing images, Involution-generative adversarial network (GAN) is utilized to extract the discriminative features from remote-sensing images via unsupervised learning. Then, Siamese network with a representation consistency loss is proposed for semi-supervised classification, which aims to reduce the differences of labeled and unlabeled data. Experimental results on UC Merced dataset, RESICS-45 dataset, aerial image dataset (AID), and RS dataset demonstrate that our method yields superior classification performance compared with other semi-supervised learning (SSL) methods.

Multilayer Feature Fusion Network With Spatial Attention and Gated Mechanism for Remote Sensing Scene Classification

Remote sensing (RS) scene classification has attracted extensive attention due to its large number of applications. Recently, convolutional neural networks (CNNs) methods have shown impressive ability of feature learning in RS scene classification. However, the performance is still limited by large-scale variance and complex background. To address these problems, we present a multilayer feature fusion network with spatial attention and gated mechanism (MLF2Net_SAGM) for RS scene classification. At first, the backbone is employed to extract multilayer convolutional features. Then, a residual spatial attention module (RSAM) is proposed to enhance discriminative regions of the multilayer feature maps, and key areas can be harvested. Finally, the multilayer spatial calibration features are fused to form the final feature map, and a gated fusion module (GFM) is designed to eliminate feature redundancy and mutual exclusion (FRME). To verify the effectiveness of the proposed method, we conduct comparative experiments based on three widely used RS image scene classification benchmarks. The results show that the direct fusion of multilayer features via element-wise addition leads to FRME, whereas our method fuses multilayer features more effectively and improves the performance of scene classification.

DeepScene: Scene classification via convolutional neural network with spatial pyramid pooling

Dissimilar to object classification, scene classification needs to consider not only the components that exist in the image but also their corresponding distribution. The greatest challenge of scene classification, especially indoor scene classification, is that many classes share the same representative components whereas the degree of similarity can be low within the same class. Some images have no clear indication that they belong to a particular class. In view of this, we propose a DeepScene model that leverages Convolutional Neural Network as the base architecture. As color cues are important for scene classification, two solutions are proposed to convert grayscale scene images to RGB images, which are replication and deep neural network based style transfer for colorization. To address the challenge of objects with varying sizes and positions in the scene, Spatial Pyramid Pooling is incorporated into the Convolutional Neural Network. The Spatial Pyramid Pooling performs multi-level pooling to enable the multi-size training of the model for improved scale and translational invariance. Ensemble learning is then adopted to boost the overall performance in scene classification. The proposed DeepScene model outshines the state-of-art methods with accuracy of 98.1% on Event-8, 95.6% on Scene-15 and 71.0% on MIT-67.

Integrating Image Quality Enhancement Methods and Deep Learning Techniques for Remote Sensing Scene Classification

Through the continued development of technology, applying deep learning to remote sensing scene classification tasks is quite mature. The keys to effective deep learning model training are model architecture, training strategies, and image quality. From previous studies of the author using explainable artificial intelligence (XAI), image cases that have been incorrectly classified can be improved when the model has adequate capacity to correct the classification after manual image quality correction; however, the manual image quality correction process takes a significant amount of time. Therefore, this research integrates technologies such as noise reduction, sharpening, partial color area equalization, and color channel adjustment to evaluate a set of automated strategies for enhancing image quality. These methods can enhance details, light and shadow, color, and other image features, which are beneficial for extracting image features from the deep learning model to further improve the classification efficiency. In this study, we demonstrate that the proposed image quality enhancement strategy and deep learning techniques can effectively improve the scene classification performance of remote sensing images and outperform previous state-of-the-art approaches.

An Attention-Guided Multilayer Feature Aggregation Network for Remote Sensing Image Scene Classification

Remote sensing image scene classification (RSISC) has broad application prospects, but related challenges still exist and urgently need to be addressed. One of the most important challenges is how to learn a strong discriminative scene representation. Recently, convolutional neural networks (CNNs) have shown great potential in RSISC due to their powerful feature learning ability; however, their performance may be restricted by the complexity of remote sensing images, such as spatial layout, varying scales, complex backgrounds, category diversity, etc. In this paper, we propose an attention-guided multilayer feature aggregation network (AGMFA-Net) that attempts to improve the scene classification performance by effectively aggregating features from different layers. Specifically, to reduce the discrepancies between different layers, we employed the channel–spatial attention on multiple high-level convolutional feature maps to capture more accurately semantic regions that correspond to the content of the given scene. Then, we utilized the learned semantic regions as guidance to aggregate the valuable information from multilayer convolutional features, so as to achieve stronger scene features for classification. Experimental results on three remote sensing scene datasets indicated that our approach achieved competitive classification performance in comparison to the baselines and other state-of-the-art methods.

An adaptive multilayer feature fusion strategy for remote sensing scene classification

ABSTRACT Remote sensing scene classification (RSSC) is one of the fundamental and challenging tasks for remote-sensing understanding and interpretation. How to construct discriminative features is crucial for scene classification. It is generally acknowledged that integrating convolutional features from different layers can significantly improve scene classification performance. However, some existing methods directly concatenate the high-layer and low-layer features without considering feature redundancy, semantic ambiguity and background noise, resulting in sub-optimal performance. In this paper, inspired by the attention mechanism, we propose a simple but effective strategy to fuse different convolutional features after feature selection operation, instead of directing concatenation. The basic idea of the feature selection operation is that we need to fuse those low-layer features, which are consistent with high-layer semantics. In particular, the proposed strategy is flexible and can be embedded into many neural architectures. Experimental results on two benchmark datasets demonstrate that the proposed method can obtain more valuable features and achieve competitive performance than other scene classification approaches.

A Dual-Model Architecture with Grouping-Attention-Fusion for Remote Sensing Scene Classification

Remote sensing images contain complex backgrounds and multi-scale objects, which pose a challenging task for scene classification. The performance is highly dependent on the capacity of the scene representation as well as the discriminability of the classifier. Although multiple models possess better properties than a single model on these aspects, the fusion strategy for these models is a key component to maximize the final accuracy. In this paper, we construct a novel dual-model architecture with a grouping-attention-fusion strategy to improve the performance of scene classification. Specifically, the model employs two different convolutional neural networks (CNNs) for feature extraction, where the grouping-attention-fusion strategy is used to fuse the features of the CNNs in a fine and multi-scale manner. In this way, the resultant feature representation of the scene is enhanced. Moreover, to address the issue of similar appearances between different scenes, we develop a loss function which encourages small intra-class diversities and large inter-class distances. Extensive experiments are conducted on four scene classification datasets include the UCM land-use dataset, the WHU-RS19 dataset, the AID dataset, and the OPTIMAL-31 dataset. The experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-arts.

SEMSDNet: A Multiscale Dense Network With Attention for Remote Sensing Scene Classification

Remote sensing image scene classification plays an important role in remote sensing image interpretation. Deep learning brings prosperity to the research in this field, and numerous deep learning models are proposed in order to improve the performance of scene classification. However, images of different remote sensing scenes vary a lot, showing similar or diverse textures and simple or complex contents. Using a fixed convolutional neural network framework to classify scene images is performance-limited and not practice-flexible. To address this issue, in this article, we propose the SEMSDNet (multiscale dense networks with squeeze and excitation attention). The framework multiscale dense convolutional network (MSDNet) with multiple classifiers and dense connections can automatically transform between a small network and a deep network according to the complexity of test samples and the limitation of computational resources. Moreover, in order to extract more effective features, the squeeze-and-excitation (SE) attention mechanism is introduced into the framework to process the features of various scenes self-adaptively. In addition, considering the limited computing resources, we impose two settings with computational constraints at the test time: budgeted batch classification, which is a fixed computational budget setting for sample classification, and anytime prediction, which forces the network to output a prediction at any given point-in-time. Experimental results on several public datasets show that the proposed SEMSDNet method is superior to the state-of-the-art methods on both performance and efficiency. Experiments also reveal its capability to treat samples of different classification difficulties with uneven resource allocation and flexible network architecture, showing its potentials in practical applications.