Performance Of Scene Classification Research Articles

CFCANet: A Complete Frequency Channel Attention Network for SAR Image Scene Classification

An important means of remote sensing imagery interpretation, remote sensing scene classification technology has recently achieved great success, especially based on deep learning. However, most of these methods are designed for noise-free images. The scene classification performance for noisy remote sensing images (RS), i.e., SAR images with speckle noise, is poor due to the sufficient effect of noise. An intuitive solution is denoising firstly and then classifying the image, which makes the whole process cumbersome. To address this problem, we design a new complete frequency channel attention network (CFCANet) that can handle noisy RS images directly without any filtering operation. CFCANet selects part of the low-frequency information to interact with the feature map adequately. For the original feature map, a corresponding 2D DCT frequency component is assigned, from which the most significant eigenvalue of each channel is obtained by maximization. Compared with the frequency channel attention network (FcaNet), the proposed network has better anti-noise ability as it exploits low frequency information of the images. The effectiveness of our method has been proved by experiments based on public datasets and some simulated datasets. Moreover, we build a new SAR scene classification dataset: WHU-SAR6. The comprehensive evaluation shows that the proposed method consistently outperforms several advanced methods, including ResNet, SENet, CBAM, EcaNet, and FcaNet.

MSNet: A Multiple Supervision Network for Remote Sensing Scene Classification

Remote sensing scene classification is a complex task due to large intraclass variations in object appearances with a small number of samples per class and high interclass similarities due to shared objects in different classes, which usually cause model overfitting and high interclass confusion. To address these challenges, a multiple supervision approach, called multiple supervision network (MSNet), consisting of the ResNet-50 backbone, a feature discriminative branch (FDB), and a feature confusion branch (FCB) is proposed in this letter. The FDB selects discriminative features per class and suppresses peaks in feature maps to examine more informative regions with lower feature magnitudes. Meanwhile, the FCB reduces overfitting by introducing confusion to the input of a fully connected layer which also enhances the robust features. The FDB and FCB are only used in training of the backbone and not used in inference. Thus, the proposed method does not introduce additional computing time on the backbone while it significantly boosts its performance in scene classification. The experimental results show that MSNet outperforms the methods considered in this letter.

AMN: Attention Metric Network for One-Shot Remote Sensing Image Scene Classification

In recent years, deep neural network (DNN) based scene classification methods have achieved promising performance. However, the data-driven training strategy requires a large number of labeled samples, making the DNN-based methods unable to solve the scene classification problem in the case of a small number of labeled images. As the number and variety of scene images continue to grow, the cost and difficulty of manual annotation also increase. Therefore, it is significant to deal with the scene classification problem with only a few labeled samples. In this paper, we propose an attention metric network (AMN) in the framework of the few-shot learning (FSL) to improve the performance of one-shot scene classification. AMN is composed of a self-attention embedding network (SAEN) and a cross-attention metric network (CAMN). In SAEN, we adopt the spatial attention and the channel attention of feature maps to obtain abundant features of scene images. In CAMN, we propose a novel cross-attention mechanism which can highlight the features that are more concerned about different categories, and improve the similarity measurement performance. A loss function combining mean square error (MSE) loss with multi-class N-pair loss is developed, which helps to promote the intra-class similarity and inter-class variance of embedding features, and also improve the similarity measurement results. Experiments on the NWPU-RESISC45 dataset and the RSD-WHU46 dataset demonstrate that our method achieves the state-of-the-art results on one-shot remote sensing image scene classification tasks.

High-Resolution Remote Sensing Image Scene Classification via Key Filter Bank Based on Convolutional Neural Network

High-resolution remote sensing (HRRS) image scene classification has attracted an enormous amount of attention due to its wide application in a range of tasks. Due to the rapid development of deep learning (DL), models based on convolutional neural network (CNN) have made competitive achievements on HRRS image scene classification because of the excellent representation capacity of DL. The scene labels of HRRS images extremely depend on the combination of global information and information from key regions or locations. However, most existing models based on CNN tend only to represent the global features of images or overstate local information capturing from key regions or locations, which may confuse different categories. To address this issue, a key region or location capturing method called key filter bank (KFB) is proposed in this article, and KFB can retain global information at the same time. This method can combine with different CNN models to improve the performance of HRRS imagery scene classification. Moreover, for the convenience of practical tasks, an end-to-end model called KFBNet where KFB combined with DenseNet-121 is proposed to compare the performance with existing models. This model is evaluated on public benchmark data sets, and the proposed model makes better performance on benchmarks than the state-of-the-art methods.

Deep Discriminative Representation Learning with Attention Map for Scene Classification

In recent years, convolutional neural networks (CNNs) have shown great success in the scene classification of computer vision images. Although these CNNs can achieve excellent classification accuracy, the discriminative ability of feature representations extracted from CNNs is still limited in distinguishing more complex remote sensing images. Therefore, we propose a unified feature fusion framework based on attention mechanism in this paper, which is called Deep Discriminative Representation Learning with Attention Map (DDRL-AM). Firstly, by applying Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm, attention maps associated with the predicted results are generated in order to make CNNs focus on the most salient parts of the image. Secondly, a spatial feature transformer (SFT) is designed to extract discriminative features from attention maps. Then an innovative two-channel CNN architecture is proposed by the fusion of features extracted from attention maps and the RGB (red green blue) stream. A new objective function that considers both center and cross-entropy loss are optimized to decrease the influence of inter-class dispersion and within-class variance. In order to show its effectiveness in classifying remote sensing images, the proposed DDRL-AM method is evaluated on four public benchmark datasets. The experimental results demonstrate the competitive scene classification performance of the DDRL-AM approach. Moreover, the visualization of features extracted by the proposed DDRL-AM method can prove that the discriminative ability of features has been increased.

Multi-scale semantic feature fusion and data augmentation for acoustic scene classification

This paper investigates a multi-scale semantic feature fusion and data augmentation approach for deep convolutional neural network (CNN) based acoustic scene classification. To ensemble the multi-scale semantic information of CNN and improve the performance of acoustic scene classification, a multi-scale feature fusion framework, which consists of a simplified Xception backbone and a semantic feature fusion strategy, is presented. A novel label smoothing mixup data augmentation method, which is a generalization of mixup and label smoothing, is proposed to alleviate the over-confident problem of network training. A spatial-mixup technique is presented to generate meaningful mixup virtual data for acoustic scene classification. Extensive experiments on synthetic data and real acoustic scene classification dataset demonstrate that both multi-scale semantic feature fusion and label smoothing spatial-mixup data augmentation are effective for improving the acoustic scene classification performance of a deep neural network.

An Augmentation Attention Mechanism for High-Spatial-Resolution Remote Sensing Image Scene Classification

High-spatial-resolution remote sensing (HRRS) image scene classification, which categorizes HRRS images into an independent set of semantic-level land use and land cover classes based on image contents, has attracted much attention, and many methods have been proposed due to its wide application in earth observation tasks. In fact, categories of HRRS images depend on regions containing class-specific ground objects, while most of the existing methods for HRRS image scene classification only focus on global information, which introduces redundant information and results in the poor performance of HRRS image scene classification. To overcome the shortcomings of the existing methods, an attention mechanism-based convolutional neural network with multiaugmented schemes is proposed in this article. In the proposed method, augmentation operations over attention mechanism feature maps are used to force the model to capture class-specific features and eliminate redundant information and push the model to capture discriminative regions as much as possible, instead of using all global information without favor. Moreover, a bilinear pooling is utilized to expand the interclass discrimination. Still, feature center loss motivated by center loss is applied to narrow the intraclass gap. To verify the effectiveness of the proposed end-to-end model, three benchmarks are used for testing, and the experimental results have proven the superiority of the proposed method, compared with current state-of-the-art end-to-end methods for HRRS image scene classification.

Semisupervised Center Loss for Remote Sensing Image Scene Classification

High-resolution remote sensing image scene classification is a scene-level classification task. Driven by a wide range of applications, accurate scene annotation has become a hot and challenging research topic. In recent years, convolutional neural networks (ConvNets) have achieved promising performance among a variety of supervised classification methods. However, due to the lack of clearly labeled remote sensing images, it may be difficult to further improve the performance of scene classification. To address this issue, we propose a novel semisupervised center loss for scene classification. The main innovation of our method is to develop a cooperative framework of supervised and unsupervised branches in an end-to-end way. Specifically, we consider the class centers as guiding factors between the supervised and unsupervised branches. The supervised branch relies on a small number of labeled samples to generate class centers, which serve as initialization centers for the unsupervised branch. Meanwhile, the unsupervised branch utilizes the easily available remote sensing images to correct the class centers for enhancing the discriminative power of supervised ConvNets. Experimental results on three public benchmarks have indicated that the proposed method is superior to supervised center loss based methods.

Remote Sensing Scene Classification Using Heterogeneous Feature Extraction and Multi-Level Fusion

Understanding the scenes provided by remote sensing (RS) images has drawn increasing attention in the last decade. It is to automatically classify a RS scene image by feature extraction and label assignment. Although much effort has been dedicated to developing discriminative feature extraction as well as automatic classification techniques, it is still very challenging owing to the complex distributions of the ground objects in high spatial resolution scenes. To enhance the ability to represent the RS scenes, an integration of multiple types of features for remote sensing scenes is considered as an effective way. Nevertheless, different kinds of features possess different characteristics, and how to fuse them together is a critical problem. In this paper, to fuse three different but complementary types of features that are extracted to characterize global attributes of scenes together, a discriminant correlation analysis based feature-level fusion approach is proposed, which maximizes the correlation of corresponding features across the two feature sets and in addition de-correlates features that belong to different classes within each feature set. In addition, to further improve the scene classification performance, another kind of information fusion form, called decision-level fusion is adopted in the classification stage. In our proposed decision-level fusion technique, the final results of multiple classifiers are combined via majority voting. Extensive experiments results conducted on the well-known SIRI-WHU dataset, the WHU-RS dataset and the UC Merced Land Use dataset have demonstrated that the proposed remote sensing scene classification method based on heterogeneous feature extraction and multi-level fusion is superior to many state-of-the-art scene classification algorithms.

Synthetic Aperture Radar Scene Classification Using Multiview Cross Correlation Attention Network

The second-order pooling manner, exploring higher feature statistics than the first-order pooling, has achieved impressive performance in scene classification. However, the object not only presents similarity but also exhibits diversified singularity on the synthetic aperture radar (SAR) image. These make the second-order pooling approaches to explore the single-view second-order feature statistics less adaptable for SAR scene classification. To solve this issue, an end-to-end training framework based on the multiview cross correlation attention network (MCAN) is proposed. The spatial and channelwise self-attention modules are first employed to model the interdependences between the spatial and channel dimensions of their convolutional features. Subsequently, the global spatial and channelwise covariance pooling layers are drawn into the MCAN, and then, they learn the spatial and channel cross correlations within the feature statistics, respectively. Finally, an iterative matrix square-root normalization layer, which owns the capability to fast computing approximate square root of the covariance matrix, is introduced for making the feature representation more discriminative. Experiments on the SAR data set from the TerraSAR-X images for scene classification demonstrate that the MCAN performs better than other related works.

RADC-Net: A residual attention based convolution network for aerial scene classification

With rapid development of satellite and airplane platforms, aerial image has become more and more accessible. Aerial image scene classification plays an important role in many remote sensing and geo-science applications. Deep learning methods, especially convolutional neural network (CNN), have boosted the performance of aerial image scene classification significantly because of the strong feature representation ability. However, the distribution of geo-spatial objects and the spatial arrangement of aerial image scenes is often more complicated than natural image scenes. While current CNNs usually highlight the global semantics, more local semantics and more discriminative features need to be preserved to deal with the aforementioned challenges in aerial scenes. In this paper, we propose a residual attention based dense connected convolutional neural network (RADC-Net) to tackle this problem. This framework contains three dense block each followed by an attention block and an enhanced classification layer. Firstly, we simplify the current dense connection structure so that our dense block has much fewer parameters while maintaining discriminative convolutional feature representation ability. Then, we propose a novel residual attention block for our framework to highlight the local semantics relevant to the aerial scenes. Finally, we introduce an enhanced classification layer in our framework to further refine the extracted convolutional features and highlight local semantic information. We comprehensively evaluate the performance of our proposed RADC-Net on three publicly available benchmark datasets. Experimental results demonstrate that our proposed RADC-Net outperforms some state-of-the-art methods with much fewer parameters.

RETRACTED: Attention-Based Deep Feature Fusion for the Scene Classification of High-Resolution Remote Sensing Images

Scene classification of high-resolution remote sensing images (HRRSI) is one of the most important means of land-cover classification. Deep learning techniques, especially the convolutional neural network (CNN) have been widely applied to the scene classification of HRRSI due to the advancement of graphic processing units (GPU). However, they tend to extract features from the whole images rather than discriminative regions. The visual attention mechanism can force the CNN to focus on discriminative regions, but it may suffer from the influence of intra-class diversity and repeated texture. Motivated by these problems, we propose an attention-based deep feature fusion (ADFF) framework that constitutes three parts, namely attention maps generated by Gradient-weighted Class Activation Mapping (Grad-CAM), a multiplicative fusion of deep features and the center-based cross-entropy loss function. First of all, we propose to make attention maps generated by Grad-CAM as an explicit input in order to force the network to concentrate on discriminative regions. Then, deep features derived from original images and attention maps are proposed to be fused by multiplicative fusion in order to consider both improved abilities to distinguish scenes of repeated texture and the salient regions. Finally, the center-based cross-entropy loss function that utilizes both the cross-entropy loss and center loss function is proposed to backpropagate fused features so as to reduce the effect of intra-class diversity on feature representations. The proposed ADFF architecture is tested on three benchmark datasets to show its performance in scene classification. The experiments confirm that the proposed method outperforms most competitive scene classification methods with an average overall accuracy of 94% under different training ratios.

Combing Triple-Part Features of Convolutional Neural Networks for Scene Classification in Remote Sensing

High spatial resolution remote sensing (HSRRS) images contain complex geometrical structures and spatial patterns, and thus HSRRS scene classification has become a significant challenge in the remote sensing community. In recent years, convolutional neural network (CNN)-based methods have attracted tremendous attention and obtained excellent performance in scene classification. However, traditional CNN-based methods focus on processing original red-green-blue (RGB) image-based features or CNN-based single-layer features to achieve the scene representation, and ignore that texture images or each layer of CNNs contain discriminating information. To address the above-mentioned drawbacks, a CaffeNet-based method termed CTFCNN is proposed to effectively explore the discriminating ability of a pre-trained CNN in this paper. At first, the pretrained CNN model is employed as a feature extractor to obtain convolutional features from multiple layers, fully connected (FC) features, and local binary pattern (LBP)-based FC features. Then, a new improved bag-of-view-word (iBoVW) coding method is developed to represent the discriminating information from each convolutional layer. Finally, weighted concatenation is employed to combine different features for classification. Experiments on the UC-Merced dataset and Aerial Image Dataset (AID) demonstrate that the proposed CTFCNN method performs significantly better than some state-of-the-art methods, and the overall accuracy can reach 98.44% and 94.91%, respectively. This indicates that the proposed framework can provide a discriminating description for HSRRS images.

Convolutional neural network based heterogeneous transfer learning for remote-sensing scene classification

ABSTRACTDeep convolutional neural network (CNN) transfer has recently shown strong performance in scene classification of high-resolution remote-sensing images. However, the majority of transfer learning solutions are categorized as homogeneous transfer learning, which ignores differences between target and source domains. In this paper, we propose a heterogeneous model to transfer CNNs to remote-sensing scene classification to correct input feature differences between target and source datasets. First, we extract filters from source images using the principal component analysis (PCA) method. Next, we convolute the target images with the extracted PCA filters to obtain an adopted target dataset. Then, a pretrained CNN is transferred to the adopted target dataset as a feature extractor. Finally, a classifier is used to accomplish remote-sensing scene classification. We conducted extensive experiments on the UC Merced dataset, the Brazilian coffee scene dataset and the Aerial Images Dataset to verify the effectiveness of the proposed heterogeneous model. The experimental results show that the proposed heterogeneous model outperforms the homogeneous model that uses pretrained CNNs as feature extractors by a wide margin and gains similar accuracies by fine-tuning a homogeneous transfer learning model with few training iterations.

Can additional spectral bands be estimated from aerial color images?

Inspired by the surprising performances of deep generative models, in this paper we present the preliminary results of an overly ambitious task: estimating computationally the additional spectral bands of a color aerial image. We have harnessed the expressive power of deep generative models to estimate the distribution of mostly infrared bands of aerial scenes, using only color RGB channels as input. Our approach has been tested from multiple aspects, including the reconstruction error of the additional bands and the effect of estimated bands on scene classification performance, as well as through the transfer potential of the trained network to a distinct dataset. To our surprise, the initial experiments have shown us that deep generative models can indeed learn to estimate additional bands up to a certain degree and can thus computationally reinforce datasets stemming from color-only sensors.

An Indoor Scene Classification Method for Service Robot Based on CNN Feature

Indoor scene classification plays a vital part in environment cognition of service robot. With the development of deep learning, fine-tuning CNN (Convolutional Neural Network) on target datasets has become a popular way to solve classification problems. However, this method cannot obtain satisfying indoor scene classification results because of overfitting when scene training datasets are insufficient. To solve this problem, an indoor scene classification method is proposed in this paper, which utilizes CNN feature of scene images to generate scene category features to classify scenes by a novel feature matching algorithm. The novel feature matching algorithm can further improve the speed of scene classification. In addition, overfitting is eliminated by our method even though the training data is limited. The presented method was evaluated on two benchmark scene datasets, Scene 15 dataset and MIT 67 dataset, acquiring 96.49% and 81.69% accuracy, respectively. The experiment results showed that our method was superior to other scene classification methods in terms of accuracy, speed, and robustness. To further evaluate our method, test experiments on unknown scene images from SUN 397 dataset had been done, and the models based on different training datasets obtained 94.34% and 79.80% test accuracy severally, which proved that the proposed method owned good performance in indoor scene classification.

Multiple resolution block feature for remote-sensing scene classification

ABSTRACT Great efforts have been devoted to improving the performance of scene classification. However, it is still a challenging task because of the complex background and diverse objects in scene images. To address this issue, multiple resolution block feature (MRBF) is proposed for remote-sensing scene classification. It is a unified and effective scene representation, consisting of completed double cross pattern (CDCP) combined with fisher vectors (FV). Specifically, in order to capture more robust and richer scene information, multiple resolution block descriptor is devised based on CDCP. After that, it is combined with FV to construct unified MRBF, which can fully exploit discriminative information from the block descriptor. Finally, the scene classification is achieved by kernel extreme learning machine. Extensive evaluations on four benchmark scene data-sets demonstrate the effectiveness and superiority of the proposed MRBF method for scene classification.

SiftingGAN: Generating and Sifting Labeled Samples to Improve the Remote Sensing Image Scene Classification Baseline In Vitro

Lack of annotated samples greatly restrains the direct application of deep learning in remote sensing image scene classification. Although researches have been done to tackle this issue by data augmentation with various image transformation operations, they are still limited in quantity and diversity. Recently, the advent of the unsupervised learning based generative adversarial networks (GANs) bring us a new way to generate augmented samples. However, such GAN-generated samples are currently only served for training GANs model itself and for improving the performance of the discriminator in GANs internally (in vivo). It becomes a question of serious doubt whether the GAN-generated samples can help better improve the scene classification performance of other deep learning networks (in vitro), compared with the widely used transformed samples. To answer this question, this paper proposes a SiftingGAN approach to generate more numerous, more diverse and more authentic labeled samples for data augmentation. SiftingGAN extends traditional GAN framework with an Online-Output method for sample generation, a Generative-Model-Sifting method for model sifting, and a Labeled-Sample-Discriminating method for sample sifting. Experiments on the well-known AID dataset demonstrate that the proposed SiftingGAN method can not only effectively improve the performance of the scene classification baseline that is achieved without data augmentation, but also significantly excels the comparison methods based on traditional geometric/radiometric transformation operations.

Aerial scene classification via an ensemble extreme learning machine classifier based on discriminative hybrid convolutional neural networks features

ABSTRACTIdentifying a discriminative feature can effectively improve the classification performance of aerial scene classification. Deep convolutional neural networks (DCNN) have been widely used in aerial scene classification for its learning discriminative feature ability. The DCNN feature can be more discriminative by optimizing the training loss function and using transfer learning methods. To enhance the discriminative power of a DCNN feature, the improved loss functions of pretraining models are combined with a softmax loss function and a centre loss function. To further improve performance, in this article, we propose hybrid DCNN features for aerial scene classification. First, we use DCNN models with joint loss functions and transfer learning from pretrained deep DCNN models. Second, the dense DCNN features are extracted, and the discriminative hybrid features are created using linear connection. Finally, an ensemble extreme learning machine (EELM) classifier is adopted for classification due to its general superiority and low computational cost. Experimental results based on the three public benchmark data sets demonstrate that the hybrid features obtained using the proposed approach and classified by the EELM classifier can result in remarkable performance.

Projection learning with local and global consistency constraints for scene classification

Besides the problem of high within-class variation and between-class ambiguity in high spatial resolution (HSR) remote sensing images, the dimension of data representation is very high, which poses a challenge for scene classification. To achieve high scene classification performance, it is important to uncover a discriminative subspace for data representation and scene classification. In this paper, we propose a projection learning framework with local and global consistency constraints for aerial scene classification. During the learning process, the within-class compactness and between-class separation of the data representation are enforced. To guarantee the subspace to be locally smooth, we obtain the local geometric structures of data including the similarity of features and geospatial adjacency of image patches. We utilize the global label consistency constraint to enforce the discrimination of the subspace. To make the projection optimal for classification, the projection learning with local and global constraints is integrated with the classification error to form a unified objective function. An efficient iteration algorithm is employed to solve the objective function. Experimental results demonstrate the superior performance of our method over state-of-the-art algorithms on aerial scene classification tasks.