Remote Sensing Image Classification Research Articles

A Comparative Study of the CNN Based Models Used for Remote Sensing Image Classification

Remotely sensed images, their classification and accuracy play a vital role in measuring a country’s scientific growth and technological development. Remote Sensing (RS) can be interpreted as a way of assessing the characteristics of a surface or an entity from a distance. This task of identifying and classifying datasets of RS images can be done using Convolutional Neural Network (CNN). For classifying images of large-scale areas, the traditional CNN approach produces coarse maps. For addressing this issue, Object based CNN method can be used. Classifying images with high spatial resolution can be done effectively using Object based image analysis. Deep learning methods offer the strength of auto learning the spatial features of an image. Object scale based adaptive CNN is a novel technique that can improve the accuracy of image classification of high spatial resolution images. For efficient RS image classification, a novel Deep learning approach called distributed CNN can be used which leads to enhanced accuracy of RS image classification. In this paper, three CNN models have been compared while considering the training time and efficiency to classify RS images as parameters of measure to assess the CNN models.

Change Detection Trend Prediction of Leisure Agriculture and Rural Tourism Based on High-Resolution Remote Sensing Image Classification Algorithm

Change Detection Trend Prediction of Leisure Agriculture and Rural Tourism Based on High-Resolution Remote Sensing Image Classification Algorithm

Remote Sensing Image Classification using Machine Learning Algorithms

Abstract: The classification of images can be influenced by various factors. When multiple sources of data are available for earth exploration, extracting valuable information or achieving effective outcomes can be both fascinating and challenging. Remote Sensing Images (RSI) classification plays a crucial role in remote sensing applications and serves as the foundation for efficient classification. Various classifiers are employed to classify different types of targets in remote sensing images. Achieving improved target classification is of utmost importance in both military and civilian domains. This paper focuses on summarizing advanced approaches used to enhance classification accuracy. The findings indicate that the Convolutional Neural Network surpasses all other conventional classification methods.

Deep Graph-Convolutional Generative Adversarial Network for Semi-Supervised Learning on Graphs

Graph convolutional networks (GCNs) are neural network frameworks for machine learning on graphs. They can simultaneously perform end-to-end learning on the attribute information and the structure information of graph data. However, most existing GCNs inevitably encounter the limitations of non-robustness and low classification accuracy when labeled nodes are scarce. To address the two issues, the deep graph convolutional generative adversarial network (DGCGAN), a model combining GCN and deep convolutional generative adversarial networks (DCGAN), is proposed in this paper. First, the graph data is mapped to a highly nonlinear space by using the topology and attribute information of the graph for symmetric normalized Laplacian transform. Then, through the feature-structured enhanced module, the node features are expanded into regular structured data, such as images and sequences, which are input to DGCGAN as positive samples, thus expanding the sample capacity. In addition, the feature-enhanced (FE) module is adopted to enhance the typicality and discriminability of node features, and to obtain richer and more representative features, which is helpful for facilitating accurate classification. Finally, additional constraints are added to the network model by introducing DCGAN, thus enhancing the robustness of the model. Through extensive empirical studies on several standard benchmarks, we find that DGCGAN outperforms state-of-the-art baselines on semi-supervised node classification and remote sensing image classification.

Completed homogeneous LBP for remote sensing image classification

ABSTRACT Land cover (LC) classification remains a challenging task due to the diversity of terrain and topography, limited prior knowledge, and complex data sets. These variations can lead to differences in illumination and shading, which can affect the appearance of objects in the image. The local binary pattern (LBP) model is an effective technique for capturing local texture information in an image, which can help to overcome the effects of topography diversity by analysing patterns in different image regions, even if the illumination or shading conditions are different. However, LBP alone is inadequate for characterizing high-resolution remotely sensed images with complex semantic content as it only utilizes sign information in the local region. In this paper, a new texture characterization descriptor, known as completed homogeneous LBP (CHLBP), is proposed as an improved version of homogeneous LBP (HLBP) for LC classification of remotely sensed images. The CHLBP method mainly involves the following steps: first, sign and magnitude information from the HLBP descriptor is extracted, providing an effective alternative to the HLBP complementary contrast measure. Second, for each sign and magnitude function, a new splitting factor is used to obtain a depth relationship between the centre and its neighbouring pixels and to enhance noise robustness. Finally, the centre pixels representing the image grey level are also considered to contain discriminative information. The performance of our descriptor is evaluated using four challenging texture databases: Outex (TC10, TC12), Geofen Image Dataset (GID), and large-scale aerial (AID). Extensive experiments were performed using four classifiers: Support Vector Machine (SVM), K-Nearest Neighbours (KNN), Random Forest (RF), and Multi-Layer Perceptron (MLP), demonstrating the effectiveness and robustness of our descriptor against noise and free noise conditions in public remote sensing datasets.

Neural Network Compression via Low Frequency Preference

Network pruning has been widely used in model compression techniques, and offers a promising prospect for deploying models on devices with limited resources. Nevertheless, existing pruning methods merely consider the importance of feature maps and filters in the spatial domain. In this paper, we re-consider the model characteristics and propose a novel filter pruning method that corresponds to the human visual system, termed Low Frequency Preference (LFP), in the frequency domain. It is essentially an indicator that determines the importance of a filter based on the relative low-frequency components across channels, which can be intuitively understood as a measurement of the “low-frequency components”. When the feature map of a filter has more low-frequency components than the other feature maps, it is considered more crucial and should be preserved during the pruning process. We conduct the proposed LFP on three different scales of datasets through several models and achieve superior performances. The experimental results obtained on the CIFAR datasets and ImageNet dataset demonstrate that our method significantly reduces the model size and FLOPs. The results on the UC Merced dataset show that our approach is also significant for remote sensing image classification.

Three-dimensional Softmax Mechanism Guided Bidirectional GRU Networks for Hyperspectral Remote Sensing Image Classification

Hyperspectral data is a valuable source of both spectral and spatial information. However, to enhance the classification accuracy of hyperspectral image features, it is crucial to capture the spatial spectral features of image elements. The recent years have witnessed the potentials of deep learning methods have shown great promise in the hyperspectral image classification due to their ability to model complex structures and extract multiple features in an end-to-end fashion. Since hyperspectral images can be viewed as sequential data, we propose a novel three-dimensional Softmax mechanism-guided bidirectional GRU network (TDS-BiGRU) for HSI classification. By utilizing a bidirectional GRU to process the sequence data, our method can significantly reduce the processing time. Furthermore, the proposed three-dimensional Softmax mechanism leverages three branches to capture cross-latitude interactions and calculate Softmax weights, which enables us to obtain deeper features with greater discriminative power. The experimental results demonstrate that the proposed method outperforms several prevalent algorithms on four hyperspectral remote sensing datasets. Additionally, we conduct thorough comparisons and ablation tests, which further confirm the effectiveness of our approach.

Semi-Supervised Tree Species Classification for Multi-Source Remote Sensing Images Based on a Graph Convolutional Neural Network

As a current research hotspot, graph convolution networks (GCNs) have provided new opportunities for tree species classification in multi-source remote sensing images. To solve the challenge of limited label information, a new tree species classification model was proposed by using the semi-supervised graph convolution fusion method for hyperspectral images (HSIs) and multispectral images (MSIs). In the model, the graph-based attribute features and pixel-based features are fused to deepen the correlation of multi-source images to improve accuracy. Firstly, the model employs the canonical correlation analysis (CCA) method to maximize the correlation of multi-source images, which explores the relationship between information from various sources further and offers more valuable insights. Secondly, convolution calculations were made to extract features and then map graph node fusion, which not only reduces redundancy features but also enhances compelling features. Finally, the relationship between representative descriptors is captured through the use of hyperedge convolution in the training process, and the dominant features on the graph are fully mined. The tree species are classified through two fusion feature operations, leading to improved classification performance compared to state-of-the-art methods. The fusion strategy can produce a complete classification map of the study areas.

Real-time scene classification of unmanned aerial vehicles remote sensing image based on Modified GhostNet.

Unmanned Aerial Vehicles (UAVs) play an important role in remote sensing image classification because they are capable of autonomously monitoring specific areas and analyzing images. The embedded platform and deep learning are used to classify UAV images in real-time. However, given the limited memory and computational resources, deploying deep learning networks on embedded devices and real-time analysis of ground scenes still has challenges in actual applications. To balance computational cost and classification accuracy, a novel lightweight network based on the original GhostNet is presented. The computational cost of this network is reduced by changing the number of convolutional layers. Meanwhile, the fully connected layer at the end is replaced with the fully convolutional layer. To evaluate the performance of the Modified GhostNet in remote sensing scene classification, experiments are performed on three public datasets: UCMerced, AID, and NWPU-RESISC. Compared with the basic GhostNet, the Floating Point Operations (FLOPs) are reduced from 7.85 MFLOPs to 2.58 MFLOPs, the memory is reduced from 16.40 MB to 5.70 MB, and the predicted time is improved by 18.86%. Our modified GhostNet also increases the average accuracy (Acc) (4.70% in AID experiments, 3.39% in UCMerced experiments). These results indicate that our Modified GhostNet can improve the performance of lightweight networks for scene classification and effectively enable real-time monitoring of ground scenes.

Hyperspectral Remote Sensing Images Feature Extraction Based on Spectral Fractional Differentiation

To extract effective features for the terrain classification of hyperspectral remote-sensing images (HRSIs), a spectral fractional-differentiation (SFD) feature of HRSIs is presented, and a criterion for selecting the fractional-differentiation order is also proposed based on maximizing data separability. The minimum distance (MD) classifier, support vector machine (SVM) classifier, K-nearest neighbor (K-NN) classifier, and logistic regression (LR) classifier are used to verify the effectiveness of the proposed SFD feature, respectively. The obtained SFD feature is sent to the full connected network (FCN) and 1-dimensionality convolutional neural network (1DCNN) for deep-feature extraction and classification, and the SFD-Spa feature cube containing spatial information is sent to the 3-dimensionality convolutional neural network (3DCNN) for deep-feature extraction and classification. The SFD-Spa feature after performing the principal component analysis (PCA) on spectral pixels is directly connected with the first principal component of the original data and sent to 3DCNNPCA and hybrid spectral net (HybridSN) models to extract deep features. Experiments on four real HRSIs using four traditional classifiers and five network models have shown that the extracted SFD feature can effectively improve the accuracy of terrain classification, and sending SFD feature to deep-learning environments can further improve the accuracy of terrain classification for HRSIs, especially in the case of small-size training samples.

USING SNN NEURAL NETWORKS TRAINED WITH HIGH RESOLUTION DATA 2 AND APPLIED TO COPERNICUS SENTINEL-2 DATA

Abstract. Data from the Copernicus project are proving to be fundamental in so many areas of spatial governance, and in particular in the context of safeguarding the cultural and natural heritage. This is mainly due to their dissemination mode, which allows them to be used by a countless number of institutions and organizations. With regard to their use in order to create effective supervised-training AI (Artificial Intelligence) classifiers (useful for continuous land-use monitoring), their low resolution could be a problem in terms of accuracy or in any case during the training process. This process which could be very costly (large training sets and high-end hardware required). In this paper, a methodology is tested that creates an AI classifier from high-resolution training data, Remote Sensing Image Classification Benchmark (RSI-CB128) and using Self-Normalising Neural Networks (SNNs). The efficiency of the Neural Network is, however, measured with a test dataset constructed with Raster Sentinel-2. In other words, the aim is to assess the goodness of an AI classifier on Sentinel-2 images built from Rasters generated by high-resolution sensors. Therefore, there seem to be clear advantages over the classification methodologies in use today.

Analysis of the driving role and impact of road construction on carbon stock

Road construction will have a large impact on the ecosystem in the road area, and carbon stock, as an important indicator to measure the scale and quantity of primary productivity of the ecosystem, will also change, but the specific pattern is not clear. It is important to study the impact of road construction on carbon stock for regional ecosystem protection and sustainable economic and social development. Based on the InVEST model, this paper quantifies the spatial and temporal changes of carbon stocks in Jinhua, Zhejiang Province, from 2002 to 2017, using remote sensing image classification data land cover types as model driving data, geodetector, trend analysis, and buffer zone analysis methods, explores the driving effect of road construction on carbon stocks, and analyzes the spatial and temporal impacts of road construction on carbon stocks within the buffer zone. Results indicate that the total carbon stock in the Jinhua area showed a decreasing trend during the 16 years, decreasing by about 8.58 × 106 t. The spatial changes in the areas with higher carbon stocks were not significant. The explanatory power of road network density on carbon stock reaches 37%, and the anisotropic effect of road construction on carbon stock is strong and had a significant driving effect on carbon storage reduction. The new highway construction will accelerate the rate of carbon stock decline in the buffer zone, and the spatial situation is generally "the farther away from the highway, the higher the carbon stock."

Event-driven spiking neural network based on membrane potential modulation for remote sensing image classification

Spiking neural network (SNN) based on sparse triggering and event-driven is a hardware-friendly model. SNN can provide an ultra-low power alternative for the deep neural network (DNN) to process remote sensing images. Brain information processing depends on the action potential of neurons. Therefore, the biological rationality of the artificial neural network (ANN) has been questioned. SNN is a more suitable model for brain information processing mechanisms. At present, the SNN obtained by ANN conversion has achieved the best performance in the current image processing tasks. However, the method based on ANN to SNN will have performance loss in the conversion process. Herein, we proposed a spiking neuron threshold-following reset (TF-reset) method and a membrane potential modulation method to reduce the loss of network conversion. We theoretically analyzed the proposed TF-reset and deduced the relationship between spike firing rate and neuron activation. In the experiment, we used an improved VGG-15 architecture combined with the method of transfer learning to apply the model to the classification task of remote sensing images. SNN-VGG-15 based on TF-reset and membrane potential modulation algorithm achieved a classification accuracy of 99.14%, 94.54%, and 95.00% on UCM, RSSCN7, and AID. Our algorithm can not only realize the lossless conversion of SNN but also outperforms the original network in classification performance on UCM and RSSCN7. In addition, our model also has advantages in energy consumption and noise robustness. The algorithm in this paper can provide a reference for the research remote sensing images procession using SNN.

TPENAS: A Two-Phase Evolutionary Neural Architecture Search for Remote Sensing Image Classification

The application of deep learning in remote sensing image classification has been paid more and more attention by industry and academia. However, manually designed remote sensing image classification models based on convolutional neural networks usually require sophisticated expert knowledge. Moreover, it is notoriously difficult to design a model with both high classification accuracy and few parameters. Recently, neural architecture search (NAS) has emerged as an effective method that can greatly reduce the heavy burden of manually designing models. However, it remains a challenge to search for a classification model with high classification accuracy and few parameters in the huge search space. To tackle this challenge, we propose TPENAS, a two-phase evolutionary neural architecture search framework, which optimizes the model using computational intelligence techniques in two search phases. In the first search phase, TPENAS searches for the optimal depth of the model. In the second search phase, TPENAS searches for the structure of the model from the perspective of the whole model. Experiments on three open benchmark datasets demonstrate that our proposed TPENAS outperforms the state-of-the-art baselines in both classification accuracy and reducing parameters.

Remote Sensing Image Classification with Few Labeled Data Using Semisupervised Learning

Synthetic aperture radar (SAR) as an imaging radar is capable of high-resolution remote sensing, independent of flight altitude, and independent of weather. Traditional SAR ship image classification tends to extract features manually. It relies too much on expert experience and is sensitive to the scale of SAR images. Recently, with the development of deep learning, deep neural networks such as convolutional neural networks are widely used to complete feature extraction and classification tasks, which improves algorithm accuracy and normalization capabilities to a large extent. However, deep learning requires a large number of labeled samples, and the vast bulk of SAR images are unlabeled. Therefore, the classification accuracy of deep neural networks is limited. To tackle the problem, we propose a semisupervised learning-based SAR image classification method considering that only few labeled images are available. The proposed method can train the classification model using both labeled and unlabeled samples. Moreover, we improve the unsupervised data augmentation (UDA) strategy by designing a symmetric function for unsupervised loss calculation. Experiments are carried out on the OpenSARShip dataset, and results show that the proposed method reaches a much higher accuracy than the original UDA.

Self-supervision assisted multimodal remote sensing image classification with coupled self-looping convolution networks

Recently, remote sensing community has seen a surge in the use of multimodal data for different tasks such as land cover classification, change detection and many more. However, handling multimodal data requires synergistically using the information from different sources. Currently, deep learning (DL) techniques are being religiously used in multimodal data fusion owing to their superior feature extraction capabilities. But, DL techniques have their share of challenges. Firstly, DL models are mostly constructed in the forward fashion limiting their feature extraction capability. Secondly, multimodal learning is generally addressed in a supervised setting, which leads to high labelled data requirement. Thirdly, the models generally handle each modality separately, thus preventing any cross-modal interaction. Hence, we propose a novel self-supervision oriented method of multimodal remote sensing data fusion. For effective cross-modal learning, our model solves a self-supervised auxiliary task to reconstruct input features of one modality from the extracted features of another modality, thus enabling more representative pre-fusion features. To counter the forward architecture, our model is composed of convolutions both in backward and forward directions, thus creating self-looping connections, leading to a self-correcting framework. To facilitate cross-modal communication, we have incorporated coupling across modality-specific extractors using shared parameters. We evaluate our approach on three remote sensing datasets, namely Houston 2013 and Houston 2018, which are HSI-LiDAR datasets and TU Berlin, which is an HSI-SAR dataset, where we achieve the respective accuracy of 93.08%, 84.59% and 73.21%, thus beating the state of the art by a minimum of 3.02%, 2.23% and 2.84%.

Deep Learning Methods Used in Remote Sensing Images: A Review

Undeniably, Deep Learning (DL) has rapidly eroded traditional machine learning in Remote Sensing (RS) and geoscience domains with applications such as scene understanding, material identification, extreme weather detection, oil spill identification, among many others. Traditional machine learning algorithms are given less and less attention in the era of big data. Recently, a substantial amount of work aimed at developing image classification approaches based on the DL model's success in computer vision. The number of relevant articles has nearly doubled every year since 2015. Advances in remote sensing technology, as well as the rapidly expanding volume of publicly available satellite imagery on a worldwide scale, have opened up the possibilities for a wide range of modern applications. However, there are some challenges related to the availability of annotated data, the complex nature of data, and model parameterization, which strongly impact performance. In this article, a comprehensive review of the literature encompassing a broad spectrum of pioneer work in remote sensing image classification is presented including network architectures (vintage Convolutional Neural Network, CNN; Fully Convolutional Networks, FCN; encoder-decoder, recurrent networks; attention models, and generative adversarial models). The characteristics, capabilities, and limitations of current DL models were examined, and potential research directions were discussed.

P2FEViT: Plug-and-Play CNN Feature Embedded Hybrid Vision Transformer for Remote Sensing Image Classification

Remote sensing image classification (RSIC) is a classical and fundamental task in the intelligent interpretation of remote sensing imagery, which can provide unique labeling information for each acquired remote sensing image. Thanks to the potent global context information extraction ability of the multi-head self-attention (MSA) mechanism, visual transformer (ViT)-based architectures have shown excellent capability in natural scene image classification. However, in order to achieve powerful RSIC performance, it is insufficient to capture global spatial information alone. Specifically, for fine-grained target recognition tasks with high inter-class similarity, discriminative and effective local feature representations are key to correct classification. In addition, due to the lack of inductive biases, the powerful global spatial context representation capability of ViT requires lengthy training procedures and large-scale pre-training data volume. To solve the above problems, a hybrid architecture of convolution neural network (CNN) and ViT is proposed to improve the RSIC ability, called P2FEViT, which integrates plug-and-play CNN features with ViT. In this paper, the feature representation capabilities of CNN and ViT applying for RSIC are first analyzed. Second, aiming to integrate the advantages of CNN and ViT, a novel approach embedding CNN features into the ViT architecture is proposed, which can make the model synchronously capture and fuse global context and local multimodal information to further improve the classification capability of ViT. Third, based on the hybrid structure, only a simple cross-entropy loss is employed for model training. The model can also have rapid and comfortable convergence with relatively less training data than the original ViT. Finally, extensive experiments are conducted on the public and challenging remote sensing scene classification dataset of NWPU-RESISC45 (NWPU-R45) and the self-built fine-grained target classification dataset called BIT-AFGR50. The experimental results demonstrate that the proposed P2FEViT can effectively improve the feature description capability and obtain outstanding image classification performance, while significantly reducing the high dependence of ViT on large-scale pre-training data volume and accelerating the convergence speed. The code and self-built dataset will be released at our webpages.

A multimodal hyper-fusion transformer for remote sensing image classification

The multispectral (MS) and the panchromatic (PAN) images represent complementary and synergistic spatial spectral information, how to make optimal use of the advantages of them has become a hot research topic. This paper proposes a selectable Transformer and Gist CNN network (STGC-Net). It designs a subspace similar recombination module (SSR-Module) based on non-negative matrix factorization (NMF) and the self-attention mechanism for feature decomposition. This can alleviate the redundant information of multi-modal data and extract their own singular and common features. Considering that the MS and the PAN images exhibit different advantageous properties, a selectable self-attention spectral feature extraction module (S3FE-Module) and a multi-stream Gist spatial feature extraction module (MGSFE-Module) are proposed for the different singular features. The former can refine the Transformer’s input and simultaneously characterize the sequence information between channels for the MS image. The latter introduces the positional relationship between local features while extracting spatial features for the PAN image, thereby improving the accuracy of scene classification. Experimental results indicate that the proposed method performs better than the other methods. The relevant code of this paper is provided at: https://github.com/ru-willow/ST-GC-Net.

Visual explanations with detailed spatial information for remote sensing image classification via channel saliency

For the past few years, the barrier of explainability accompanying by deep neural networks (DNNs) has been increasingly studied. The methods based on class activation map (CAM) which interpret the model decision by mapping the output back to the input space, have achieved a notable momentum among the research. However, the CAM-based methods cannot stably produce effective explanation results on remote sensing images (RSIs), owing to the coarse location map generated by high-level features, whereas, the RSIs contain abundant detailed spatial information and multi-scale objects. To address this issue, this article proposes class activation map weighted with channel saliency and gradient (CSG-CAM) to enhance the low-level features in saliency map. To do this, we firstly introduce the idea of dynamic channel pruning and propose the channel saliency to describe the channel importance of specific layer. Then the channel saliency, instead of gradient, is considered as the neuron importance weights to calculate the saliency map on shallow layer. Furthermore, the channel saliency also participates in the neuron importance weights of final layer, jointly with a gradient weighted combination of the positive partial derivatives. Finally, the saliency map of proposed CSG-CAM is fused by the explanation heat maps from shallow and final layer of networks. The sufficient experimental results on two publicly available datasets designed for RSI scene classification and three practical networks demonstrate the effectiveness of the CSG-CAM in terms of both faithfulness and explainability.