Remote Sensing Image Classification Research Articles

Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification

Remote sensing (RS) image classification has attracted much attention recently and is widely used in various fields. Different to natural images, the RS image scenes consist of complex backgrounds and various stochastically arranged objects, thus making it difficult for networks to focus on the target objects in the scene. However, conventional classification methods do not have any special treatment for remote sensing images. In this paper, we propose a two-stream swin transformer network (TSTNet) to address these issues. TSTNet consists of two streams (i.e., original stream and edge stream) which use both the deep features of the original images and the ones from the edges to make predictions. The swin transformer is used as the backbone of each stream given its good performance. In addition, a differentiable edge Sobel operator module (DESOM) is included in the edge stream which can learn the parameters of Sobel operator adaptively and provide more robust edge information that can suppress background noise. Experimental results on three publicly available remote sensing datasets show that our TSTNet achieves superior performance over the state-of-the-art (SOTA) methods.

Dynamic Evolution Analysis of Desertification Images Based on BP Neural Network

In recent years, with the rise of artificial intelligence, deep neural network models have been used in various image recognition researches. Land desertification is a major environmental problem facing the world at present, and how to do a good job in dynamic monitoring is particularly important. For remote sensing images, this paper constructs a GA-PSO-BP analysis model based on BP neural network, genetic algorithm, and particle swarm algorithm and compares the classification training accuracies of the four models of BP, GA-BP, PSO-BP, and GA-PSO-BP; GA-PSO-BP was selected for dynamic analysis of desertification images, and the results showed the following: (1) By comparing the regional classification training accuracies of the four models of BP, GA-BP, PSO-BP, and GA-PSO-BP, the GA-PSO-BP neural network remote sensing image classification method proposed in this paper is simple and easy to operate. Compared with traditional remote sensing image classification methods and traditional neural network classification methods, the classification accuracy of remote sensing effects is improved. (2) Carrying out desertification analysis on remote sensing images of Horqin area, from 2010 to 2015, the desertified land area in the test area increased by 1.56 km2; from 2015 to 2020, the desertified land area in the test area decreased by 1.131 km2, and the desertified land in the test area from 2010 to 2020 showed a trend of increasing first and then decreasing, which is consistent with the actual situation. The GA-PSO-BP remote sensing image classification model has a good performance portability.

HybridGBN-SR: A Deep 3D/2D Genome Graph-Based Network for Hyperspectral Image Classification

The successful application of deep learning approaches in remote sensing image classification requires large hyperspectral image (HSI) datasets to learn discriminative spectral–spatial features simultaneously. To date, the HSI datasets available for image classification are relatively small to train deep learning methods. This study proposes a deep 3D/2D genome graph-based network (abbreviated as HybridGBN-SR) that is computationally efficient and not prone to overfitting even with extremely few training sample data. At the feature extraction level, the HybridGBN-SR utilizes the three-dimensional (3D) and two-dimensional (2D) Genoblocks trained using very few samples while improving HSI classification accuracy. The design of a Genoblock is based on a biological genome graph. From the experimental results, the study shows that our model achieves better classification accuracy than the compared state-of-the-art methods over the three publicly available HSI benchmarking datasets such as the Indian Pines (IP), the University of Pavia (UP), and the Salinas Scene (SA). For instance, using only 5% labeled data for training in IP, and 1% in UP and SA, the overall classification accuracy of the proposed HybridGBN-SR is 97.42%, 97.85%, and 99.34%, respectively, which is better than the compared state-of-the-art methods.

Remote Sensing Image Classification Using CNN-LSTM Model

The image classification of remote sensing (RS) plays a significant role in earth observation technology using RS data, extensively used in the military and civic sectors. However, the RS image classification confronts substantial scientific and practical difficulties because of RS data features, such as high dimensionality and relatively limited quantities of labeled examples accessible. In recent years, as new methods of deep learning (DL) have emerged, RS image classification approaches using DL have made significant advances, providing new possibilities for RS image classification research and development. Most of the researchers are using CNN to classify remote sensing images, but CNN alone problem with sequence data processing. But to get some sense out of the classification of remote sensing images. To avoid this in this paper, we use the CNN-LSTM model. The model performed ineffective classification of remote sensing images; the experimental results show that the proposed model is effective in classifying remote sensing images.

Graph-Based Embedding Smoothing Network for Few-Shot Scene Classification of Remote Sensing Images

As a fundamental task in the field of remote sensing, scene classification is increasingly attracting attention. The most popular way to solve scene classification is to train a deep neural network with a large-scale remote sensing dataset. However, given a small amount of data, how to train a deep neural network with outstanding performance remains a challenge. Existing methods seek to take advantage of transfer knowledge or meta-knowledge to resolve the scene classification issue of remote sensing images with a handful of labeled samples while ignoring various class-irrelevant noises existing in scene features and the specificity of different tasks. For this reason, in this paper, an end-to-end graph neural network is presented to enhance the performance of scene classification in few-shot scenarios, referred to as the graph-based embedding smoothing network (GES-Net). Specifically, GES-Net adopts an unsupervised non-parametric regularizer, called embedding smoothing, to regularize embedding features. Embedding smoothing can capture high-order feature interactions in an unsupervised manner, which is adopted to remove undesired noises from embedding features and yields smoother embedding features. Moreover, instead of the traditional sample-level relation representation, GES-Net introduces a new task-level relation representation to construct the graph. The task-level relation representation can capture the relations between nodes from the perspective of the whole task rather than only between samples, which can highlight subtle differences between nodes and enhance the discrimination of the relations between nodes. Experimental results on three public remote sensing datasets, UC Merced, WHU-RS19, and NWPU-RESISC45, showed that the proposed GES-Net approach obtained state-of-the-art results in the settings of limited labeled samples.

Meta captioning: A meta learning based remote sensing image captioning framework

Remote sensing image captioning models require large amounts of caption-labeled training data. Though image classification models are normally trained with sufficient training data, they cannot be straightforwardly applied to remote sensing image captioning, because the labels for classification and captioning arise from different task domains. Additionally, remote sensing images with caption labels are not as sufficient as images with class labels. Such limitations render difficulty to effective remote sensing image captioning. To address these limitations, we develop a meta captioning framework that conducts remote sensing image captioning with meta learning. The meta captioning framework extracts meta features from two support tasks, i.e., natural image classification and remote sensing image classification, and transfers the meta features to one target task, i.e., remote sensing image captioning. The two support tasks train classification models with big amounts of class-labeled data such that they extract meta features that comprehensively represent image visual features from the perspective of classification. The target task, equipped by the meta features, just requires a relatively small amount of caption-labeled training data for achieving effective remote sensing image captioning. Experimental evaluations on three public datasets validate that the meta captioning framework achieves state-of-the-art performance on remote sensing image captioning. We release the code for our work at: https://github.com/QiaoqiaoYang/MetaCaptioning.

DKDFN: Domain Knowledge-Guided deep collaborative fusion network for multimodal unitemporal remote sensing land cover classification

Land use and land cover maps provide fundamental information that has been used in different types of studies, ranging from public health to carbon cycling. However, the existing remote sensing image classification methods thus far suffer from the insufficient usage of multiple modalities, underconsideration of prior domain knowledge, and poor performance on minority classes. To alleviate these problems, we propose a novel domain knowledge-guided deep collaborative fusion network (DKDFN) with performance boosting for minority categories for land cover classification. More specifically, the DKDFN adopts a multihead encoder and a multibranch decoder structure. The architecture of the encoder probablizes sufficient mining of complementary information from multiple modalities, which are Sentinel-2, Sentinel-1, and SRTM Digital Elevation Data (SRTM) in our case. The multibranch decoder enables land cover classification in a multitask learning setup, performing semantic segmentation and reconstructing multimodal remote sensing indices, which are selected as representatives of domain knowledge. This design incorporates domain knowledge in an effective end-to-end manner. The training stage of our DKDFN is supervised by our proposed asymmetry loss function (ALF), which boosts performance on nearly all categories, especially the categories with a low frequency of occurrence. Ablation studies of the network suggest that our design logic is worth testing in any network with an encoder-decoder structure. The study is conducted in Hunan, China and is verified using a self-labeled multimodal unitemporal remote sensing image dataset. The comparative experiments between DKDFN and 6 state-of-the-art models (U-Net, SegNet, PSPNet, DeepLab, HRNet, MP-ResNet) testify to the superiority of our method and suggest its potential to be applied more widely to map land cover in other geographical areas given the availability of Sentinel-2, Sentinel-1, and SRTM data. The dataset can be downloaded by https://github.com/LauraChow/HunanMultimodalDataset.

Improved U-Net Remote Sensing Classification Algorithm Based on Multi-Feature Fusion Perception

The selection and representation of remote sensing image classification features play crucial roles in image classification accuracy. To effectively improve the classification accuracy of features, an improved U-Net network framework based on multi-feature fusion perception is proposed in this paper. This framework adds the channel attention module (CAM-UNet) to the original U-Net framework and cascades the shallow features with the deep semantic features, replaces the classification layer in the original U-Net network with a support vector machine, and finally uses the majority voting game theory algorithm to fuse the multifeature classification results and obtain the final classification results. This study used the forest distribution in Xingbin District, Laibin City, Guangxi Zhuang Autonomous Region as the research object, which is based on Landsat 8 multispectral remote sensing images, and, by combining spectral features, spatial features, and advanced semantic features, overcame the influence of the reduction in spatial resolution that occurs with the deepening of the network on the classification results. The experimental results showed that the improved algorithm can improve classification accuracy. Before the improvement, the overall segmentation accuracy and segmentation accuracy of the forestland increased from 90.50% to 92.82% and from 95.66% to 97.16%, respectively. The forest cover results obtained by the algorithm proposed in this paper can be used as input data for regional ecological models, which is conducive to the development of accurate and real-time vegetation growth change models.

Damage characteristics of the Qinghai-Tibet Highway in permafrost regions based on UAV imagery

ABSTRACT For the particular engineering-geological conditions and natural environment in permafrost regions, the highway has a high damage ratio during the operation. In warm and ice-rich permafrost regions, highway pavement and embankment have poor serviceability, which threatens driving the vehicle and safe operation. Traditional field surveys on highway damage cannot entirely, rapidly and precisely obtain the distress information along the whole road. In this paper, the Qinghai-Tibet Highway (QTH) pavement distress was investigated in 550-km-long permafrost regions based on the field surveys in 2014 and 2019 and the unmanned aerial vehicle (UAV) imagery. The type and damage ratio of distress were extracted by the remote sensing image classification method. The pavement roughness and embankment height were acquired by the spatial analysis of geographical information system. This paper provides a novel insight and efficient method for the distress investigation and exploration of distress formation in permafrost region, which can be applied in other similar engineering projects in the cold and inclement permafrost region. Furthermore, this paper also presents valuable and first-hand field data for evaluating highway serviceability and prevention of road damage during operation and maintenance stages.

Quantifying the impacts of land cover change on catchment-scale urban flooding by classifying aerial images

Stormwater urban drainage systems are historically designed to mitigate certain recurrence interval flooding events. However, increases in impervious land cover due to urban redevelopment enhance the surface imperviousness and then magnifies urban floods over the design criteria by elevating flooding peaks and volumes. In this paper, the object-based image classification was developed to evaluate the impervious land cover impacts on system peak flooding rate and total flooding volume. The object-based aerial image classification quantified the spatiotemporal changes in impervious surfaces with an average accuracy of 92%. A drainage model, based on a stormwater drainage system in Salt Lake City, Utah, USA, was calibrated and validated under four storm events with an average of Nash-Sutcliffe efficiency of 0.69 and 0.46 for calibration and validation outcomes. Results show that impervious areas increased up to 32% under impacts induced by urban redevelopment from 1950 to 2018. The land cover imperviousness resulted in a maximum of 575% and 753% growth in flooding peak and volume, respectively, under storm events from 10-year to 100-year return periods. Implications of this research seek to inform homeowners and engineers of the flooding risks in human-altered landscapes based on remote sensing image classification.

Edge protection filtering and convolutional neural network for hyperspectral remote sensing image classification

Hyperspectral remote sensing (HRS) image has abundant spectral information and spatial information, which provides the possibility for high precision ground object recognition. Aiming at the problem that spatial information is not fully utilized in the classification of HRS images, a classification method based on edge protection filtering combined with deep learning is proposed. First, the first three principal components of the original image are extracted as the input image of classification. Then, the input image is filtered with the first principal component as the calculation basis to realize the enhancement of spatial features. Finally, the neighborhood structure of the enhanced features is extracted and the convolutional neural network is utilized for deep feature extraction and classification. The results of experiments performed on two well-known HRS image datasets show that compared with other related methods, this method has higher classification accuracy. When randomly selecting different numbers of samples for each class as training samples, the overall classification accuracy of the two datasets is increased to 98.47% and 99.30%, respectively. The results show that the proposed method can enhance the edge structure of the HRS image, and the constructed network model can extract deep features that are conducive to classification, which effectively improves the accuracy of image classification.

Self-Training with Spatially Confident and Diverse Samples for the Classification of Remote Sensing Images

Self-Training with Spatially Confident and Diverse Samples for the Classification of Remote Sensing Images

Hyperspectral remote sensing image classification based on spectral-spatial feature fusion and PSO algorithm

Hyperspectral images(HSI) have rich spectral information and spatial information. In the classification of hyperspectral images, the combination of spectral information and spatial information has become an effective means to obtain good classification results. Specifically, firstly, PCA algorithm is introduced to extract the spectral information of the image. Secondly, a bilateral filter is introduced for each band to extract the spatial information of the image. Thirdly, the image is classified by using SVM to get the final classification result. Considering that the performance of SVM depends on the choice of parameters, particle swarm optimization algorithm (PSO) is combined with the proposed method to find the optimal penalty parameters and kernel parameters. We find that the classification accuracy is effectively improved by using PCA, bilateral filtering and optimizing SVM-PSO parameters on this basis. Experimental results based on one real hyperspectral dataset show that the proposed algorithm can achieve higher classification accuracy in a shorter time than other algorithms.

Comparison of Random Forest and Support Vector Machine Classifiers for Regional Land Cover Mapping Using Coarse Resolution FY-3C Images

The type of algorithm employed to classify remote sensing imageries plays a great role in affecting the accuracy. In recent decades, machine learning (ML) has received great attention due to its robustness in remote sensing image classification. In this regard, random forest (RF) and support vector machine (SVM) are two of the most widely used ML algorithms to generate land cover (LC) maps from satellite imageries. Although several comparisons have been conducted between these two algorithms, the findings are contradicting. Moreover, the comparisons were made on local-scale LC map generation either from high or medium resolution images using various software, but not Python. In this paper, we compared the performance of these two algorithms for large area LC mapping of parts of Africa using coarse resolution imageries in the Python platform by the employing Scikit-Learn (sklearn) library. We employed a big dataset, 297 metrics, comprised of systematically selected 9-month composite FegnYun-3C (FY-3C) satellite images with 1 km resolution. Several experiments were performed using a range of values to determine the best values for the two most important parameters of each classifier, the number of trees and the number of variables, for RF, and penalty value and gamma for SVM, and to obtain the best model of each algorithm. Our results showed that RF outperformed SVM yielding 0.86 (OA) and 0.83 (k), which are 1–2% and 3% higher than the best SVM model, respectively. In addition, RF performed better in mixed class classification; however, it performed almost the same when classifying relatively pure classes with distinct spectral variation, i.e., consisting of less mixed pixels. Furthermore, RF is more efficient in handling large input datasets where the SVM fails. Hence, RF is a more robust ML algorithm especially for heterogeneous large area mapping using coarse resolution images. Finally, default parameter values in the sklearn library work well for satellite image classification with minor/or no adjustment for these algorithms.

Impacts of Temporal Changes in Land Use/Cover on the Remaining Historical Forests in Guiyang, China

The remaining historical forests are crucial for maintaining biodiversity in urban areas. However, the integrity and stability of historical forests are affected by land-use/cover change. A better understanding of these impacts can help prioritize protection and restoration. In this study, we estimated the loss of area and levels of threats and analyzed the importance of critical areas for historical forests in the built-up area of a rapidly urbanizing city—Guiyang, China. We used the threat indicator, morphological spatial pattern analysis, and probability of connectivity based on InVEST, GuidosToolbox, and Conefor software. The results based on remote sensing image classification showed that 1988.46 ha of historical forestlands was transformed into other land-use/cover types. The mean value of the threat index of all the historical forest patches increased by 33% compared to the baseline year. The area of cores and bridges, considered as key nodes and links for the connectivity of forest networks, decreased by 193.32 and 353.61 ha, respectively. Most of the critical areas with high importance values for connectivity were located in the central part of the city and were severely threatened by the surrounding areas. We recommend that effective measures be implemented to control the further loss of historical forests and to increase the connectivity and buffering capacity of the remaining forests by creating tree belts and corridors in key locations.

RETRACTED ARTICLE: Big data classification of remote sensing image based on cloud computing and convolutional neural network

In recent years, with the continuous development of cloud computing and big data technology, information technology is penetrating all corners of enterprise development at a speed that ordinary people cannot imagine. Today's remote sensing image classification plays an important role in many applications. This article is based on cloud computing and convolutional neural network to study the remote sensing image big data classification. The article aims to fully focus on cloud computing technology and convolutional neural network technology to carry out specific application research, on this basis, to obtain a more convenient, accurate and efficient remote sensing image classification method, and to prove the feasibility of this method. This article takes cloud computing technology and convolutional neural network technology as the technical points. First, it introduces cloud computing technology in more detail from the basic concepts, characteristics and classification of cloud computing and then compares traditional methods from the selection of feature extraction methods and classifiers. The remote sensing image classification method is described, and finally, the convolutional neural network model that needs to be applied in this research is explained. In the experiment, the features extracted by multiple pre-training networks are fused. Through the study of the three feature fusion methods, it is found that the classification accuracy rate has been further improved on the three data sets. The experimental results in this paper show that the accuracy of the traditional method is below 80%, while the feature extraction of the pre-trained network has an accuracy of more than 90% on the UCMERCE data; it combines the three pre-trained networks of ResNet-50, ResNet-101 and ResNet-152, an increase of more than 2.5%.

Fusion and classification of multi-temporal SAR and optical imagery using convolutional neural network

ABSTRACT Remote sensing image classification is difficult, especially for agricultural crops with identical phenological growth periods. In this context, multi-sensor image fusion allows a comprehensive representation of biophysical and structural information. Recently, Convolutional Neural Network (CNN)-based methods are used for several applications due to their spatial-spectral interpretability. Hence, this study explores the potential of fused multi-temporal Sentinel 1 (S1) and Sentinel 2 (S2) images for Land Use/Land Cover classification over an agricultural area in India. For classification, Bayesian optimised 2D CNN-based DL and pixel-based SVM classifiers were used. For fusion, a CNN-based siamese network with Ratio-of-Laplacian pyramid method was used for the images acquired over the entire winter cropping period. This fusion strategy leads to better interpretability of results and also found that 2D CNN-based DL classifier performed well in terms of classification accuracy for both single-month (95.14% and 96.11%) as well as multi-temporal (99.87% and 99.91%) fusion in comparison to the SVM with classification accuracy for single-month (80.02% and 81.36%) and multi-temporal fusion (95.69% and 95.84%). Results indicate better performance by Vertical-Vertical polarised fused images than Vertical-Horizontal polarised fused images. Thus, implying the need to analyse classified images obtained by DL classifiers along with the classification accuracy.

Deep Transfer Learning based Fusion Model for Environmental Remote Sensing Image Classification Model

ABSTRACT Remote-sensing images comprise massive amount of spatial and semantic data that can be employed for several applications. Presently, deep learning (DL) models for RS image processing become a familiar research area. Due to the advancements of recent satellite imaging sensors , the issue of huge amount of data processing becomes a challenging problem. To accomplish this, deep transfer learning (DTL) models are developed to resolve the semantic gap among various datasets This study develops a new DTL-based fusion model for environmental remote-sensing image classification, called DTLF-ERSIC technique. The proposed technique focuses on the design of fusion model to combine multiple feature vectors and thereby attains maximum classification performance. The DTLF-ERSIC technique incorporates the entropy-based fusion of three feature extraction techniques, namely, Discrete Local Binary Pattern (DLBP), Residual Network (ResNet50), and EfficientNet models. Besides, a rain optimization algorithm (ROA) with fuzzy rule-based classifier (FRC) is applied to predict the class labels of the test RS images and shows the novelty of the work. A comprehensive experimental analysis of the DTLF-ERSIC technique takes place on benchmark dataset and examined the results in terms of different performance measures. The simulation results reported the supremacy of the DTLF-ERSIC technique over the recent state-of-art techniques.

Patch-U-Net: tree species classification method based on U-Net with class-balanced jigsaw resampling

ABSTRACT Automatic tree species classification based on remote-sensing images can significantly improve the efficiency of tree species investigation and save considerable cost of human labor. A fully convolutional network (FCN) can automatically extract tree species-related features to achieve higher classification performance. However, this kind of method needs a large quantity of training data. Due to few samples and unbalanced sample distribution of tree species remote-sensing images, directly applying FCN to tree species image classification task could not achieve good results. We proposed Patch-U-Net to tackle the above problem. Our method adopts the class-balanced jigsaw resampling strategy to explicitly balance inter-class distribution and augment data in patch-wise. Besides, it extracts multi-scale information of each patch by combining the encoder–decoder and skip connection structure. We compared Patch-U-Net with six existing methods, and Patch-U-Net achieved the best performance. Specifically, Pixel Accuracy (PA), Mean Intersection over Union (MIoU), and Frequency Weighted Intersection over Union (FWIoU) of Patch-U-Net are 80.33%, 57.46%, and 67.37%, which are 14.3%, 33.16%, and 17.81% higher than those of the baseline model U-Net, respectively. The results show that Patch-U-Net can improve the performance of remote-sensing tree species classification by solving the problem of unbalanced samples, which is more suitable for the remote-sensing image classification of tree species with imbalance species.

Domain Adaptation in Remote Sensing Image Classification: A Survey

Traditional remote sensing (RS) image classification methods heavily rely on labeled samples for model training. When labeled samples are unavailable or labeled samples have different distributions from that of the samples to be classified, the classification model may fail. The cross-domain or cross-scene remote sensing image classification is developed for this case where an existing image for training and an unknown image from different scenes or domains for classification. The distribution inconsistency problem may be caused by the differences in acquisition environment conditions, acquisition scene, acquisition time or/and changing sensors. To cope with the cross-domain remote sensing image classification problem, many domain adaptation (DA) techniques have been developed. In this article, we review DA methods in the fields of RS, especially hyperspectral image classification, and provide a survey of DA methods into traditional shallow DA methods (e.g., instance-based, feature-based, and classifier-based adaptations) and recently developed deep DA methods (e.g., discrepancy-based and adversarial-based adaptations).