Remote Sensing Image Classification Research Articles

Deep Learning for Urban Tree Canopy Coverage Analysis: A Comparison and Case Study

Urban tree canopy (UTC) coverage, or area, is an important metric for monitoring changes in UTC over large areas within a municipality. Several methods have been used to obtain these data, but remote sensing image classification is one of the fastest and most reliable over large areas. However, most studies have tested only one or two classification methods to accomplish this while using costly satellite imagery or LiDAR data. This study seeks to compare three urban tree canopy cover classifiers by testing a deep learning U-Net convolutional neural network (CNN), support vector machine learning classifier (SVM) and a random forests machine learning classifier (RF) on cost-free 2012 aerial imagery over a small southern USA city and midsize, growing southern USA city. The results of the experiment are then used to decide the best classifier and apply it to more recent aerial imagery to determine canopy changes over a 10-year period. The changes are subsequently compared visually and statistically with recent urban heat maps derived from thermal Landsat 9 satellite data to compare the means of temperatures within areas of UTC loss and no change. The U-Net CNN classifier proved to provide the best overall accuracy for both cities (89.8% and 91.4%), while also requiring the most training and classification time. When compared spatially with city heat maps, city periphery regions were most impacted by substantial changes in UTC area as cities grow and the outer regions get warmer. Furthermore, areas of UTC loss had higher temperatures than those areas with no canopy change. The broader impacts of this study reach the urban forestry managers at the local, state/province, and national levels as they seek to provide data-driven decisions for policy makers.

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

The advancement in satellite image sensors has enabled the acquisition of high-resolution remote sensing (HRRS) images. However, interpreting these images accurately and obtaining the computational power needed to do so is challenging due to the complexity involved. This manuscript proposed a multi-stream convolutional neural network (CNN) fusion framework that involves multi-scale and multi-CNN integration for HRRS image recognition. The pre-trained CNNs were used to learn and extract semantic features from multi-scale HRRS images. Feature extraction using pre-trained CNNs is more efficient than training a CNN from scratch or fine-tuning a CNN. Discriminative canonical correlation analysis (DCCA) was used to fuse deep features extracted across CNNs and image scales. DCCA reduced the dimension of the features extracted from CNNs while providing a discriminative representation by maximizing the within-class correlation and minimizing the between-class correlation. The proposed model has been evaluated on NWPU-RESISC45 and UC Merced datasets. The accuracy associated with DCCA was 10% and 6% higher than discriminant correlation analysis (DCA) in the NWPU-RESISC45 and UC Merced datasets. The advantage of DCCA was better demonstrated in the NWPU-RESISC45 dataset due to the incorporation of richer within-class variability in this dataset. While both DCA and DCCA minimize between-class correlation, only DCCA maximizes the within-class correlation and, therefore, attains better accuracy. The proposed framework achieved higher accuracy than all state-of-the-art frameworks involving unsupervised learning and pre-trained CNNs and 2–3% higher than the majority of fine-tuned CNNs. The proposed framework offers computational time advantages, requiring only 13 s for training in NWPU-RESISC45, compared to a day for fine-tuning the existing CNNs. Thus, the proposed framework achieves a favourable balance between efficiency and accuracy in HRRS image recognition.

Text-aware small sample remote sensing image classification based on contrastive learning

In view of the fact that existing methods mainly use a single modality of remote sensing images to solve the problem of low similarity of the same category, a remote sensing image classification method based on multimodal learning is proposed. Firstly, the spatial features of the image are corrected and the image encoder is pre-trained using contrastive learning to generate image features, and the text encoder is used to generate text features. Secondly, a feature decoder is introduced to obtain text-aware visual features, and a new attention mechanism method is proposed in the feature fusion stage. Then, a new image encoder is designed to improve the classification accuracy. Finally, the similarity between the support set and the query set is calculated to further predict the category. Experiments are conducted on the NWPU-RESISC45, AID, and UC Merced datasets. Its 5-way 5-shot accuracy reaches 86.46%、85.89%and 80.32%outperforms the existing small-sample remote sensing image classification methods, respectively.

Semi-Supervised Subcategory Centroid Alignment-Based Scene Classification for High-Resolution Remote Sensing Images

It is usually hard to obtain adequate annotated data for delivering satisfactory scene classification results. Semi-supervised scene classification approaches can transfer the knowledge learned from previously annotated data to remote sensing images with scarce samples for satisfactory classification results. However, due to the differences between sensors, environments, seasons, and geographical locations, cross-domain remote sensing images exhibit feature distribution deviations. Therefore, semi-supervised scene classification methods may not achieve satisfactory classification accuracy. To address this problem, a novel semi-supervised subcategory centroid alignment (SSCA)-based scene classification approach is proposed. The SSCA framework is made up of two components, namely the rotation-robust convolutional feature extractor (RCFE) and the neighbor-based subcategory centroid alignment (NSCA). The RCFE aims to suppress the impact of rotation changes on remote sensing image representation, while the NSCA aims to decrease the impact of intra-category variety across domains on cross-domain scene classification. The SSCA algorithm and several competitive approaches are validated on two datasets to demonstrate its effectiveness. The results prove that the proposed SSCA approach performs better than most competitive approaches by no less than 2% overall accuracy.

Classification of high-resolution remote sensing images based on interval type-2 fuzzy logic system

Abstract Aiming at the complexity and uncertainty of remote sensing image classification, this paper proposes a high-resolution remote sensing image classification method based on Interval type-2 Fuzzy Logic Systems for Remote Sensing (IT2FLS-RS), which focuses on dealing with multiple uncertainties and aims to improve the accuracy and reliability of remote sensing image classification. The method establishes a more complex interval two-type fuzzy system covering two types of fuzzifiers, a rule base and an inference machine, and finally uses an integrated algorithm as a defuzzifier to achieve accurate pixel-level classification. In addition, the model uses Constrained Optimization BY Linear Approximations (COBYLA) to optimise the key parameters. In the DLRSD dataset, the accuracy of this model is improved by about 20%, 14%, 24% and 10% compared to the state-of-the-art interval two-type fuzzy neural network algorithm and the benchmark model XGBoost, respectively. On the WHDLD dataset, the accuracy of the proposed method is improved by about 12% and 10% compared to the state-of-the-art interval type-2 fuzzy neural network algorithm and the benchmark model XGBoost, respectively. The experimental results confirm the robustness of the proposed method in processing high-resolution remote sensing image classification, especially the excellent adaptability and scalability in complex feature scenes.

Remote sensing image classification based on non-linear enhanced attention mechanism

Abstract In recent years, Transformer technology has gradually shown promising applications in the field of computer vision, becoming a research hotspot. However, traditional vision Transformers suffer from significant computational burdens. Although previous studies have attempted to alleviate this issue by reducing the computational load of attention mechanisms, their methods still require improvement. Additionally, past research often overlooked the non-linear representation of values within attention mechanisms, posing another challenge to address. Therefore, this paper proposes a novel attention mechanism aimed at reducing the computational burden of traditional attention mechanisms while enhancing their ability to express non-linearities in values. Furthermore, conventional feedforward neural networks typically output results directly after passing through an activation function, which may limit the model’s representational capacity due to the simplistic structure leading to singular feature information. To address this issue, this study introduces a more robust feature fusion convolutional feedforward network. Through these methods, the aim is to enhance the accuracy of Transformers in the field of remote-sensing image classification.

Forest Change Monitoring Based on Block Instance Sampling and Homomorphic Hypothesis Margin Evaluation

Forests play a crucial role in maintaining the integrity of natural ecosystems. Accurate mapping of windfall damages following storms is essential for effective post-disaster management. While remote sensing image classification offers substantial advantages over ground surveys for monitoring changes in forests, it encounters several challenges. Firstly, training samples in classification algorithms are typically selected through pixel-based random sampling or manual regional sampling. This approach struggles with accurately modeling complex patterns in high-resolution images and often results in redundant samples. Secondly, the limited availability of labeled samples compromises the classification accuracy when they are divided into training and test sets. To address these issues, two innovative approaches are proposed in this paper. The first is a new sample selection method which combines block-based sampling with spatial features extracted by single or multiple windows. Second, a new evaluation criterion is proposed by using the homomorphic hypothesis margin map with out-of-bag (OOB) accuracy. The former can not only assess the confidence level of each pixel category but also make regional boundaries clearer, and the latter can replace the test set so that all samples can be used for change detection. The experimental results show that the OOB accuracy obtained by spatial features with whole block sampling was 7.2% higher than that obtained by spectral features with pixel-based sampling and 2–3% higher than that for block center sampling, of which the highest value reached 98.8%. Additionally, the feasibility of identifying storm-damaged forests using only post-storm images has been demonstrated.

Intelligent classification of water bodies with different turbidity levels based on Gaofen-1 multispectral imagery

The turbidity of water is crucial for the health of river and lake ecosystems, necessitating efficient monitoring for effective water management. Existing methods for studying water turbidity's spatial and temporal distribution rely mostly on measured data. There is limited research on the classification of water bodies with different turbidity levels. The main challenge lies in determining the boundaries of liquid water bodies at various turbidity levels, making it challenging to classify them accurately using traditional remote sensing image classification methods. This paper proposes and validates an intelligent turbidity classification method based on deep learning using GaoFen-1 multispectral remote sensing imagery. An adaptive threshold water extraction method based on the Normalized Difference Water Index is proposed to capture water boundaries more accurately to improve the accuracy of extracting nearshore water bodies. A semi-automatic semantic annotation method for water turbidity is introduced to reduce manual labeling costs. The paper applies mode filtering to address edge noise issues and establishes a high-quality training sample dataset. After comparing the accuracy of various neural network models, DeepLab V3+ is selected for intelligent turbidity classification. The results show high accuracy, with mean intersection over union (MIoU), mean F1 score (MF1), and overall accuracy (OA) reaching 94.73%, 97.29%, and 97.54%, respectively. The proposed method and experiments demonstrate the feasibility of intelligent classification of water bodies with different turbidity levels using deep learning networks. This provides a new approach for large-scale and efficient remote sensing water turbidity monitoring.

Gini Coefficient-Based Feature Learning for Unsupervised Cross-Domain Classification with Compact Polarimetric SAR Data

Remote sensing image classification usually needs many labeled samples so that the target nature can be fully described. For synthetic aperture radar (SAR) images, variations of the target scattering always happen to some extent due to the imaging geometry, weather conditions, and system parameters. Therefore, labeled samples in one image could not be suitable to represent the same target in other images. The domain distribution shift of different images reduces the reusability of the labeled samples. Thus, exploring cross-domain interpretation methods is of great potential for SAR images to improve the reuse rate of existing labels from historical images. In this study, an unsupervised cross-domain classification method is proposed that utilizes the Gini coefficient to rank the robust and stable polarimetric features in both the source and target domains (GRFST) such that an unsupervised domain adaptation (UDA) can be achieved. This method selects the optimal features from both the source and target domains to alleviate the domain distribution shift. Both fully polarimetric (FP) and compact polarimetric (CP) SAR features are explored for crop-domain terrain type classification. Specifically, the CP mode refers to the hybrid dual-pol mode with an arbitrary transmitting ellipse wave. This is the first attempt in the open literature to investigate the representing abilities of different CP modes for cross-domain terrain classification. Experiments are conducted from four aspects to demonstrate the performance of CP modes for cross-data, cross-scene, and cross-crop type classification. Results show that the GRFST-UDA method yields a classification accuracy of 2% to 12% higher than the traditional UDA methods. The degree of scene similarity has a certain impact on the accuracy of cross-domain crop classification. It was also found that when both the FP and circular CP SAR data are used, stable, promising results can be achieved.

Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classification: A review

Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classification: A review

TCPSNet: Transformer and Cross-Pseudo-Siamese Learning Network for Classification of Multi-Source Remote Sensing Images

TCPSNet: Transformer and Cross-Pseudo-Siamese Learning Network for Classification of Multi-Source Remote Sensing Images

State space models meet transformers for hyperspectral image classification

In recent years, convolutional neural networks and vision transformers have emerged as predominant models for hyperspectral remote sensing image classification task, leveraging staked convolution layers and self-attention mechanisms with high computation costs, respectively. Recent studies, such as the Mamba model, have showcased the ability of state space model (SSM) with efficient hardware-aware designs in efficiently modeling sequences and extracting implicit features along tokens, which is precisely needed for accurate hyperspectral image (HSI) classification. Thus making SSM-based model potentially a new architecture for remote sensing HSI classification task. However, SSM encounters challenges in modeling HSI due to the insensitivity of spatial information and redundant spectral characteristics. Given SSM-based methods rarely explored in HSI classification, in this work, we present the first exploration of SSM-based models for HSI classification task. Our proposed method MamTrans effectively leverages the capacity of transformer for capturing spatial tokens relationships and Mamba for extracting implicit features along tokens. Besides, we propose a Bidirectional Mamba Module to enhance SSM’s spatial perception ability of extracting spatial features in HSI. Our proposed MamTrans obtains a new state-of-the-art performance across five commonly employed HSI classification benchmarks, demonstrating the robust generalization of MamTrans and effectiveness of SSM-based structure for HSI classification task. Our codes could be found at https://github.com/PPPPPsanG/MamTrans.

Leveraging Pre-trained Deep Learning Models for Remote Sensing Image Classification: A Case Study with ResNet50 and EfficientNet

The procedure of categorizing images from remote sensing is also another application of machine learning not just ground-based platforms (for instance satellites), aerial platforms become platforms sometimes in aviation either. They erase the counterparts that were based on individual categories and are portrayed on a specific part of the image. Geospatial Supply of gravel mainly is used for producing railway track, road and concrete surface. Data by analyzing their buildup, dams, bridges, extraordinary open spaces, reservoirs and canals. It targets to be specific and exact as possible in a different specific area of the land. Aspects of the enlarged portrait or distinctions weaved into the completed arts. This might have aspects such as mapping of the trees, plants, rivers, cities, farms and woodlands, and other items. Geospatial image classification is necessary for the identification and real-time analysis of different hazards and unrests. Provide numerous applications, including waste management, water resources, air quality, and traffic control in the urban contexts. Planning, monitoring the environment, land cover, mapping, as well as post-disaster recovery. Management team, traffic control, and situation assessments. In the past, human experts situated in a selected area classified geographical images by means of manual processing. One that involved the allocation of too much time. As this is one of the two broad categories, how to get rid of it is consequently. Applying machine learning and deep learning methods we analyze and interpret the data in order to reduce the time required to provide feedback which allows the system to reach a higher accuracy. The procedure will also be more reliable and the outcome will hopefully be more efficient CNNs are one of the deep learning subclasses in which the network learns and improves without the need for human intervention. It extracts features from images. They are main for the performance and metrics to help the organization to decide on whether they have accomplished their goals, using visual imagery.

Few-Shot Hyperspectral Remote Sensing Image Classification via an Ensemble of Meta-Optimizers with Update Integration

Few-Shot Hyperspectral Remote Sensing Image Classification via an Ensemble of Meta-Optimizers with Update Integration

Application of Remote Sensing Image Classification Utilising Deep Learning in Technological Domains

Application of Remote Sensing Image Classification Utilising Deep Learning in Technological Domains

Implicit Sharpness-Aware Minimization for Domain Generalization

Domain generalization (DG) aims to learn knowledge from multiple related domains to achieve a robust generalization performance in unseen target domains, which is an effective approach to mitigate domain shift in remote sensing image classification. Although the sharpness-aware minimization (SAM) method enhances DG capability and improves remote sensing image classification performance by promoting the convergence of the loss minimum to a flatter loss surface, the perturbation loss (maximum loss within the neighborhood of a local minimum) of SAM fails to accurately measure the true sharpness of the loss landscape. Furthermore, its variants often overlook gradient conflicts, thereby limiting further improvement in DG performance. In this paper, we introduce implicit sharpness-aware minimization (ISAM), a novel method that addresses the deficiencies of SAM and mitigates gradient conflicts. Specifically, we demonstrate that the discrepancy in training loss during gradient ascent or descent serves as an equivalent measure of the dominant eigenvalue of the Hessian matrix. This discrepancy provides a reliable measure for sharpness. ISAM effectively reduces sharpness and mitigates potential conflicts between gradients by implicitly minimizing the discrepancy between training losses while ensuring a sufficiently low minimum through minimizing perturbation loss. Extensive experiments and analyses demonstrate that ISAM significantly enhances the model’s generalization ability on remote sensing and DG datasets, outperforming existing state-of-the-art methods.

Learning by counterexamples in remote sensing image classification: a case study for road extraction

ABSTRACT The classification of multi- and hyper-spectral imagery is essential for several remote sensing applications. However, several challenges still hinder the accuracy and efficiency of learning algorithms. In diverse remote sensing applications, the high spectral heterogeneity between and within classes is often an issue that adds uncertainties to the classification. Most classifiers do not consider the influence of non-target classes and variations in the spectral similarity level between classes during classification. This paper presents a novel learning strategy for remote sensing image classification, which exploits non-target classes to enhance the generation of classification masks and the performance of supervised methods for specific classes of interest. It produces multiple classification instances, based on the incorporation of counterexample information into the training data set, in order to refine the separation of classes of interest in the available feature space. The experimental results demonstrate the potential of the proposed counterexample assisted learning for road extraction, using high spatial-resolution imagery data.

An Adaptive Migration Collaborative Network for Multimodal Image Classification.

The multispectral (MS) and the panchromatic (PAN) images belong to different modalities with specific advantageous properties. Therefore, there is a large representation gap between them. Moreover, the features extracted independently by the two branches belong to different feature spaces, which is not conducive to the subsequent collaborative classification. At the same time, different layers also have different representation capabilities for objects with large size differences. In order to dynamically and adaptively transfer the dominant attributes, reduce the gap between them, find the best shared layer representation, and fuse the features of different representation capabilities, this article proposes an adaptive migration collaborative network (AMC-Net) for multimodal remote-sensing (RS) images classification. First, for the input of the network, we combine principal component analysis (PCA) and nonsubsampled contourlet transformation (NSCT) to migrate the advantageous attributes of the PAN and the MS images to each other. This not only improves the quality of images themselves, but also increases the similarity between the two images, thereby reducing the representational gap between them and the pressure on the subsequent classification network. Second, for the interaction on the feature migrate branch, we design a feature progressive migration fusion unit (FPMF-Unit) based on the adaptive cross-stitch unit of correlation coefficient analysis (CCA), which can make the network automatically learn the features that need to be shared and migrated, aiming to find the best shared-layer representation for multifeature learning. And we design an adaptive layer fusion mechanism module (ALFM-Module), which can adaptively fuse features of different layers, aiming to clearly model the dependencies among multiple layers for different sized objects. Finally, for the output of the network, we add the calculation of the correlation coefficient to the loss function, which can make the network converge to the global optimum as much as possible. The experimental results indicate that AMC-Net can achieve competitive performance. And the code for the network framework is available at: https://github.com/ru-willow/A-AFM-ResNet.

Border-Enhanced Triple Attention Mechanism for High-Resolution Remote Sensing Images and Application to Land Cover Classification

With the continuous development and popularization of remote sensing technology, remote sensing images have been widely used in the field of land cover classification. Since remote sensing images have complex spatial structure and texture features, it is becoming a challenging problem to accurately categorize them. Land cover classification has practical application value in various fields, such as environmental monitoring and protection, urban and rural planning and management, and climate change research. In recent years, remote sensing image classification methods based on deep learning have been rapidly developed, in which semantic segmentation technology has become one of the mainstream methods for land cover classification using remote sensing image. Traditional semantic segmentation algorithms tend to ignore the edge information, resulting in poor classification of the edge part in land cover classification, and there are numerous attention mechanisms to make improvements for these problems. In this paper, a triple attention mechanism, BETAM (Border-Enhanced Triple Attention Mechanism), for edge feature enhancement of high-resolution remote sensing images is proposed. Furthermore, a new model on the basis of the semantic segmentation network model DeeplabV3+ is also introduced, which is called DeepBETAM. The triple attention mechanism BETAM is able to capture feature dependencies in three dimensions: position, space, and channel, respectively. Through feature importance weighting, modeling of spatial relationships, and adaptive learning capabilities, the model BETAM pays more attention to edge features, thus improving the accuracy of edge detection. A remote sensing image dataset SMCD (Subject Meticulous Categorization Dataset) is constructed to verify the robustness of the attention mechanism BETAM and the model DeepBETAM. Extensive experiments were conducted on the two self-built datasets FRSID and SMCD. Experimental results showed that the mean Intersection over Union (mIoU), mean Pixel Accuracy (mPA), and mean Recall (mRecall) of DeepBETAM are 63.64%, 71.27%, and 71.31%, respectively. These metrics are superior to DeeplabV3+, DeeplabV3+(SENet), DeeplabV3+(CBAM), DeeplabV3+(SAM), DeeplabV3+(ECANet), and DeeplabV3+(CAM), which are network models that incorporate different attention mechanisms. The reason is that BETAM has better edge segmentation results and segmentation accuracy. Meanwhile, on the basis of the self-built dataset, the four main classifications of buildings, cultivated land, water bodies and vegetation were subdivided and detected, and good experimental results were obtained, which verified the robustness of the attention mechanism BETAM and the model DeepBETAM. The method has broad application prospects and can provide favorable support for research and application in the field of surface classification.

Assessing interpreter’s disagreements in land cover reference data collection from historical Landsat time series in Amazon

ABSTRACT Land cover information, derived from the classification of Remote Sensing images, is only useful if accompanied by a rigorous accuracy assessment, usually dependent on reference samples. As field data can be very rare for multi-temporal analysis, many studies use samples visually interpreted from Remote Sensing images. This subjective process is prone to errors in interpretation, leading to uncertainties not fully quantified in the reference sample collection of historical time series. We evaluated the agreement rate among independent interpreters in the identification of land cover classes assigned to pixels for yearly Landsat observations from 1984 to 2020, in the Lower Tapajós Basin region (within the Brazilian Amazon). Considering 10 classes, we found that sets of three interpreters completely disagreed on the label of 14.8% of the pixels for each year, and completely agreed in less than half (39.6%). These values change to 2.7% and 68.8% when we consider only five classes in the analysis. We observed time-dependent variations in agreement for specific land cover classes: whereas forest areas tend to present an increase in the disagreement rate over time, land cover types that happen only in previously deforested areas tend to present a higher proportion of pixels with better agreement rates. Disagreements tend to be higher in pixels identified as borders or near borders between targets. However, there is a non-negligible amount of disagreements in pixels identified as not borders caused by limitations derived from the unit of analysis, image composition techniques, and intrinsic similarities among classes. We highlight problems with the identification of classes that happen in complex spatiotemporal patterns (Secondary Vegetation, Small-scale Agriculture, and Degraded Forest). These results alert to possible inconsistencies in reference samples collected for the accuracy assessment of land cover time series and the correspondent values of the calculated accuracy indexes.