Hyperspectral Image Classification Research Articles

Lightweight omni-dimensional dynamic convolution with spatial-spectral self-attention network for hyperspectral image classification

ABSTRACT Omni-dimensional dynamic convolution can adjust spatial sizes, the amount of input/output channels, and kernel size based on the characteristics of the input, thus improving the capacity to extract characteristics through convolution, resulting in increased computation and model complexity. As a result, it is challenging to deploy these models on resource-limited airborne high-spectral testing systems. This paper proposes a lightweight omni-dimensional dynamic convolution network as a possible solution to this issue. In this network, the LS2CM module is used for spatial-spectral information extraction, and the pyramidal residual network is the basic architecture for extracting features from shallow to deep layers, and lightweight omni-dimensional dynamic convolution is adopted in the convolution module. Overcoming the pyramidal structure can only extract local information, we proposed a new simple spatial-spectral self-attention mechanism to extract global information from hyperspectral. Besides, the Huber function as an activation function. The IP, WHU-Hi-LongKou, SA, and PU datasets were used to verify the accuracy of the suggested network model. The results show that the proposed lightweight network maintains an appropriate equilibrium among model complexity as well as accuracy.

AMHFN: Aggregation Multi-Hierarchical Feature Network for Hyperspectral Image Classification

Deep learning methods like convolution neural networks (CNNs) and transformers are successfully applied in hyperspectral image (HSI) classification due to their ability to extract local contextual features and explore global dependencies, respectively. However, CNNs struggle in modeling long-term dependencies, and transformers may miss subtle spatial-spectral features. To address these challenges, this paper proposes an innovative hybrid HSI classification method aggregating hierarchical spatial-spectral features from a CNN and long pixel dependencies from a transformer. The proposed aggregation multi-hierarchical feature network (AMHFN) is designed to capture various hierarchical features and long dependencies from HSI, improving classification accuracy and efficiency. The proposed AMHFN consists of three key modules: (a) a Local-Pixel Embedding module (LPEM) for capturing prominent spatial-spectral features; (b) a Multi-Scale Convolutional Extraction (MSCE) module to capture multi-scale local spatial-spectral features and aggregate hierarchical local features; (c) a Multi-Scale Global Extraction (MSGE) module to explore multi-scale global dependencies and integrate multi-scale hierarchical global dependencies. Rigorous experiments on three public hyperspectral image (HSI) datasets demonstrated the superior performance of the proposed AMHFN method.

Review of Hyperspectral Image Classification Based on Deep Learning

Review of Hyperspectral Image Classification Based on Deep Learning

A hybrid approach consisting of 3D depthwise separable convolution and depthwise squeeze-and-excitation network for hyperspectral image classification

A hybrid approach consisting of 3D depthwise separable convolution and depthwise squeeze-and-excitation network for hyperspectral image classification

A semantic edge-aware parameter efficient image filtering technique

The success of a structure preserving filtering technique has relied on its capability to recognize structures and textures present in the input image. In this paper a novel structure preserving filtering technique is presented that first, generates an edge-map of the input image by exploiting semantic information. Then, an edge-aware adaptive recursive median filter is utilized to produce the filter image. The technique provides satisfactory results for a wide variety of images with minimal fine-tuning of its parameters. Moreover, along with the various computer graphics applications the proposed technique also shows its robustness to incorporate spatial information for spectral-spatial classification of hyperspectral images. A MATLAB implementation of the proposed technique is available at-https://www.github.com/K-Pradhan/A-semantic-edge-aware-parameter-efficient-image-filtering-technique

Structural similarity-based noise-robust band selection model for hyperspectral image classification

Structural similarity-based noise-robust band selection model for hyperspectral image classification

A Novel Hybrid Fuzzy-based Deep Convolutional Neural Network for Big-Data-based Hyperspectral Image Classification

A Novel Hybrid Fuzzy-based Deep Convolutional Neural Network for Big-Data-based Hyperspectral Image Classification

Hyperspectral image classification with spectral-spatial feature integration and ensemble learning

<p>Hyperspectral imaging (HSI) has emerged as a robust remote sensing and medical imaging tool. However, HSI classification remains a challenging problem due to the high-dimensional data and the need for efficient feature selection and enhancement techniques. The proposed work addresses the problem of spatial feature extraction in spectral-spatial HSI classification tasks. This paper introduces an innovative model addressing the intricacies of spatial feature extraction in spectral-spatial HSI classification tasks, employing a fusion of spectral and spatial features through an adaptive kernel-based Gaussian filtering mechanism to elevate the quality of HSI data and augment classification performance. The classification is executed using three distinct classifiers, whose decisions are harmoniously integrated within an ensemble learning framework to optimize outcomes. The effectiveness of the proposed system is meticulously evaluated across three diverse datasets, Indian Pine, Pavia, and Salinas. This study also compares the model's efficiency against the existing similar work presented in the literature. The results show that the proposed work outperforms existing methods with constantly showing 99% accuracy and kappa score for each dataset, demonstrating its potential applications in various domains such as remote sensing and medical imaging.</p>

Spectral-Spatial Global Graph Reasoning for Hyperspectral Image Classification.

Convolutional neural networks (CNNs) have been widely applied to hyperspectral image classification (HSIC). However, traditional convolutions can not effectively extract features for objects with irregular distributions. Recent methods attempt to address this issue by performing graph convolutions on spatial topologies, but fixed graph structures and local perceptions limit their performances. To tackle these problems, in this article, different from previous approaches, we perform the superpixel generation on intermediate features during network training to adaptively produce homogeneous regions, obtain graph structures, and further generate spatial descriptors, which are served as graph nodes. Besides spatial objects, we also explore the graph relationships between channels by reasonably aggregating channels to generate spectral descriptors. The adjacent matrices in these graph convolutions are obtained by considering the relationships among all descriptors to realize global perceptions. By combining the extracted spatial and spectral graph features, we finally obtain a spectral-spatial graph reasoning network (SSGRN). The spatial and spectral parts of SSGRN are separately called spatial and spectral graph reasoning subnetworks. Comprehensive experiments on four public datasets demonstrate the competitiveness of the proposed methods compared with other state-of-the-art graph convolution-based approaches.

LatLBP: Spatial-spectral latent local binary pattern for hyperspectral image classification

The rich spatial and spectral information brings great potential for pixel-wise classification of hyperspectral image (HSI). Recently, Local Binary Pattern (LBP) as a prominent texture operator has been introduced for discriminative feature extraction of complex volumetric HSI. However, the popular two-dimensional LBP series can only capture spatial structure features and lack pattern descriptions of the spectral domain. The extended three-dimensional LBP series are capable of local multidimensional descriptions, while the information mining for narrow and continuous bands in the spectral domain is still limited. To tackle these issues, we develop a novel local binary pattern named LatLBP for investigating spatial-spectral latent information, which provides a low-dimensional, fine-grained, high-level representation of dual-domain latent semantic information by exploring the group structure of data. LatLBP consists of representative spatial latent (RSLat) LBP encoding and grouped spectral latent (GSLat) LBP encoding, where RSLat describes the local spatial potential features of each grouped representative band, and GSLat characterizes information about each group of latent spectral factors. Besides, a supporting neighborhood band grouping (NBG) algorithm is also designed to provide a finer group structure. Extensive experiments conducted on four HSI datasets indicate that the optimal classification results, i.e., 86.94% for Pavia University, 88.66% for Indian Pines, 77.75% for HanChuan, and 85.54% for HongHu, are achieved by either the proposed LatLBP or GSLat compared to competitive operators.

Multi-Scale Depthwise Separable Capsule Network for hyperspectral image classification.

Addressing the challenges in effectively extracting multi-scale features and preserving pose information during hyperspectral image (HSI) classification, a Multi-Scale Depthwise Separable Capsule Network (MDSC-Net) is proposed in this article for HSI classification. Initially, hierarchical features are extracted by MDSC-Net through the employment of parallel multi-scale convolutional kernels, while computational complexity is reduced via depthwise separable convolutions, thus reducing the overall computational load and achieving efficient feature extraction. Subsequently, to enhance the translational invariance of features and reduce the loss of pose information, features of various scales are processed in parallel by independent capsule networks, with improvements in max pooling achieved through dynamic routing. Lastly, features of different scales are concatenated and integrated through the concatenate operation, thereby facilitating precise analysis of multi-level information in the hyperspectral image classification process. Experimental comparisons demonstrate that MDSC-Net achieves average accuracies of 94%, 98%, and 99% on the Kennedy Space Center, University of Pavia, and Salinas datasets, respectively, indicating a significant performance advantage over recent HSI classification models and validating the effectiveness of the proposed model.

EAS$$^2$$KAM: enhanced adaptive source-selection kernel with attention mechanism for hyperspectral image classification

EAS$$^2$$KAM: enhanced adaptive source-selection kernel with attention mechanism for hyperspectral image classification

Spatial Feature Enhancement and Attention-Guided Bidirectional Sequential Spectral Feature Extraction for Hyperspectral Image Classification

Hyperspectral images have the characteristics of high spectral resolution and low spatial resolution, which will make the extracted features insufficient and lack detailed information about ground objects, thus affecting the accuracy of classification. The numerous spectral bands of hyperspectral images contain rich spectral features but also bring issues of noise and redundancy. To improve the spatial resolution and fully extract spatial and spectral features, this article proposes an improved feature enhancement and extraction model (IFEE) using spatial feature enhancement and attention-guided bidirectional sequential spectral feature extraction for hyperspectral image classification. The adaptive guided filtering is introduced to highlight details and edge features in hyperspectral images. Then, an image enhancement module composed of two-dimensional convolutional neural networks is used to improve the resolution of the image after adaptive guidance filtering and provide a high-resolution image with key features emphasized for the subsequent feature extraction module. The proposed spectral attention mechanism helps to extract more representative spectral features, emphasizing useful information while suppressing the interference of noise. Experimental results show that our method outperforms other comparative methods even with very few training samples.

Noise-Disruption-Inspired Neural Architecture Search with Spatial–Spectral Attention for Hyperspectral Image Classification

In view of the complexity and diversity of hyperspectral images (HSIs), the classification task has been a major challenge in the field of remote sensing image processing. Hyperspectral classification (HSIC) methods based on neural architecture search (NAS) is a current attractive frontier that not only automatically searches for neural network architectures best suited to the characteristics of HSI data, but also avoids the possible limitations of manual design of neural networks when dealing with new classification tasks. However, the existing NAS-based HSIC methods have the following limitations: (1) the search space lacks efficient convolution operators that can fully extract discriminative spatial–spectral features, and (2) NAS based on traditional differentiable architecture search (DARTS) has performance collapse caused by unfair competition. To overcome these limitations, we proposed a neural architecture search method with receptive field spatial–spectral attention (RFSS-NAS), which is specifically designed to automatically search the optimal architecture for HSIC. Considering the core needs of the model in extracting more discriminative spatial–spectral features, we designed a novel and efficient attention search space. The core component of this innovative space is the receptive field spatial–spectral attention convolution operator, which is capable of precisely focusing on the critical information in the image, thus greatly enhancing the quality of feature extraction. Meanwhile, for the purpose of solving the unfair competition issue in the traditional differentiable architecture search (DARTS) strategy, we skillfully introduce the Noisy-DARTS strategy. The strategy ensures the fairness and efficiency of the search process and effectively avoids the risk of performance crash. In addition, to further improve the robustness of the model and ability to recognize difficult-to-classify samples, we proposed a fusion loss function by combining the advantages of the label smoothing loss and the polynomial expansion perspective loss function, which not only smooths the label distribution and reduces the risk of overfitting, but also effectively handles those difficult-to-classify samples, thus improving the overall classification accuracy. Experiments on three public datasets fully validate the superior performance of RFSS-NAS.

A novel spectral-spatial 3D auxiliary conditional GAN integrated convolutional LSTM for hyperspectral image classification

A novel spectral-spatial 3D auxiliary conditional GAN integrated convolutional LSTM for hyperspectral image classification

State space models meet transformers for hyperspectral image classification

In recent years, convolutional neural networks and vision transformers have emerged as predominant models for hyperspectral remote sensing image classification task, leveraging staked convolution layers and self-attention mechanisms with high computation costs, respectively. Recent studies, such as the Mamba model, have showcased the ability of state space model (SSM) with efficient hardware-aware designs in efficiently modeling sequences and extracting implicit features along tokens, which is precisely needed for accurate hyperspectral image (HSI) classification. Thus making SSM-based model potentially a new architecture for remote sensing HSI classification task. However, SSM encounters challenges in modeling HSI due to the insensitivity of spatial information and redundant spectral characteristics. Given SSM-based methods rarely explored in HSI classification, in this work, we present the first exploration of SSM-based models for HSI classification task. Our proposed method MamTrans effectively leverages the capacity of transformer for capturing spatial tokens relationships and Mamba for extracting implicit features along tokens. Besides, we propose a Bidirectional Mamba Module to enhance SSM’s spatial perception ability of extracting spatial features in HSI. Our proposed MamTrans obtains a new state-of-the-art performance across five commonly employed HSI classification benchmarks, demonstrating the robust generalization of MamTrans and effectiveness of SSM-based structure for HSI classification task. Our codes could be found at https://github.com/PPPPPsanG/MamTrans.

MTLSC-Diff: Multitask learning with diffusion models for hyperspectral image super-resolution and classification

Hyperspectral image (HSIs) super-resolution (SR) can improve the spatial resolution of images for subsequent application tasks. In recent years, SR methods based on deep learning have gained widespread attention. However, most of the existing SR methods do not take into account the needs of specific application tasks when designing the network structure. These methods may not be able to efficiently generate high-quality images that satisfy the specific application tasks, leading to degradation of the performance of subsequent application tasks. To solve this problem, we propose a multi-task learning architecture based on the diffusion model, namely MTLSC-Diff. MTLSC-Diff combines the SR network and the classification network in a multi-task learning manner on the basis of the diffusion model. MTLSC-Diff achieves mutual guidance of the two tasks by iterating the image super-resolution and classification tasks, thus gradually reconstructing high-quality images and improving classification accuracy. The guided operations for each time step are performed by the specially designed Mutual-Guidance SR-Classification Synergy Module (M-GSCS). M-GSCS refines the multi-scale image obtained at the previous time step and uses the predicted high spatial resolution image for classification. Meanwhile, a class-guided SR dynamic refinement strategy (C-GSR) is proposed in M-GSCS, which uses multi-scale classification results to guide target scale images to learn new knowledge to further reconstruct high-quality images. Experimental results on relevant datasets show that our method significantly improves the super-resolution performance as well as the classification performance.

A cross-modal feature aggregation and enhancement network for hyperspectral and LiDAR joint classification

Advancements in Earth observation technologies have greatly enhanced the potential of integrating hyperspectral (HS) images with Light Detection and Ranging (LiDAR) data for land use and land cover classification. Despite this, most existing methods primarily focus on employing deep network layers to extract features from two heterogeneous data modalities, often overlooking a gradual modeling data representation approach from shallow to deep layers. Furthermore, excessive network layers can result in the deterioration of modality-specific features, therefore lowering the classification performance. The paper proposed a novel cross-modal feature aggregation and enhancement network for the joint classification of HSI and LiDAR data. Initially, a cross-modal feature fusion module is developed to exploit spatial scale consistency to complete the interchange and fusion of feature embedding at the pixel level, preserving the original information from the two heterogeneous modalities to a certain degree. Then two straightforward strategies (i.e., addition and concatenation) are employed in the shallow network layers before being sent to the transformer encoder. The former facilitates the model’s ability to discern more subtle distinctions and refine spatial location details. The latter ensures the preservation of information integrity, effectively mitigating the risk of feature loss. Moreover, invertible neural networks and a feature enhancement module are introduced, leveraging the complementary information of HSI and LiDAR data to enhance the detail and texture information extracted in deeper layers. Extensive experiments on Houston2013, Trento, and MUUFL datasets demonstrate that the proposed method outperforms several state-of-the-art models in three evaluation metrics, achieving an accuracy improvement of up to 2%. The proposed model brings new inspirations for HSI and LiDAR classification, which is critical for accurate environmental monitoring, urban planning, and precision agriculture. The source code is publicly accessible at https://github.com/zhangyiyan001/CMFAEN.

Disentanglement-inspired single-source domain-generalization network for cross-scene hyperspectral image classification

Cross-scene classification stands as a pivotal frontier in hyperspectral image (HSI) processing, aiming to enhance the generalization capabilities of classification models. However, the diversity of sensor type, shooting environments, and shooting times leads to the spectral heterogeneity problem in HSI. As a result, the same land cover may exhibit varying spectral traits in different domains, posing challenges for cross-scene HSI classification. Drawing inspiration from image disentanglement, we have identified that extracting the latent domain-invariant representation (DIR) of HSI could potentially mitigate the spectral heterogeneity issue. Therefore, we propose a Disentanglement-Inspired Single-Source Domain Generalization Network (DSDGnet) for cross-scene HSI classification in this paper. Firstly, a style transfer module based on a Transformer encoder-transfer-decoder is designed to expand the single source domain to an extended domain. Then, a progressive disentanglement module is proposed to decompose the domain-invariant features and domain-specific features of HSI. Furthermore, a domain combination module is designed to guarantee the accuracy of the progressive disentanglement module and ensure the effectiveness of the domain-invariant feature of HSI. Finally, the domain-invariant features are applied to the classification task, and the domain-specific features are separated to reduce their impact on the generalization ability of classification models. Extensive experiments on three HSI datasets have demonstrated the advanced classification performance of DSDGnet compared to existing domain-generalization methods.

Advanced Global Prototypical Segmentation Framework for Few-Shot Hyperspectral Image Classification.

With the advancement of deep learning, related networks have shown strong performance for Hyperspectral Image (HSI) classification. However, these methods face two main challenges in HSI classification: (1) the inability to capture global information of HSI due to the restriction of patch input and (2) insufficient utilization of information from limited labeled samples. To overcome these challenges, we propose an Advanced Global Prototypical Segmentation (AGPS) framework. Within the AGPS framework, we design a patch-free feature extractor segmentation network (SegNet) based on a fully convolutional network (FCN), which processes the entire HSI to capture global information. To enrich the global information extracted by SegNet, we propose a Fusion of Lateral Connection (FLC) structure that fuses the low-level detailed features of the encoder output with the high-level features of the decoder output. Additionally, we propose an Atrous Spatial Pyramid Pooling-Position Attention (ASPP-PA) module to capture multi-scale spatial positional information. Finally, to explore more valuable information from limited labeled samples, we propose an advanced global prototypical representation learning strategy. Building upon the dual constraints of the global prototypical representation learning strategy, we introduce supervised contrastive learning (CL), which optimizes our network with three different constraints. The experimental results of three public datasets demonstrate that our method outperforms the existing state-of-the-art methods.