Spectral-spatial Classification Research Articles

Superpixel‐guided locality preserving projection and spatial–spectral classification for hyperspectral image

Superpixel‐guided locality preserving projection and spatial–spectral classification for hyperspectral image

Multi-scale superpixel-based nearest subspace classifier for mucilage detection from hyperspectral PRISMA data

Hyperspectral imaging has a critical role in observing the Earth's surface, providing images with rich spectral information. As a result, it has become an essential tool for monitoring and addressing environmental issues such as pollution, water balance, and climate change. Among these challenges, marine pollution poses a significant ecological threat, as evidenced by the recent mucilage incident in Turkey in April 2021. Mucilage, a dense mucus-like substance resulting from the overgrowth of marine plankton and microorganisms, detrimentally affects marine ecosystems when its levels surge. Traditional mucilage detection involves labour-intensive sampling and laboratory analysis, rendering it impractical for vast marine environments. Remote sensing techniques offer a promising alternative for automated mucilage detection from satellite imagery. In this study, we propose a multi-scale superpixel-based classification approach for mucilage detection, using the PRISMA satellite's hyperspectral mucilage datasets captured over the Sea of Marmara. Our method performs classification by analyzing the closeness between the subspace spanned by the samples in the superpixel and the subspace spanned by the within-class training samples. We formulate the solution to this approach via a Canonical Correlation Analysis framework and show that our proposed method outperforms its competitors, especially when the training data is limited. Our comprehensive experiments include assessing the transferability of classification methods on datasets collected from different locations and at different times. We compared the performance of our proposed method with state-of-the-art spectral-spatial classification methods, including deep learning-based random patches network (RPNet), structural-kernel collaborative representation (SKCR), and window based nearest subspace classifier (WBNSC). The results show that the proposed method outperforms existing techniques in most cases. The datasets and source code are publicly available on GitHub at https://github.com/kgtoker/MSBNSC, enabling more comprehensive access and collaboration in environmental monitoring efforts.

Hyperspectral image classification based on superpixel merging and broad learning system

AbstractMost spectral–spatial classification methods for hyperspectral images (HSIs) can achieve satisfactory classification results. However, the common problem faced with these approaches is the need for a long training time and sufficient training samples. To address this issue, this study proposes an effective spectral–spatial HSI classification method based on superpixel merging, superpixel smoothing and broad learning system (SMS‐BLS). The newly introduced parameter‐free superpixel merging technique based on local modularity not only enhances the role of local spatial information in classification, but also maintains class boundary information as much as possible. In addition, the spectral and spatial information of HSIs is further fused during the superpixel smoothing process. As a result, with limited training samples, using merged and smoothed superpixels instead of pixels as input to the broad learning system significantly improves its classification performance. Moreover, the merged superpixels weaken the dependence of the classification results on the superpixel segmentation scale. The effectiveness of the proposed method was validated on three HSI benchmarks, namely Indian Pines, Pavia University and Salinas. Experimental and comparative results show the superiority of the method to other state‐of‐the‐art approaches in terms of overall accuracy and running time.

Hyperspectral Image Classification Based on Mutually Guided Image Filtering

Hyperspectral remote sensing images (HSIs) have both spectral and spatial characteristics. The adept exploitation of these attributes is central to enhancing the classification accuracy of HSIs. In order to effectively utilize spatial and spectral features to classify HSIs, this paper proposes a method for the spatial feature extraction of HSIs based on a mutually guided image filter (muGIF) and combined with the band-distance-grouped principal component. Firstly, aiming at the problem that previously guided image filtering cannot effectively deal with the inconsistent information structure between the guided and target information, a method for extracting spatial features using muGIF is proposed. Then, aiming at the problem of the information loss caused by a single principal component as a guided image in the traditional GIF-based spatial–spectral classification, a spatial feature-extraction framework based on the band-distance-grouped principal component is proposed. The method groups the bands according to the band distance and extracts the principal components of each set of band subsets as the guide map of the current band subset to filter the HSIs. A deep convolutional neural network model and a generative adversarial network model for the filtered HSIs are constructed and then trained using samples for HSIs’ spatial–spectral classification. Experiments show that compared with the traditional methods and several popular spatial–spectral HSI classification methods based on a filter, the proposed methods based on muGIF can effectively extract the spatial–spectral features and improve the classification accuracy of HSIs.

Spectral–Spatial Classification With Naive Bayes and Adaptive FFT for Improved Classification Accuracy of Hyperspectral Images

Spectral–Spatial Classification With Naive Bayes and Adaptive FFT for Improved Classification Accuracy of Hyperspectral Images

Active Learning-Based Spectral–Spatial Classification for Discriminating Tree Species in Hyperspectral Images

Active Learning-Based Spectral–Spatial Classification for Discriminating Tree Species in Hyperspectral Images

An Adaptive Atrous Spatial Pyramid Pooling Network for Hyperspectral Classification

Hyperspectral imaging (HSI) offers rich spectral and spatial data, beneficial for a variety of applications. However, challenges persist in HSI classification due to spectral variability, non-linearity, limited samples, and a dearth of spatial information in conventional spectral classifiers. While various spectral–spatial classifiers and dimension reduction techniques have been developed to mitigate these issues, they are often constrained by the utilization of handcrafted features. Deep learning has been introduced to HSI classification, with pixel- and patch-level deep learning (DL) classifiers gaining substantial attention. Yet, existing patch-level DL classifiers encounter difficulties in concentrating on long-distance dependencies and managing category areas of diverse sizes. The proposed Self-Adaptive 3D atrous spatial pyramid pooling (ASPP) Multi-Scale Feature Fusion Network (SAAFN) addresses these challenges by simultaneously preserving high-resolution spatial detail data and high-level semantic information. This method integrates a modified hyperspectral superpixel segmentation technique, a multi-scale 3D ASPP convolution block, and an end-to-end framework to extract and fuse multi-scale features at a self-adaptive rate for HSI classification. This method significantly enhances the classification accuracy of HSI with limited samples.

A Spatial-Spectral Classification Method Based on Deep Learning for Controlling Pelagic Fish Landings in Chile.

Fishing has provided mankind with a protein-rich source of food and labor, allowing for the development of an important industry, which has led to the overexploitation of most targeted fish species. The sustainable management of these natural resources requires effective control of fish landings and, therefore, an accurate calculation of fishing quotas. This work proposes a deep learning-based spatial-spectral method to classify five pelagic species of interest for the Chilean fishing industry, including the targeted Engraulis ringens, Merluccius gayi, and Strangomera bentincki and non-targeted Normanichthtys crockeri and Stromateus stellatus fish species. This proof-of-concept method is composed of two channels of a convolutional neural network (CNN) architecture that processes the Red-Green-Blue (RGB) images and the visible and near-infrared (VIS-NIR) reflectance spectra of each species. The classification results of the CNN model achieved over 94% in all performance metrics, outperforming other state-of-the-art techniques. These results support the potential use of the proposed method to automatically monitor fish landings and, therefore, ensure compliance with the established fishing quotas.

NSCKL: Normalized Spectral Clustering With Kernel-Based Learning for Semisupervised Hyperspectral Image Classification.

Spatial-spectral classification (SSC) has become a trend for hyperspectral image (HSI) classification. However, most SSC methods mainly consider local information, so that some correlations may not be effectively discovered when they appear in regions that are not contiguous. Although many SSC methods can acquire spatial-contextual characteristics via spatial filtering, they lack the ability to consider correlations in non-Euclidean spaces. To address the aforementioned issues, we develop a new semisupervised HSI classification approach based on normalized spectral clustering with kernel-based learning (NSCKL), which can aggregate local-to-global correlations to achieve a distinguishable embedding to improve HSI classification performance. In this work, we propose a normalized spectral clustering (NSC) scheme that can learn new features under a manifold assumption. Specifically, we first design a kernel-based iterative filter (KIF) to establish vertices of the undirected graph, aiming to assign initial connections to the nodes associated with pixels. The NSC first gathers local correlations in the Euclidean space and then captures global correlations in the manifold. Even though homogeneous pixels are distributed in noncontiguous regions, our NSC can still aggregate correlations to generate new (clustered) features. Finally, the clustered features and a kernel-based extreme learning machine (KELM) are employed to achieve the semisupervised classification. The effectiveness of our NSCKL is evaluated by using several HSIs. When compared with other state-of-the-art (SOTA) classification approaches, our newly proposed NSCKL demonstrates very competitive performance. The codes will be available at https://github.com/yuanchaosu/TCYB-nsckl.

A spatial–spectral classification framework for multispectral LiDAR

ABSTRACT Precise classification of Light Detection and Ranging (LiDAR) point cloud is a fundamental process in various applications, such as land cover mapping, forestry management, and autonomous driving. Due to the lack of spectral information, the existing research on single wavelength LiDAR classification is limited. Spectral information from images could address this limitation, but data fusion suffers from varying illumination conditions and the registration problem. A novel multispectral LiDAR successfully obtains spatial and spectral information as a brand-new data type, namely, multispectral point cloud, thereby improving classification performance. However, spatial and spectral information of multispectral LiDAR has been processed separately in previous studies, thereby possibly limiting the classification performance of multispectral LiDAR. To explore the potential of this new data type, the current spatial–spectral classification framework for multispectral LiDAR that includes four steps: (1) neighborhood selection, (2) feature extraction and selection, (3) classification, and (4) label smoothing. Three novel highlights were proposed in this spatial – spectral classification framework. (1) We improved the popular eigen entropy-based neighborhood selection by spectral angle match to extract a more precise neighborhood. (2) We evaluated the importance of geometric and spectral features to compare their contributions and selected the most important features to reduce feature redundancy. (3) We conducted spatial label smoothing by a conditional random field, accounting for the spatial and spectral information of the neighborhood points. The proposed method demonstrated by a multispectral LiDAR with three channels: 466 nm (blue), 527 nm (green), and 628 nm (red). Experimental results demonstrate the effectiveness of the proposed spatial – spectral classification framework. Moreover, this research takes advantages of the complementation of spatial and spectral information, which could benefit more precise neighborhood selection, more effective features, and satisfactory refinement of classification result. Finally, this study could serve as an inspiration for future efficient spatial–spectral process for multispectral point cloud.

Ensemble of Support Vector Machines for spectral-spatial classification of hyperspectral and multispectral images

Ensemble of Support Vector Machines for spectral-spatial classification of hyperspectral and multispectral images

DGSSC: A Deep Generative Spectral-Spatial Classifier for Imbalanced Hyperspectral Imagery

In recent years, hyperspectral image classification (HSIC) has achieved impressive progress with emerging studies on deep learning models. However, the classification performance downgrades due to the limited number of annotated samples, especially for minority classes. Notably, the imbalanced data dilemma is familiar in remote sensing hyperspectral image because the ground objects are commonly distributed without evenness. Therefore, this paper proposes a novel deep generative spectral-spatial classifier (DGSSC) for addressing the issues of imbalanced HSIC. Specifically, the DGSSC comprises three components, a two-stage encoder, a decoder, and a classifier, which are trained in an end-to-end manner. In particular, to exploit the abundant spectral-spatial features with relatively low computational complexity, the first stage of the encoder comprises successive three-dimensional (3D) and two-dimensional (2D) convolutions, exploring the spectral-spatial and deep spatial information. In addition, the second stage involves the deep latent variable model to achieve minority-class data augmentation. Furthermore, a patch distance-based reconstruction loss function is designed to facilitate the outputs of the decoder being more similar to the input 3D patch samples. The proposed DGSSC can outperform the state-of-the-art methods on three benchmark datasets, especially with its more robust prediction results. For instance, the DGSSC achieves a remarkable 97.85% mean overall accuracy with 0.24% standard deviation over ten independent runs with randomly selected imbalanced 1% training samples on the University of Pavia dataset.

A novel meta-learning-based hyperspectral image classification algorithm

Aimed at the hyperspectral image (HSI) classification under the condition of limited samples, this paper designs a joint spectral–spatial classification network based on metric meta-learning. First, in order to fully extract HSI fine features, the squeeze and excitation (SE) attention mechanism is introduced into the spectrum dimensional channel to selectively extract useful HSI features to improve the sensitivity of the network to information features. Second, in the part of spatial feature extraction, the VGG16 model parameters trained in advance on the HSRS-SC dataset are used to realize the transfer and learning of spatial feature knowledge, and then, the higher-level abstract features are extracted to mine the intrinsic attributes of ground objects. Finally, the gated feature fusion strategy is introduced to connect the extracted spectral and spatial feature information on HSI for mining more abundant feature information. In this paper, a large number of experiments are carried out on the public hyperspectral dataset, including Pavia University and Salinas. The results show that the meta-learning method can achieve fast learning of new categories with only a small number of labeled samples and has good generalization ability for different HSI datasets.

Machine Learning and Deep Learning Techniques for Spectral Spatial Classification of Hyperspectral Images: A Comprehensive Survey

The growth of Hyperspectral Image (HSI) analysis is due to technology advancements that enable cameras to collect hundreds of continuous spectral information of each pixel in an image. HSI classification is challenging due to the large number of redundant spectral bands, limited training samples and non-linear relationship between the collected spatial position and the spectral bands. Our survey highlights recent research in HSI classification using traditional Machine Learning techniques like kernel-based learning, Support Vector Machines, Dimension Reduction and Transform-based techniques. Our study also digs into Deep Learning (DL) techniques that involve the usage of Autoencoders, 1D, 2D and 3D-Convolutional Neural Networks to classify HSI. From the comparison, it is observed that DL-based classification techniques outperform ML-based techniques. It has also been observed that spectral-spatial HSI classification outperforms pixel-by-pixel classification because it incorporates spectral signatures and spatial domain information. The performance of ML and DL-based classification techniques has been reviewed on commonly used land cover datasets like Indian Pines, Salinas valley and Pavia University.

AAtt-CNN: Automatic Attention-Based Convolutional Neural Networks for Hyperspectral Image Classification

Convolutional models have provided outstanding performance in the analysis of hyperspectral images (HSIs). These architectures are carefully designed to extract intricate information from non-linear features for classification tasks. Notwithstanding their results, model architectures are manually engineered and further optimized for generalized feature extraction. In general terms, deep architectures are time consuming for complex scenarios since they require fine tuning. Neural architecture search (NAS) has emerged as a suitable approach to tackle this shortcoming. In parallel, modern attention-based methods have boosted the recognition of sophisticated features. The search for optimal neural architectures combined with attention procedures motivates the development of this work. This paper develops a new method to automatically design and optimize convolutional neural networks (CNNs) for HSI classification using channel-based attention mechanisms. Specifically, one-dimensional (1D) and spectral-spatial (3D) classifiers are considered to handle the large amount of information contained in HSIs from different perspectives. Furthermore, the proposed AAtt-CNN method meets the requirement to lower the large computational overheads associated with architectural search. It is compared with current state-of-the-art (SOTA) classifiers. Our experiments, conducted using a wide range of HSI images, demonstrate that AAtt-CNN succeeds in finding optimal architectures for classification, leading to SOTA results.

Grouped Multi-Attention Network for Hyperspectral Image Spectral-Spatial Classification

Deep learning has been a powerful tool for hyperspectral image (HSI) classification. However, it is still an open issue to effectively learn highly discriminative features from the HSI, due to the high-dimensionality and complex spectral-spatial characteristics. To settle this issue, we propose a new band-grouping guided multi-attention module for the performance promotion of spectral-spatial feature learning. First, based on the fact of high relevance between adjacent spectral bands and low dependencies across long-range ones, all the spectral bands are adaptively divided into multiple non-overlapping groups where relevant bands are included. The advantage is to reduce the spectral dimension and data complexity when processing and analyzing each group. Then, a multi-attention mechanism, which not only explore the intra-group salient information but also propagate the inter-group difference information, is embedded into the convolutional neural networks to learn group-specific spectral-spatial features. By emphasizing useful spectral/spatial information and squeezing useless information with attention mechanism, the severability of learned features is enhanced. Based on this module, a spectral-spatial classification network is built, named by grouped multi-attention network (GMA-Net). The GMA-Net contains a two-branch architecture, i.e., pixel-wise spectral feature learning and patch-wise spectral-spatial feature learning. Via fusing the features from two branches, the complementary and discriminative features provided by pixel-wise and patch-wise learning manner can be integrated to further boost classification performance. Experimental results demonstrate that the proposed method is superior than several state-of-the-art approaches. Codes are available at: https://github.com/luting-hnu.

Spectral-spatial classification of hyperspectral images by combining hierarchical and marker-based Minimum Spanning Forest algorithms

Spectral-spatial classification of hyperspectral images by combining hierarchical and marker-based Minimum Spanning Forest algorithms

Iterative Random Training Sampling Convolutional Neural Network for Hyperspectral Image Classification

Convolution neural network (CNN) has received considerable interest in hyperspectral image classification (HSIC) lately due to its excellent spectral-spatial feature extraction capability. To improve CNN, many approaches have been directed to exploring the infrastructure of its network by introducing different paradigms. This paper takes a rather different approach by developing an iterative CNN which extends a CNN by including a feedback system to repeatedly process the same CNN in an iterative manner. Its idea is to take advantage of a recently developed iterative training sampling spectral-spatial classification (IRTS-SSC) that allows CNN to update its spatial information of classification maps through a feedback spatial filtering system via IRTS. The resulting CNN is called iterative random training sampling CNN (IRTS-CNN) with several unique features. First, IRTS-CNN combines CNN and IRTS-SSC into one paradigm, an architecture which has never investigated in the past. Second, it implements a series of spatial filters to capture spatial information of classified data samples and further feeds this information back via an iterative process to expand the current input data cube for the next iteration. Third, it utilizes the expanded data cube to randomly re-select training samples and then to re-implement CNN iteratively. Last but not least, IRTS-CNN provides a general framework which can implement any arbitrary CNN as an initial classifier to improve its performance through an iterative process. Extensive experiments are conducted to demonstrate that IRTS-CNN indeed significantly improves CNN, specifically, when only a small size of limited training samples is used.

Spatial-spectral classification of hyperspectral remote sensing images using 3D CNN based LeNet-5 architecture

Hyperspectral remote sensing image (HRSI) analysis are commonly used in a wide variety of remote sensing applications such as land cover analysis, military surveillance, object detection and precision agriculture. Deep learning is seen as an effective method to improve HRSI classification. In particular, convolutional neural networks (CNNs) are increasingly used in this field. The high dimensionality of the HRSIs increases the computational complexity. Thus, most of studies apply dimension reduction as preprocessing. Another problem in HRSI classification is that spatial-spectral features must be considered in order to obtain accurate results. Because, HRSI classification results are highly dependent on spatiospectral information. The aim of this paper is to build a 3D CNN-based LeNet-5 method for HRSI classification. Principal component analysis (PCA) is used as the pre-processing step for optimum spectral band extraction. 3D CNN is used to simultaneously extract spatial-spectral features in HRSIs. LeNet-5 architecture has a simple and straightforward architecture. At the same time, the number of trainable parameters is very low. With the use of the LeNet-5 architecture, the number of trainable parameters of the proposed method is considerably reduced. This is one of the most important features that distinguish the proposed method from other deep learning methods. The proposed method is tested with Indian pines, Pavia University and Salinas datasets. As a result of experimental studies, 100% overall accuracy result is obtained in all datasets. The proposed 3DLeNet method is compared against various state-of-the-art CNN based methods. From the experimental results, it is seen that our 3DLeNet method performs more accurate result. It has also been found that the proposed 3DLeNet method shows a satisfactory result with less computational complexity.

Spectral-spatial classification of hyperspectral images based on non-linear principal component analysis and deep learning models

ABSTRACT Hyperspectral sensors, due to the acquisition of a large number of spectral bands, always have particular importance in monitoring the phenomena of the earth’s surface. The classification of hyperspectral images is the most crucial method of processing hyperspectral data, and many efforts have been made to increase its accuracy. In recent years, convolutional neural networks (CNNs) and spatial features have been a big part of how well hyperspectral images can be classified. CNNs, due to the automatic generation of features and reduction of parameters, compared to multilayer perceptron neural networks by sharing parameters in each layer, have received much attention from researchers in the field of pattern recognition. In previous research, much attention has not been paid to the simultaneous use of spatial feature extraction methods in CNNs. For this reason, in this paper, a new CNNs architecture has been introduced to classify hyperspectral images, which uses the spectral-spatial vector obtained from the morphological profiles method as the network’s input. Morphological profiles, which include a set of opening and closing filters, are generally applied to the components of the image that contain the most information. In this research, non-linear principal component analysis (non-linear PCA) implemented by neural networks was used to summarize hyperspectral image information. Therefore, the main innovation of this paper is to provide a three-stage framework for applying deep learning. The first step is to reduce the dimensions of the hyperspectral image using non-linear PCA, the second stage is to extract spatial features using the morphological profiles method, and the third step is to prepare the inputs and design the CNNs architecture. The proposed method was evaluated on three benchmark hyperspectral images of Pavia, Berlin and Telops. The results of the experiments show the superiority of the proposed method by 13, 11 and 17% in the overall accuracy parameter compared to the support vector machines (SVM) classification algorithm in the Pavia, Berlin and Telops images, respectively.