Hyperspectral Classification Method Research Articles

Noise-Disruption-Inspired Neural Architecture Search with Spatial–Spectral Attention for Hyperspectral Image Classification

In view of the complexity and diversity of hyperspectral images (HSIs), the classification task has been a major challenge in the field of remote sensing image processing. Hyperspectral classification (HSIC) methods based on neural architecture search (NAS) is a current attractive frontier that not only automatically searches for neural network architectures best suited to the characteristics of HSI data, but also avoids the possible limitations of manual design of neural networks when dealing with new classification tasks. However, the existing NAS-based HSIC methods have the following limitations: (1) the search space lacks efficient convolution operators that can fully extract discriminative spatial–spectral features, and (2) NAS based on traditional differentiable architecture search (DARTS) has performance collapse caused by unfair competition. To overcome these limitations, we proposed a neural architecture search method with receptive field spatial–spectral attention (RFSS-NAS), which is specifically designed to automatically search the optimal architecture for HSIC. Considering the core needs of the model in extracting more discriminative spatial–spectral features, we designed a novel and efficient attention search space. The core component of this innovative space is the receptive field spatial–spectral attention convolution operator, which is capable of precisely focusing on the critical information in the image, thus greatly enhancing the quality of feature extraction. Meanwhile, for the purpose of solving the unfair competition issue in the traditional differentiable architecture search (DARTS) strategy, we skillfully introduce the Noisy-DARTS strategy. The strategy ensures the fairness and efficiency of the search process and effectively avoids the risk of performance crash. In addition, to further improve the robustness of the model and ability to recognize difficult-to-classify samples, we proposed a fusion loss function by combining the advantages of the label smoothing loss and the polynomial expansion perspective loss function, which not only smooths the label distribution and reduces the risk of overfitting, but also effectively handles those difficult-to-classify samples, thus improving the overall classification accuracy. Experiments on three public datasets fully validate the superior performance of RFSS-NAS.

Hyperspectral image classification based on deep separable residual attention network

Hyperspectral image have rich spatial and spectral information, and how to fully extract and utilize the features of these two dimensions is a research hotspot in hyperspectral classification methods. At present, the unique convolutional operation and deep feature extraction structure of convolutional neural network enable them to have stronger feature representation capabilities and achieve good results in hyperspectral image classification. However, CNN methods do not assign different weights based on the importance of features in the feature extraction process, making it difficult to effectively utilize key features, and most importantly, using fixed shaped convolution kernel can easily overlook the differences between hyperspectral image features. A hyperspectral image classification method based on deep separable residual attention network is proposed to address the above issues. Firstly, to reduce the correlation between hyperspectral image data and minimize the interference of redundant information, principal component analysis is used to reduce the dimensionality of hyperspectral image. Secondly, a shallow feature extraction module is constructed, which can dynamically adjust the size of the receptive field according to the actual situation of the image, adaptively extract shallow features, and reduce the loss of original image features. Then, a depthwise separable residual attention mechanism module is proposed, based on which features are extracted. Starting from global and local features, contextual information on image features in channel and spatial domains is extracted. Finally, use a multi-scale feature fusion module to fully integrate feature maps at different scales. Using Indian Pines, Pavia University and Botswana as experimental datasets, the overall classification accuracy of this paper's method is 98.47 %, 98.70 %, 98.83 % with only 50, 50, 30 training samples per class. The Kappa coefficient is 98.25 %, 98.27 %, and 98.73 %, respectively. Compared with advanced methods, this method not only has higher classification accuracy, but also fully utilizes key features at various network levels.

Masked Spatial-Spectral Autoencoders Are Excellent Hyperspectral Defenders.

Deep learning (DL) methodology contributes a lot to the development of hyperspectral image (HSI) analysis community. However, it also makes HSI analysis systems vulnerable to adversarial attacks. To this end, we propose a masked spatial-spectral autoencoder (MSSA) in this article under self-supervised learning theory, for enhancing the robustness of HSI analysis systems. First, a masked sequence attention learning (MSAL) module is conducted to promote the inherent robustness of HSI analysis systems along spectral channel. Then, we develop a graph convolutional network (GCN) with learnable graph structure to establish global pixel-wise combinations. In this way, the attack effect would be dispersed by all the related pixels among each combination, and a better defense performance is achievable in spatial aspect. Finally, to improve the defense transferability and address the problem of limited labeled samples, MSSA employs spectra reconstruction as a pretext task and fits the datasets in a self-supervised manner. Comprehensive experiments over three benchmarks verify the effectiveness of MSSA in comparison with the state-of-the-art hyperspectral classification methods and representative adversarial defense strategies.

Hyperspectral classification method based on M-ResHSDC

Hyperspectral classification method based on M-ResHSDC

Remote Vegetation Diagnostics in Ghana with a Hyperspectral Fluorescence Lidar

Spectral imaging and lidar methods for the characterization of vegetation are vital for understanding plants and, in turn, food security, biodiversity, and vegetation health in a changing climate. While novel hyperspectral imaging, canopy lidar,and solar-induced fluorescence provide details on species, state, structure, and plant physiology, such data come from different instruments. Thus, post-processing and data fusion struggles with synchronization, spatial overlap, and resolution issues. Especially in the tropical regions of sub-Saharan Africa, these complex, expensive, and bulky instruments remain inaccessible. Here, in Ghana, we have built a low-cost, lightweight, and realistic instrument for simultaneously acquiring hyperspectral data of vegetation fluorescence and canopy structure with perfect spatial overlap. In this paper, we demonstrate the application of the hyperspectral fluorescence lidar for diagnostics and species specificity of locally significant crops. We demonstrate simultaneous range and fluorescence measurements of forest canopy, conducted in full sunlight. Our results indicate that the upper side of the leaves shows a more substantial deviation for stressed plants, whilethe lower side shows greater contrast for plant species. This new and simple tool provides a combined method for hyperspectral classification and assessment of the physiological state while also reporting the vegetation height over ground and its diversity.

Lightweight Spatial-Spectral Network Based on 3D-2D Multi-Group Feature Extraction Module for Hyperspectral Image Classification

ABSTRACT Each pixel in the hyperspectral image contains tens to hundreds of narrow bands of detailed spectral information captured by a hyperspectral sensor. Hyperspectral image classification (HSIC) is widely used in remote sensing image analysis. Convolutional neural network (CNN) is one of the most commonly used deep learning methods based on visual data processing. In recent years, hyperspectral classification method based on the convolutional neural network has shown excellent performance. However, single-scale feature extraction may lose some important detail information and cannot guarantee that the best spatial features are captured. Multi-scale character extraction and increasingly deep network structures can improve classification accuracy, but also bring a large number of computational parameters, resulting in higher computational costs. In response to this series of problems, this paper proposes a lightweight spatial-spectral network based on 3D − 2D multi-group feature extraction module (MGFM) for HSCI. In the 3D-MGFM, the input feature maps are grouped and processed in parallel, and then the information of each channel is integrated by point-wise convolution to achieve spatial-spectral feature fusion. We use dilated convolutions with different dilation rate on the convolution kernel to improve the problem of feature information loss, so that we can better learn deep-level spatial-spectral features. In the 2D-MGFM, we selectively emphasize spectral feature information through SENet and extract multi-scale features based on Depthwise Separable Convolution (DSC) grouping. Experimental results on four hyperspectral datasets demonstrate that the proposed method achieves better classification performance than other state-of-the-art methods.

Hyperspectral Image Classification Based on a 3D Octave Convolution and 3D Multiscale Spatial Attention Network

Convolutional neural networks are widely used in the field of hyperspectral image classification. After continuous exploration and research in recent years, convolutional neural networks have achieved good classification performance in the field of hyperspectral image classification. However, we have to face two main challenges that restrict the improvement of hyperspectral classification accuracy, namely, the high dimension of hyperspectral images and the small number of training samples. In order to solve these problems, in this paper, a new hyperspectral classification method is proposed. First, a three-dimensional octave convolution (3D-OCONV) is proposed. Subsequently, a dense connection structure of three-dimensional asymmetric convolution (DC-TAC) is designed. In the spectral branch, the spectral features are extracted through a combination of the 3D-OCONV and spectral attention modules, followed by the DC-TAC. In the spatial branch, a three-dimensional, multiscale spatial attention module (3D-MSSAM) is presented. The spatial information is fully extracted using the 3D-OCONV, 3D-MSSAM, and DC-TAC. Finally, the spectral and spatial information extracted from the two branches is fully fused with an interactive information fusion module. Compared to some state-of-the-art classification methods, the proposed method shows superior classification performance with a small number of training samples on four public datasets.

Spatial-spectral feature extraction of hyperspectral images for wheat seed identification

Hyperspectral images of wheat can identify seeds quickly, accurately, and nondestructively. However, most of the existing hyperspectral classification methods only use spectral information but ignore spatial information, resulting in unsatisfactory classification performance. To address these issues, we propose a spatial-spectral feature extraction method to identify seeds. Specifically, we first fuse the spatial and spectral features and then perform denoising. Subsequently, the principal component analysis is employed to extract features from the spatial-spectral data. Ultimately, the support vector machine trains and optimizes the model. Experimental results demonstrate that our method has the highest classification accuracy compared with the state-of-the-art methods. The classification accuracy of our method is achieved at 97.64% on the whole dataset. In addition, our method achieves better classification performance for small sample data.

A Novel Method for Hyperspectral Mineral Mapping Based on Clustering-Matching and Nonnegative Matrix Factorization

The emergence of hyperspectral imagery paved a new way for rapid mineral mapping. As a classical hyperspectral classification method, spectral matching (SM) can automatically map the spatial distribution of minerals without the need for selecting training samples. However, due to the influence of noise, the mapping accuracy of SM is usually poor, and its per-pixel matching method is inefficient to some extent. To solve these problems, we propose an unsupervised clustering-matching mapping method, using a combination of k-means and SM (KSM). First, nonnegative matrix factorization (NMF) is used and combined with a simple and effective NMF initialization method (SMNMF) for feature extraction. Then, k-means is implemented to get the cluster centers of the extracted features and band depth, which are used for clustering and matching, respectively. Finally, dimensionless matching methods, including spectral angle mapper (SAM), spectral correlation angle (SCA), spectral gradient angle (SGA), and a combined matching method (SCGA) are used to match the cluster centers of band depth with a spectral library to obtain the mineral mapping results. A case study on the airborne hyperspectral image of Cuprite, Nevada, USA, demonstrated that the average overall accuracies of KSM based on SAM, SCA, SGA, and SCGA are approximately 22%, 22%, 35%, and 33% higher than those of SM, respectively, and KSM can save more than 95% of the mapping time. Moreover, the mapping accuracy and efficiency of SMNMF are about 15% and 38% higher than those of the widely used NMF initialization method. In addition, the proposed SCGA could achieve promising mapping results at both high and low signal-to-noise ratios compared with other matching methods. The mapping method proposed in this study provides a new solution for the rapid and autonomous identification of minerals and other fine objects.

Multiscale Information Fusion for Hyperspectral Image Classification Based on Hybrid 2D-3D CNN

Hyperspectral images are widely used for classification due to its rich spectral information along with spatial information. To process the high dimensionality and high nonlinearity of hyperspectral images, deep learning methods based on convolutional neural network (CNN) are widely used in hyperspectral classification applications. However, most CNN structures are stacked vertically in addition to using a onefold size of convolutional kernels or pooling layers, which cannot fully mine the multiscale information on the hyperspectral images. When such networks meet the practical challenge of a limited labeled hyperspectral image dataset—i.e., “small sample problem”—the classification accuracy and generalization ability would be limited. In this paper, to tackle the small sample problem, we apply the semantic segmentation function to the pixel-level hyperspectral classification due to their comparability. A lightweight, multiscale squeeze-and-excitation pyramid pooling network (MSPN) is proposed. It consists of a multiscale 3D CNN module, a squeezing and excitation module, and a pyramid pooling module with 2D CNN. Such a hybrid 2D-3D-CNN MSPN framework can learn and fuse deeper hierarchical spatial–spectral features with fewer training samples. The proposed MSPN was tested on three publicly available hyperspectral classification datasets: Indian Pine, Salinas, and Pavia University. Using 5%, 0.5%, and 0.5% training samples of the three datasets, the classification accuracies of the MSPN were 96.09%, 97%, and 96.56%, respectively. In addition, we also selected the latest dataset with higher spatial resolution, named WHU-Hi-LongKou, as the challenge object. Using only 0.1% of the training samples, we could achieve a 97.31% classification accuracy, which is far superior to the state-of-the-art hyperspectral classification methods.

Deep Metric Learning with Online Hard Mining for Hyperspectral Classification

Recently, deep learning has developed rapidly, while it has also been quite successfully applied in the field of hyperspectral classification. Generally, training the parameters of a deep neural network to the best is the core step of a deep learning-based method, which usually requires a large number of labeled samples. However, in remote sensing analysis tasks, we only have limited labeled data because of the high cost of their collection. Therefore, in this paper, we propose a deep metric learning with online hard mining (DMLOHM) method for hyperspectral classification, which can maximize the inter-class distance and minimize the intra-class distance, utilizing a convolutional neural network (CNN) as an embedded network. First of all, we utilized the triplet network to learn better representations of raw data so that raw data were capable of having their dimensionality reduced. Afterward, an online hard mining method was used to mine the most valuable information from the limited hyperspectral data. To verify the performance of the proposed DMLOHM, we utilized three well-known hyperspectral datasets: Salinas Scene, Pavia University, and HyRANK for verification. Compared with CNN and DMLTN, the experimental results showed that the proposed method improved the classification accuracy from 0.13% to 4.03% with 85 labeled samples per class.

A Neural Network Method for Classification of Sunlit and Shaded Components of Wheat Canopies in the Field Using High-Resolution Hyperspectral Imagery

(1) Background: Information rich hyperspectral sensing, together with robust image analysis, is providing new research pathways in plant phenotyping. This combination facilitates the acquisition of spectral signatures of individual plant organs as well as providing detailed information about the physiological status of plants. Despite the advances in hyperspectral technology in field-based plant phenotyping, little is known about the characteristic spectral signatures of shaded and sunlit components in wheat canopies. Non-imaging hyperspectral sensors cannot provide spatial information; thus, they are not able to distinguish the spectral reflectance differences between canopy components. On the other hand, the rapid development of high-resolution imaging spectroscopy sensors opens new opportunities to investigate the reflectance spectra of individual plant organs which lead to the understanding of canopy biophysical and chemical characteristics. (2) Method: This study reports the development of a computer vision pipeline to analyze ground-acquired imaging spectrometry with high spatial and spectral resolutions for plant phenotyping. The work focuses on the critical steps in the image analysis pipeline from pre-processing to the classification of hyperspectral images. In this paper, two convolutional neural networks (CNN) are employed to automatically map wheat canopy components in shaded and sunlit regions and to determine their specific spectral signatures. The first method uses pixel vectors of the full spectral features as inputs to the CNN model and the second method integrates the dimension reduction technique known as linear discriminate analysis (LDA) along with the CNN to increase the feature discrimination and improves computational efficiency. (3) Results: The proposed technique alleviates the limitations and lack of separability inherent in existing pre-defined hyperspectral classification methods. It optimizes the use of hyperspectral imaging and ensures that the data provide information about the spectral characteristics of the targeted plant organs, rather than the background. We demonstrated that high-resolution hyperspectral imagery along with the proposed CNN model can be powerful tools for characterizing sunlit and shaded components of wheat canopies in the field. The presented method will provide significant advances in the determination and relevance of spectral properties of shaded and sunlit canopy components under natural light conditions.

Densely Connected Multiscale Attention Network for Hyperspectral Image Classification

Hyperspectral images (HSIs) are characterized by high spatial resolution and are rich in spectral information. In the process of HSI classification, the extraction of spectral-spatial features directly influences the classification results. In recent years, the hyperspectral classification method based on convolutional neural networks has demonstrated excellent performance. However, as the network structure deepens, degradation occurs, and the features learned from the fixed-scale convolutional kernels are usually specific, which is not conducive to feature learning and thus impairs the classification accuracy. To solve the problem of difficult extraction of features and underutilization of information from HSI data, a densely connected multiscale attention network based on 3-D convolution is proposed for HSI classification. First, to reduce the spectral redundancy of the HSIs, the principal component analysis algorithm is performed on the raw HSI data; then, several multiscale blocks comprised of parallel factorized spatial-spectral convolution modules of different sizes are adopted to extract the enriched spectral-spatial features from HSIs; furthermore, dense connections are introduced to further fuse features obtained from blocks of different depths, thereby enhancing feature reuse and propagation and helping to alleviate the problem of vanishing gradients. Besides, the channel-spectral-spatial attention block is put forward to spontaneously reweight the fused features to emphasize the features that are more relevant to the classification results while weakening the less relevant ones. The experimental results show that the proposed method is effective in extracting discriminative features of the target and outperforms the other state-of-the-art methods.

WHU-Hi: UAV-borne hyperspectral with high spatial resolution (H2) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF

Unmanned aerial vehicle (UAV)-borne hyperspectral systems can acquire hyperspectral imagery with a high spatial resolution (which we refer to here as H2 imagery). As a result of the low operating cost, high flexibility, and the ability to achieve real-time data acquisition, UAV-borne hyperspectral systems have become an important data source for remote sensing based agricultural monitoring. However, precise crop classification based on UAV-borne H2 imagery is a challenging task when faced with a number of different crop classes. The traditional hyperspectral classification methods, such as the spectral-based and object-oriented classification methods, have difficulty in classifying H2 imagery, faced with the problems of salt-and-pepper (SP) noise and scale selection. In this article, the deep convolutional neural network with a conditional random field classifier (CNNCRF) framework is proposed for precise crop classification with UAV-borne H2 imagery. In the proposed method, a deep convolutional neural network (CNN) is designed to extract and fuse in-depth spectral and local spatial features, and the conditional random field (CRF) model further incorporates the spatial-contextual information to improve the problem of holes and isolated regions in the classification map. Meanwhile, virtual sample augmentation based on the hyperspectral imaging mechanism is used to lessen the issue of the limited labeled samples. To validate the results, a new dataset—the Wuhan UAV-borne hyperspectral image (WHU-Hi) dataset—has been built for precise crop classification. The experimental results obtained using the WHU-Hi dataset confirm the accuracy and visualization performance of the proposed CNNCRF classification method, which outperforms the previous methods. In addition, the WHU-Hi dataset could serve as a benchmark dataset for hyperspectral image classification studies.

Semisupervised hyperspectral imagery classification based on a three-dimensional convolutional adversarial autoencoder model with low sample requirements

Although there are many state-of-the-art methods for hyperspectral classification, data deficiency is a problem that should be addressed before popularizing hyperspectral technology. To solve this problem, it is worth exploring methods based on small datasets. Inspired by the advanced deep learning classification methods and the autoencoder structure, we propose a structure named three-dimensional convolutional adversarial autoencoder that combines the two processes for semisupervised hyperspectral classification. Our experiments show its utility in data-deficient situations, and our study analyzes its advantages and disadvantages, and points out a probable direction toward optimization.

Spatial information-enhanced hyperspectral imagery classification based on joint spatial-aware collaborative representation

To address the insufficiency of texture information-based classification features to classify samples, we proposed two methods for spatial information-enhanced hyperspectral imagery classification based on joint spatial-aware collaborative representation (JSaCR). First, we introduce a texture regularized-based joint spatial-aware collaborative representation (TRJSaCR) method, in which prior texture is regarded as a regularization term to constrain the coefficient of the objection function of JSaCR and the closed-form solution is obtained to reconstruct the test sample. Second is a spatial information-assisted discrimination rules (SIDR) method coupled with TRJSaCR (TRJSaCR-SIDR) for classification. More precisely, the label information of the test samples and their corresponding neighborhoods are specified by TRJSaCR-SIDR, and the final labels are determined by considering their neighborhood label distribution. Our work aims to broaden the knowledge of the utilization of spatial information in hyperspectral classification. Experimental results on two benchmark hyperspectral datasets, Indian Pines and Pavia University, indicate that the proposed algorithms are superior to other state-of-the-art classifiers.

An innovative multi-kernel learning algorithm for hyperspectral classification

Many studies show that support vector machine (SVM) techniques have gotten superior performance in hyperspectral classification. SVM classifier’s improvement was fulfilled by the combination of multiple kernel tricks. Multi-kernel learning (MKL) algorithms have been used in hyperspectral processing. Combining SVM and MKL, we propose a novel method for hyperspectral classification. It adopts a boost algorithm rather than the global optimization method to search for the optimal combination of kernel SVMs. Experiments were carried out on several reflectance spectra of plankton and two hyperspectral remote sensing images to verify the approach. It turns out that the proposed method achieved better classification performance than some conventional algorithms.

Hyperspectral image classification by combining local binary pattern and joint sparse representation

ABSTRACTThis paper proposes a new hyperspectral classification method that combines the joint sparse representation classifier (JSRC) and local binary pattern (LBP) (JSR-LBP). The proposed method uses a statistical histogram of the LBP feature spectrum as a feature vector for classification and recognition. Using the LBP method let us extract the local texture feature of an image more accurately and effectively because of the forceful robustness to light. The proposed method JSR-LBP mainly includes the following steps: First, the JSRC is used to obtain the representation residuals of different pixels. Then, the LBP value is calculated for every pixel in the whole image to generate the LBP histogram; the LBP histogram of the test sample can be obtained from a series of binary codes that are generated using statistics. Next, the Bhattacharyya coefficient is employed to measure the similarity between the test and training samples unlike the traditional classifiers as -nearest neighbor that usually employ the Euclidean distance as similarity metric, and a regularization parameter is then introduced to achieve the balance between the JSRC and LBP. Finally, the test pixel’s label is determined by applying the final residual. Experimental results performed on three real hyperspectral images data demonstrate the outstanding performance of the proposed approach compared to other broader classifiers.

Probabilistic rough-set-based band selection method for hyperspectral data classification

This paper proposes an innovative band selection algorithm called probabilistic rough set-based band selection (PRSBS) algorithm. The proposed PRSBS is a supervised band selection algorithm with efficiency for it only needs to calculate the first-order significance measure. The main novelty of the proposed PRSBS algorithm lies in the criterion function which measures the effectiveness of considered band. The PRSBS algorithm uses a probabilistic distribution dependency as the relevance measure between the bands and class labels, which can effectively measure the uncertainty in both the positive and the boundary samples in a dataset. We compared the proposed PRSBS with the most relevant band selection algorithm RSBS on three different hyperspectral datasets, the experimental results show that the PRSBS has better results than the RSBS. Moreover, the PRSBS algorithm runs significantly faster than the RSBS algorithm, which makes it a proper choice for band selection in hyperspectral image dataset.

Hyperspectral Image Classification via Superpixel Spectral Metrics Representation

This letter proposes a new hyperspectral classification method that fuses superpixel spectral metrics and joint sparse representation (JSR), which is termed as superpixel spectral metrics representation (SSMR). Recently, superpixel segmentation has proven to be a powerful tool to exploit the spatial information of hyperspectral images (HSIs), since the size and shape of each superpixel can be adaptively changed in different structural textures. Moreover, spectral information divergence (SID) has superiority compared to other distance-based similarity measures, particularly when using with a JSR classifier. Taking the aforemementioned advantages into account, superpixel segmentation, SID, and JSR are availably combined to effectively utilize the spectralspatial information of the HSI. The proposed SSMR method includes the following main steps. First, superpixel segmentation is utilized to divide the original map into several superpixels. Second, similarity metric SID among test samples in all superpixels and training samples are calculated. Next, the JSR model is employed to obtain the reconstruction residuals of each class. Then, a regularization parameter λ is introduced to attain balance between JSR and SID. Finally, pixel's label is determined by the minimal total residual. Experimental results on the Indian Pines dataset show better performance than several well-known classification methods.