Spectral Information Of Hyperspectral Images Research Articles

Unsupervised hyperspectral images classification using hypergraph convolutional extreme learning machines

AbstractAiming at the problem that traditional methods are difficult to fully utilize the rich spectral information in hyperspectral images (HSI) and fail to capture the complex higher‐order relations in hyperspectral data, which leads to limited classification performance extreme learning machine and fails to further improve the classification accuracy of HSIs, the authors propose the hypergraph convolutional extreme learning machine (HGCELM) method. The method not only inherits all the advantages of extreme learning machine (ELM), but also embeds hypergraph convolution for feature selection, which is capable of handling higher‐order relations. This enables HGCELM to capture more complex relationships between nodes and provide richer representation capabilities. At the same time, the training speed advantage of ELM is retained, thus speeding up the model training process. Experimental results show that the proposed algorithm achieves better accuracy compared to other clustering algorithms.

Enhanced deep unrolling networks for snapshot compressive hyperspectral imaging

Snapshot compressive hyperspectral imaging necessitates the reconstruction of a complete hyperspectral image from its compressive snapshot measurement, presenting a challenging inverse problem. This paper proposes an enhanced deep unrolling neural network, called EDUNet, to tackle this problem. The EDUNet is constructed via the deep unrolling of a proximal gradient descent algorithm and introduces two innovative modules for gradient-driven update and proximal mapping reflectivity. The gradient-driven update module leverages a memory-assistant descent approach inspired by momentum-based acceleration techniques, for enhancing the unrolled reconstruction process and improving convergence. The proximal mapping is modeled by a sub-network with a cross-stage spectral self-attention, which effectively exploits the inherent self-similarities present in hyperspectral images along the spectral axis. It also enhances feature flow throughout the network, contributing to reconstruction performance gain. Furthermore, we introduce a spectral geometry consistency loss, encouraging EDUNet to prioritize the geometric layouts of spectral curves, leading to a more precise capture of spectral information in hyperspectral images. Experiments are conducted using three benchmark datasets including KAIST, ICVL, and Harvard, along with some real data, comprising a total of 73 samples. The experimental results demonstrate that EDUNet outperforms 15 competing models across four metrics including PSNR, SSIM, SAM, and ERGAS.

CrossU-Net: Dual-modality cross-attention U-Net for segmentation of precancerous lesions in gastric cancer

Gastric precancerous lesions (GPL) significantly elevate the risk of gastric cancer, and precise diagnosis and timely intervention are critical for patient survival. Due to the elusive pathological features of precancerous lesions, the early detection rate is less than 10%, which hinders lesion localization and diagnosis. In this paper, we provide a GPL pathological dataset and propose a novel method for improving the segmentation accuracy on a limited-scale dataset, namely RGB and Hyperspectral dual-modal pathological image Cross-attention U-Net (CrossU-Net). Specifically, we present a self-supervised pre-training model for hyperspectral images to serve downstream segmentation tasks. Secondly, we design a dual-stream U-Net-based network to extract features from different modal images. To promote information exchange between spatial information in RGB images and spectral information in hyperspectral images, we customize the cross-attention mechanism between the two networks. Furthermore, we use an intermediate agent in this mechanism to improve computational efficiency. Finally, we add a distillation loss to align predicted results for both branches, improving network generalization. Experimental results show that our CrossU-Net achieves accuracy and Dice of 96.53% and 91.62%, respectively, for GPL lesion segmentation, providing a promising spectral research approach for the localization and subsequent quantitative analysis of pathological features in early diagnosis.

Learning Spatial–Spectral-Dimensional-Transformation-Based Features for Hyperspectral Image Classification

Recently, deep learning tools have made significant progress in hyperspectral image (HSI) classification. Most of existing methods implement a patch-based classification manner which may cause training test information leakage or waste labeled information for non-central pixels of image patches. Therefore, it is challenging to achieve remarkable classification performance via the traditional convolutional neural networks (CNN) in the absence of label information. Moreover, due to the limitation of convolutional kernel sizes and convolution operations, the spectral information of HSI cannot be fully utilized with a traditional CNN framework. In this paper, we implement pixel-based classification by a special data division strategy and propose a novel spatial–spectral dimensional transformation (SSDT) to obtain spectral features containing more spectral information. Then, we construct a fully convolutional network (FCN) with two branches based on 3D-FCN and 2D-FCN to achieve broader spatial and spectral information interaction. Finally, the fused features are utilized to realize accurate pixel-based classification. We verify our proposed method on three classic publicly available datasets; the overall classification accuracy and average accuracy reach 82.27%/87.85%, 83.81%/81.55%, and 85.97%/83.89%. Compared with the latest proposed method SS3FCN in the no-information-leakage scenario, the overall classification accuracy of our proposed method is improved by 1.72%, 4.95% and 0.2%, and the average accuracy is improved by 0.95%, 3.92% and 2.67% on the three databases, respectively. Experimental results demonstrate the effectiveness of the proposed SSDT and the proposed CNN framework.

Hyperspectral Super-Resolution Reconstruction Network Based on Hybrid Convolution and Spectral Symmetry Preservation

Hyperspectral images (HSI) have high-dimensional and complex spectral characteristics, with dozens or even hundreds of bands covering the same area of pixels. The rich information of the ground objects makes hyperspectral images widely used in satellite remote sensing. Due to the limitations of remote sensing satellite sensors, hyperspectral images suffer from insufficient spatial resolution. Therefore, utilizing software algorithms to improve the spatial resolution of hyperspectral images has become an urgent problem that needs to be solved. The spatial information and spectral information of hyperspectral images are strongly correlated. If only the spatial resolution is improved, it often damages the spectral information. Inspired by the high correlation between spectral information in adjacent spectral bands of hyperspectral images, a hybrid convolution and spectral symmetry preservation network has been proposed for hyperspectral super-resolution reconstruction. This includes a model to integrate information from neighboring spectral bands to supplement target band feature information. The proposed model introduces flexible spatial-spectral symmetric 3D convolution in the network structure to extract low-resolution and neighboring band features. At the same time, a combination of deformable convolution and attention mechanisms is used to extract information from low-resolution bands. Finally, multiple bands are fused in the reconstruction module, and the high-resolution hyperspectral image containing global information is obtained by Fourier transform upsampling. Experiments were conducted on the indoor hyperspectral image dataset CAVE, the airborne hyperspectral dataset Pavia Center, and Chikusei. In the X2 super-resolution task, the PSNR values achieved on the CAVE, Pavia Center, and Chikusei datasets were 46.335, 36.321, and 46.310, respectively. In the X4 super-resolution task, the PSNR values achieved on the CAVE, Pavia Center, and Chikusei datasets were 41.218, 30.377, and 38.365, respectively. The results show that our method outperforms many advanced algorithms in objective indicators such as PSNR and SSIM while maintaining the spectral characteristics of hyperspectral images.

A novel approach to combine spatial and spectral information from hyperspectral images

This article proposes a generic framework to process jointly the spatial and spectral information of hyperspectral images. First, sub-images are extracted. Then each of these sub-images follows two parallel workflows, one dedicated to the extraction of spatial features and the other dedicated to the extraction of spectral features. Finally, the extracted features are merged, producing as many scores as sub-images. Two applications are proposed, illustrating different spatial and spectral processing methods. The first one is related to the characterization of a teak wood disk, in an unsupervised way. It implements tensors of structure for the spatial branch, simple averaging for the spectral branch and multi-block principal component analysis for the fusion process. The second application is related to the early detection of apple scab on leaves. It implements co-occurrence matrices for the spatial branch, singular value decomposition for the spectral branch and multiblock partial least squares discriminant analysis for the fusion process. Both applications demonstrate the interest of the proposed method for the extraction of relevant spatial and spectral information and show how promising this new approach is for hyperspectral imaging processing.

Hyperspectral Image Classification Based on Dense Pyramidal Convolution and Multi-Feature Fusion

In recent years, hyperspectral image classification techniques have attracted a lot of attention from many scholars because they can be used to model the development of different cities and provide a reference for urban planning and construction. However, due to the difficulty in obtaining hyperspectral images, only a limited number of pixels can be used as training samples. Therefore, how to adequately extract and utilize the spatial and spectral information of hyperspectral images with limited training samples has become a difficult problem. To address this issue, we propose a hyperspectral image classification method based on dense pyramidal convolution and multi-feature fusion (DPCMF). In this approach, two branches are designed to extract spatial and spectral features, respectively. In the spatial branch, dense pyramid convolutions and non-local blocks are used to extract multi-scale local and global spatial features in image samples, which are then fused to obtain spatial features. In the spectral branch, dense pyramidal convolution layers are used to extract spectral features in image samples. Finally, the spatial and spectral features are fused and fed into fully connected layers to obtain classification results. The experimental results show that the overall accuracy (OA) of the method proposed in this paper is 96.74%, 98.10%, 98.92% and 96.67% on the four hyperspectral datasets, respectively. Significant improvements are achieved compared to the five methods of SVM, SSRN, FDSSC, DBMA and DBDA for hyperspectral classification. Therefore, the proposed method can better extract and exploit the spatial and spectral information in image samples when the number of training samples is limited. Provide more realistic and intuitive terrain and environmental conditions for urban planning, design, construction and management.

Hybrid Multi-Dimensional Attention U-Net for Hyperspectral Snapshot Compressive Imaging Reconstruction

In order to capture the spatial-spectral (x,y,λ) information of the scene, various techniques have been proposed. Different from the widely used scanning-based methods, spectral snapshot compressive imaging (SCI) utilizes the idea of compressive sensing to compressively capture the 3D spatial-spectral data-cube in a single-shot 2D measurement and thus it is efficient, enjoying the advantages of high-speed and low bandwidth. However, the reconstruction process, i.e., to retrieve the 3D cube from the 2D measurement, is an ill-posed problem and it is challenging to reconstruct high quality images. Previous works usually use 2D convolutions and preliminary attention to address this challenge. However, these networks and attention do not exactly extract spectral features. On the other hand, 3D convolutions can extract more features in a 3D cube, but increase computational cost significantly. To balance this trade-off, in this paper, we propose a hybrid multi-dimensional attention U-Net (HMDAU-Net) to reconstruct hyperspectral images from the 2D measurement in an end-to-end manner. HMDAU-Net integrates 3D and 2D convolutions in an encoder-decoder structure to fully utilize the abundant spectral information of hyperspectral images with a trade-off between performance and computational cost. Furthermore, attention gates are employed to highlight salient features and suppress the noise carried by the skip connections. Our proposed HMDAU-Net achieves superior performance over previous state-of-the-art reconstruction algorithms.

Automatic generation of pathological benchmark dataset from hyperspectral images of double stained tissues

Artificial intelligence has been widely used for digital pathology diagnosis. However, the AI performance highly relies on the high-quality annotated datasets, pathological images need to be labeled by experienced pathologists manually, which is time consuming, laborious and expensive. In addition, small lesion areas are usually missed by human eyes, directly influencing the performance of those identification models trained by the data. This paper presents a new strategy for generating annotated pathological benchmark dataset from microscopic hyperspectral images of HE-CAM5.2 stained tissues. We design a Spatial-Spectral based Hyperspectral GAN (SSHGAN), which transforms hyperspectral images into standard histological images using networks trained by the cycle consistent adversarial model. Gradient boosting decision tree integrated with graph-cut method is used to automatically generate the annotations by adding the spectral prior. The proposed strategy can obtain both the standard H&E images and the corresponding annotation files simultaneously using spatial and spectral information of hyperspectral images. The methods have been tested on gastric cancer, lung adenocarcinoma, intrahepatic cholangiocarcinoma, and colorectal cancer tissues and evaluated by segmentation networks and experienced pathologists. Experimental results show that the proposed methods have desirable performance on small tumor targets and discrete regions, which is promising in automatically generating completely annotation pathology benchmark datasets.

A micro-hyperspectral image classification method of gallbladder cancer based on multi-scale fusion attention mechanism

A micro-hyperspectral image classification method of gallbladder cancer based on multi-scale fusion attention mechanism

GPU Implementation of Graph-Regularized Sparse Unmixing With Superpixel Structures

To enhance spectral unmixing performance, a large number of algorithms have simultaneously investigated spatial and spectral information in hyperspectral images. However, sophisticated algorithms with high computational complexity can be very time-consuming when a large amount of data are involved in processing hyperspectral images. In this paper, we first introduce a group sparse graph-regularized unmixing method with superpixel structure, to promote piece-wise consistency of abundances and reduce computational burden. Segmenting the image into several non-overlapped superpixels also enables to decompose the unmixing problem into uncoupled subproblems that can be processed in parallel. An implementation for the proposed algorithm on graphics processing units (GPUs) is then developed based on the NVIDIA Compute Unified Device Architecture (CUDA) framework. The proposed scheme achieves parallelism at both the intra-superpixel and inter-superpixel levels, where multiple concurrent streams have been used to enable multiple kernels to execute on the device simultaneously. Simulation results with a series of experiments demonstrate advantages of the proposed algorithm. The performance of the GPU implementation also illustrates that parallel scheme largely expedites the implementation.

Hyperspectral image classification based on three-dimensional adaptive sampling and improved iterative shrinkage-threshold algorithm

Abundant spectral information of hyperspectral images (HSI) provides rich information for HSI classification, which often brings high dimensional data resulting in the dilemma between the demand for fine data and the limited resources such as computation, storage as well as transmission band-width. To address this issue, we propose a deep hierarchical feature representation model based on three-dimensional adaptive sampling and improved iterative shrinkage-threshold algorithm (ISTA) for HSI classification. Due to the adaptive sampling, we improve ISTA with deep learning network for spectral–spatial feature representation since the ISTA is no longer applicable for the sampled data reconstruction. Through end-to-end joint learning, the proposed method can not only effectively reduce the required data, but also learn discriminative features for HSI classification, which will be meaningful for the HSI’s transmission from the space satellites and fast classification. Experimental results demonstrate the effectiveness and superiority of the proposed method on three public HSI datasets.

Multispectral and Hyperspectral Image Fusion Based on Regularized Coupled Non-Negative Block-Term Tensor Decomposition

The problem of multispectral and hyperspectral image fusion (MHF) is to reconstruct images by fusing the spatial information of multispectral images and the spectral information of hyperspectral images. Focusing on the problem that the hyperspectral canonical polyadic decomposition model and the Tucker model cannot introduce the physical interpretation of the latent factors into the framework, it is difficult to use the known properties and abundance of endmembers to generate high-quality fusion images. This paper proposes a new fusion algorithm. In this paper, a coupled non-negative block-term tensor model is used to estimate the ideal high spatial resolution hyperspectral images, its sparsity is characterized by adding 1-norm, and total variation (TV) is introduced to describe piecewise smoothness. Secondly, the different operators in two directions are defined and introduced to characterize their piecewise smoothness. Finally, the proximal alternating optimization (PAO) algorithm and the alternating multiplier method (ADMM) are used to iteratively solve the model. Experiments on two standard datasets and two local datasets show that the performance of this method is better than the state-of-the-art methods.

Hyperspectral image super-resolution using cluster-based deep convolutional networks

In recent years, deep convolutional neural networks (CNNs) have been widely exploited for the hyperspectral image (HSI) super-resolution and obtained remarkable performance. However, most of the existing CNN-based methods have two main problems. One is to use two-dimension (2D) convolution to extract spatial information without paying attention to the mining of spectral information of hyperspectral images. The other is to use three-dimension (3D) convolution, which reduces the efficiency of the model when the network parameters increase. To address the above issues, we propose clustering deep residual neural network (CDRNN) for hyperspectral image super-resolution in this paper. The proposed CDRNN learns the complex, nonlinear mappings between low spatial resolution HSI and high spatial resolution HSI. At first, an unsupervised clustering method is used to divide a low spatial resolution HSI into several classes according to spectral correlation. Then, the spectrum-pairs from the classified low spatial resolution HSI and the corresponding high spatial resolution HSI are used to train the CDRNN to establish the nonlinear mapping for each class. Finally, we classify the given low spatial resolution HSI into the determined category and use the trained CDRNN to reconstruct the final high spatial resolution HSI from the classified low spatial resolution HSI. We conduct extensive experiments on three simulated benchmark datasets and a real HSI to evaluate the super-resolution performance of the proposed method. Experimental results show that our proposed method achieves significant improvement over state-of-the-art methods.

Fast robust fuzzy clustering based on bipartite graph for hyper‐spectral image classification

Hyper-spectral image (HSI) clustering has become a hot spot in remote sensing image research. However, due to the large amount and high dimension of HSI data, some traditional image clustering methods are no longer suitable, and even lead to poor clustering performance and long computing time. Therefore, this paper proposes a novel clustering method for HSI to make up for the shortcomings of traditional clustering methods. Firstly, a fuzzy similarity matrix is constructed by using the bipartite graph to obtain low-dimensional hyper-spectral data, which can reduce the complexity and the operation time of the algorithm. Secondly, fuzzy membership mapping is carried out through the constructed bipartite graph, and a non-negative regularization term is added to limit ill-conditioned problems. Thirdly, this paper uses the Geman-McClure function to optimize Euclidean distance. Finally, in this paper, the spectral information of HSI is considered while the neighbourhood information of HSI is added to achieve the unity of space spectrum, which enhances the robustness of the algorithm and improves the clustering performance of the algorithm. Compared with existing HSI clustering algorithms, the proposed algorithm has higher clustering accuracy and efficiency.

Hyperspectral Anomaly Detection Based on Wasserstein Distance and Spatial Filtering

Since anomaly targets in hyperspectral images (HSIs) with high spatial resolution appear as connected areas instead of single pixels or subpixels, both spatial and spectral information of HSIs can be exploited for a hyperspectal anomaly detection (AD) task. This article proposes a hyperspectral AD method based on Wasserstein distance (WD) and spatial filtering (called AD-WDSF). Based on the assumption that both background and anomaly targets obey the multivariate Gaussian distribution, background and anomaly target distributions are estimated in the local regions of HSIs. Subsequently, the anomaly intensity of test pixels centered in the local regions are determined via measuring the WD between background and anomaly target distributions. Lastly, spatial filters, i.e., guided filter (GF), total variation curvature filter (TVCF), and Maxtree filter, are exploited to further refine detection results. Experimental results conducted on two real hyperspectral data sets demonstrate that the proposed method achieves competitive detection performance compared with the state-of-the-art AD methods.

Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network

Although the existing deep-learning-based hyperspectral image (HSI) denoising methods have achieved tremendous success, recovering high-quality HSIs in complex scenes that contain mixed noise is still challenging. Besides, these methods have not fully explored the local and global spatial–spectral information of HSIs. To address the above issues, a novel HSI mixed noise removal network called subspace projection attention and residual channel attention network (SPARCA-Net) is proposed. Specifically, we propose an orthogonal subspace projection attention (OSPA) module to adaptively learn to generate bases of the signal subspace and project the input into such space to remove noise. By leveraging the local and global spatial relations, OSPA is able to reconstruct the local structure of the feature maps more precisely. We further propose a residual channel attention (RCA) module to emphasize the interdependence between feature maps and exploit the global channel correlation of them, which could enhance the channel-wise adaptive learning. In addition, multiscale joint spatial–spectral input and residual learning strategies are employed to capture multiscale spatial–spectral features and reduce the degradation problem, respectively. Synthetic and real HSI data experiments demonstrated that the proposed HSI denoising network outperforms many of the advanced methods in both quantitative and qualitative assessments.

Hyperspectral Image Classification Based on Spectral Multiscale Convolutional Neural Network

In recent years, convolutional neural networks (CNNs) have been widely used for hyperspectral image classification, which show good performance. Compared with using sufficient training samples for classification, the classification accuracy of hyperspectral images is easily affected by a small number of samples. Moreover, although CNNs can effectively classify hyperspectral images, due to the rich spatial and spectral information of hyperspectral images, the efficiency of feature extraction still needs to be further improved. In order to solve these problems, a spatial–spectral attention fusion network using four branch multiscale block (FBMB) to extract spectral features and 3D-Softpool to extract spatial features is proposed. The network consists of three main parts. These three parts are connected in turn to fully extract the features of hyperspectral images. In the first part, four different branches are used to fully extract spectral features. The convolution kernel size of each branch is different. Spectral attention block is adopted behind each branch. In the second part, the spectral features are reused through dense connection blocks, and then the spectral attention module is utilized to refine the extracted spectral features. In the third part, it mainly extracts spatial features. The DenseNet module and spatial attention block jointly extract spatial features. The spatial features are fused with the previously extracted spectral features. Experiments are carried out on four commonly used hyperspectral data sets. The experimental results show that the proposed method has better classification performance than some existing classification methods when using a small number of training samples.

SSCAN: A Spatial–Spectral Cross Attention Network for Hyperspectral Image Denoising

Hyperspectral images (HSIs) have been widely used in a variety of applications thanks to the rich spectral information they are able to provide. Among all HSI processing tasks, HSI denoising is a crucial step. Recently, deep learning-based image denoising methods have made great progress and achieved great performance. However, existing methods tend to ignore the correlations between adjacent spectral bands, leading to problems such as spectral distortion and blurred edges in denoised results. In this study, we propose a novel HSI denoising network, termed SSCAN, that combines group convolutions and attention modules. Specifically, we use a group convolution with a spatial attention module to facilitate feature extraction by directing models' attention to band-wise important features. We propose a spectral-spatial attention block (SSAB) to exploit the spatial and spectral information in hyperspectral images in an effective manner. In addition, we adopt residual learning operations with skip connections to ensure training stability. The experimental results indicate that the proposed SSCAN outperforms several state-of-the-art HSI denoising algorithms.

Structure–Color Preserving Network for Hyperspectral Image Super-Resolution

Fusion-based hyperspectral super-resolution (HSR) algorithms usually utilize a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (MSI) to generate a high-resolution hyperspectral image (HR-HSI), which have attracted increasing attention in recent years. However, how to deal with the abundant spectral information of hyperspectral images and complex structure characteristics of MSIs has always been the focus and difficulty of fusion-based HSR. In this article, we propose a new structure–color preserving network (SCPNet) for HSR, which is developed under the basis of the joint attention mechanism. The SCPNet mainly includes three modules: structure-preserving module (SPM), color-preserving module (CPM), and cross-fusion module. The SPM is constructed based on the spatial attention, which aims to capture and enhance the significant structure information from the high-resolution MSI. Meanwhile, the CPM is constructed based on the channel attention, where the spectral characteristics in the LR-HSI are preserved during the reconstruction process. Finally, we propose a cross attention-based cross-fusion strategy to integrate the features from the two branches and reconstruct the final HR-HSI. The major contribution of SCPNet is that the structure and color information is described and preserved via the joint attention mechanism. Experimental results indicate that the proposed SCPNet has presented advantages on three benchmark datasets when compared with some state-of-the-art HSR methods.