Spectral-spatial Network Research Articles

Fast Nano-IR Hyperspectral Imaging Empowered by Large-Dataset-Free Miniaturized Spatial-Spectral Network.

The emerging field of nanoscale infrared (nano-IR) offers label-free molecular contrast, yet its imaging speed is limited by point-by-point traverse acquisition of a three-dimensional (3D) data cube. Here, we develop a spatial-spectral network (SS-Net), a miniaturized deep-learning model, together with compressive sampling to accelerate the nano-IR imaging. The compressive sampling is performed in both the spatial and spectral domains to accelerate the imaging process. The SS-Net is trained to learn the mapping from small nano-IR image patches to the corresponding spectra. With this elaborated mapping strategy, the training can be finished quickly within several minutes using the subsampled data, eliminating the need for a large-labeled dataset of common deep learning methods. We also designed an efficient loss function, which incorporates the image and spectral similarity to enhance the training. We first validate the SS-Net on an open stimulated Raman-scattering dataset; the results exhibit the potential of 10-fold imaging speed improvement with state-of-the-art performance. We then demonstrate the versatility of this approach on atomic force microscopy infrared (AFM-IR) microscopy with 7-fold imaging speed improvement, even on nanoscale Fourier transform infrared (nano-FTIR) microscopy with up to 261.6 folds faster imaging speed. We further showcase the generalization of this method on AFM-force volume-based multiparametric nanoimaging. This method establishes a paradigm for rapid nano-IR imaging, opening new possibilities for cutting-edge research in materials, photonics, and beyond.

S3Net: Spectral–Spatial Siamese Network for Few-Shot Hyperspectral Image Classification

Deep learning (DL) has shown great potentials for hyperspectral image (HSI) classification due to its powerful ability of nonlinear modeling and end-to-end optimization. However, DL models are easily get trapped into overfitting due to limited training labels since the labeling process is time-consuming and laborious in real classification scenario. To overcome this issue, we propose a novel spectral-spatial siamese network (S3Net) for few-shot HSI classification. Firstly, a lightweight spectral-spatial network (SSN) composed of 1-D and 2-D convolution is proposed to extract spectral-spatial features. Secondly, S3Net is constructed by two SSNs in dual branches, which can augment training set by feeding sample pairs into each branch, and thus enhancing the model separability. To provide more features for the model, differentiated patches are fed into each branch, where negative samples are random selected to avoid redundancy. Finally, a weighted contrastive loss is designed to promote the model to fit in the right direction by focusing on sample pairs that are hardly to be identified. Moreover, another adaptive cross entropy loss is conceived to learn the fusion ratio of the two branches. Experiments based on three commonly used HSI data sets demonstrate that S3Net outperforms traditional and state-of-the-art DL-based HSI classification methods under few-shot training scenario. In addition, the weighted contrastive loss and the adaptive cross entropy loss jointly improve the discrimination power of the model.

Deep Manifold Structure-Preserving Spectral–Spatial Feature Extraction of Hyperspectral Image

The deep network has shown its superiority to extract discriminative features for hyperspectral image (HSI) classification. However, most existing methods only exploit label information of land classes to supervise the learning of deep features, and the learning process is totally automatic. Considering the complex spectral-spatial characteristic in real HSI, it is reasonable to utilize the manifold structure of input data to provide complementary information for precise classification. This article proposes a structure-preserving spectral-spatial network (SPSSN) to exploit the manifold structure information during the feature learning process, which can extract discriminative deep structure-preserving spectral-spatial features. Specifically, a basic spectral-spatial network (SSN), comprising a spectral module and a spatial module connected in series, is first built to learn spectral-spatial features from HSI cubes. Second, a novel structure-preserving constraint is introduced to transfer the intrinsic manifold structure of the input data into the learned deep spectral-spatial features, so as to enhance the discrimination. Furthermore, a feasible algorithm of imposing the constraint is designed to make it compatible with the deep learning framework, which results in a structure-preserving loss. Finally, the SSN is combined with the structure-preserving loss to obtain the SPSSN. Experiments on real HSI datasets verify the effectiveness and superiority of the SPSSN compared with several state-of-the-art methods in HSI classification.

Spectral and Spatial Global Context Attention for Hyperspectral Image Classification

Recently, hyperspectral image (HSI) classification has attracted increasing attention in the remote sensing field. Plenty of CNN-based methods with diverse attention mechanisms (AMs) have been proposed for HSI classification due to AMs being able to improve the quality of feature representations. However, some of the previous AMs squeeze global spatial or channel information directly by pooling operations to yield feature descriptors, which inadequately utilize global contextual information. Besides, some AMs cannot exploit the interactions among channels or positions with the aid of nonlinear transformation well. In this article, a spectral-spatial network with channel and position global context (GC) attention (SSGCA) is proposed to capture discriminative spectral and spatial features. Firstly, a spectral-spatial network is designed to extract spectral and spatial features. Secondly, two novel GC attentions are proposed to optimize the spectral and spatial features respectively for feature enhancement. The channel GC attention is used to capture channel dependencies to emphasize informative features while the position GC attention focuses on position dependencies. Both GC attentions aggregate global contextual features of positions or channels adequately, following a nonlinear transformation. Experimental results on several public HSI datasets demonstrate that the spectral-spatial network with GC attentions outperforms other related methods.

A New Dataset and Deep Residual Spectral Spatial Network for Hyperspectral Image Classification

Due to the limited varieties and sizes of existing public hyperspectral image (HSI) datasets, the classification accuracies are higher than 99% with convolutional neural networks (CNNs). In this paper, we presented a new HSI dataset named Shandong Feicheng, whose size and pixel quantity are much larger. It also has a larger intra-class variance and a smaller inter-class variance. State-of-the-art methods were compared on it to verify its diversity. Otherwise, to reduce overfitting caused by the imbalance between high dimension and small quantity of labeled HSI data, existing CNNs for HSI classification are relatively shallow and suffer from low capacity of feature learning. To solve this problem, we proposed an HSI classification framework named deep residual spectral spatial setwork (DRSSN). By using shortcut connection structure, which is an asymmetry structure, DRSSN can be deeper to extract features with better discrimination. In addition, to alleviate insufficient training caused by unbalanced sample sizes between easily and hard classified samples, we proposed a novel training loss function named sample balanced loss, which allocated weights to the losses of samples according to their prediction confidence. Experimental results on two popular datasets and our proposed dataset showed that our proposed network could provide competitive results compared with state-of-the-art methods.

Hyperspectral image super-resolution with spectral–spatial network

ABSTRACTThe super-resolution problem for hyperspectral images is currently one of the most challenging topics in remote sensing. Increasingly effective methods have been presented to solve this ill-posed problem under certain circumstances. In this article, we propose a new approach named the spectral–spatial network (SSN), which can effectively increase spatial resolution while keeping spectral information. The SSN consists of two sections: a spatial section and a spectral section that contribute to enhancing spatial resolution and preserving spectral information, respectively. The spatial section is proposed to learn end-to-end mapping between single-band images, from low-resolution and high-resolution hyperspectral images. In this section, we enhance the traditional sub-pixel convolutional layer by adding a maximum variance principle that can realize nonlinear fitting through piecewise linearization. The spectral section aims to fine-tune spectral caves to keep the spectral signature with a spectral angle error loss function. In order to make the SSN converge quickly, we also develop a corresponding three-step training method. The experimental results on two databases, with both indoor and outdoor scenes, show that our proposed method performs better than the existing state-of-the-art methods.