Hyperspectral image classification via shape-adaptive deep learning

Hyperspectral Image (HSI) classification plays a key role remote sensing field. Recently, deep learning has demonstrated its effectiveness in HSI Classification field. This paper presents a spectral-spatial HSI classification technique established on the deep learning based deep belief network (DBN) for deep and abstract feature extraction and adaptive boundary adjustment based segmentation. Proposed approach focuses on integrating the deep learning based spectral features and segmentation based spatial features into a framework for improved performance. Specifically, first the deep DBN model is exploited as a spectral feature extraction based classifier to extract the deep spectral features. Second, spatial contextual features are obtained by utilizing effective adaptive boundary adjustment based segmentation technique. Finally, maximum voting based criteria is operated to integrate the results of extracted spectral and spatial information for improved HSI classification. In general, exploiting spectral features from DBN process and spatial features from segmentation and integration of spectral and spatial information by maximum voting based criteria, has a substantial effect on the performance of HSI classification. Experimental performance on real and widely used hyperspectral data sets with different contexts and resolutions demonstrates the accuracy of the proposed technique and performance is comparable to several recently proposed HSI classification techniques.

A deep-learning-based feature extraction has recently been proposed for HyperSpectral Images (HSI) classification. A Deep Belief Network (DBN), as part of deep learning, has been used in HSI classification for deep and abstract feature extraction. However, DBN has to simultaneously deal with hundreds of features from the HSI hyper-cube, which results into complexity and leads to limited feature abstraction and performance in the presence of limited training data. Moreover, a dimensional-reduction-based solution to this issue results in the loss of valuable spectral information, thereby affecting classification performance. To address the issue, this paper presents a Spectral-Adaptive Segmented DBN (SAS-DBN) for spectral-spatial HSI classification that exploits the deep abstract features by segmenting the original spectral bands into small sets/groups of related spectral bands and processing each group separately by using local DBNs. Furthermore, spatial features are also incorporated by first applying hyper-segmentation on the HSI. These results improved data abstraction with reduced complexity and enhanced the performance of HSI classification. Local application of DBN-based feature extraction to each group of bands reduces the computational complexity and results in better feature extraction improving classification accuracy. In general, exploiting spectral features effectively through a segmented-DBN process and spatial features through hyper-segmentation and integration of spectral and spatial features for HSI classification has a major effect on the performance of HSI classification. Experimental evaluation of the proposed technique on well-known HSI standard data sets with different contexts and resolutions establishes the efficacy of the proposed techniques, wherein the results are comparable to several recently proposed HSI classification techniques.

With the continuous innovation in deep learning, it has become a major direction for scholars to introduce the knowledge of deep learning into hyperspectral image classification to enhance its classification accuracy. Convolutional Neural Networks (CNN) are one of the most commonly used deep learning-based visual data processing methods, and are widely used in hyperspectral image (HSI) classification by virtue of their excellent contextual modeling capability. Since the performance of HSI classification is highly dependent on spatial and spectral information, this paper proposes a hyperspectral image classification method using 3D attention mechanism in collaboration with Transformer for hyperspectral image classification in view of the problems that the current hyperspectral image classification models with the framework of CNN have insufficient spatial spectral feature extraction and fail to excavate and represent the sequence properties of spectral features well. In this paper, we introduce a variant Transformer model based on a hybrid model of both improved 3D-CNN and 2D-CNN, combining complementary information of spatial spectrum and spectra in the form of 3D convolution and 2D convolution on CNN, and adding a variant attention mechanism module to strengthen spatial texture features, while combining grouped transfer Transformer to jump connection to enable the lower layer to better learn the upper layer features. Firstly, a variant channel attention mechanism is introduced on 3D-CNN to enhance the acquisition of spectral information of image features by 3D-CNN. Secondly, a variant spatial attention mechanism is introduced to enable 3D-CNN to better acquire the spatial information of hyperspectral images in the network, and subsequently the acquired spatial and spectral feature information is passed to 2D-CNN to enable it to better acquire local feature information. Finally, the acquired image feature information is passed to the variant Transformer model to make up for the fact that CNN can only acquire hyperspectral image features in local contexts, enabling it to better acquire global feature information on feature sequences. The experimental results show that the proposed model is experimented on two hyperspectral datasets, Indian Pines and Pavia University, and the overall classification accuracy (OA), average classification accuracy (AA), and Kappa coefficient reach up to 99.59%, 99.31%, and 99.45%, respectively, on the PU dataset, compared with the current cutting-edge techniques. The classification accuracy has been improved.

Classification of distinct classes in hyperspectral images (HSI) is one of the most pervasive problem in remote sensing field. Deep learning has recently proved its efficiency in HSI classification. However, incorporating spatial/contextual features along with spectral information in deep network is still a challenging task. In this paper, for an effective spectral-spatial feature extraction, an improved deep network, spatial updated hyper-voxel stacked auto-encoder (HVSAE) approach is proposed which exploits spatial context within spectrally similar contiguous pixels for effective HSI classification. The proposed approach involves two key steps-firstly, we compute adaptive boundary adjustment based segmentation whose size and shape can be adapted according to the spatial structures and which consists of spatially contiguous pixels with similar spectral features, followed by an object-level classification using stacked auto-encoder (SAE) based decision fusion approach that merges spatial-segmented outcome and spectral information into a SAE framework for robust spectral-spatial HSI classification. In addition, instead of directly using a large number of spectral bands, band preference and correlation based band selection approach is used to select the most informative bands without compromising the original content in HSI. Use of local spatial structural regularity and spectral similarity information from adaptive boundary adjustment based process, and fusion of spatial context and spectral features into SAE has significant effect on the accuracy of the final HSI classification. Experimental results on real divergent hyperspectral imagery with different contexts and resolutions validates the classification accuracy of the proposed method over several existing techniques.

Hyperspectral Image (HSI) classification is one of the most persistent issue in remote sensing field. Recently, deep learning has attracted attention in HSI Classification field due to its accuracy and stronger generalization. This paper proposes a new spectral-spatial HSI classification approach developed on the deep learning concept of stacked-auto-encoders (SAE) based deep feature extraction and hidden Markov random field based segmentation. Specifically, First the SAE model is implemented as a spectral information-based classifier to extract the deep spectral features. Second, spatial information is obtained by using effective Hidden Markov random field (HMRF) based segmentation technique. Finally, maximum voting based criteria is employed to merge the extracted spectral and spatial information, which results in the precise spectral-spatial HSI classification. The characterization of the HSI with spectral spatial features results into more comprehensive analysis of HSI and to a more accurate classification. In general, use of spectral information resulted from the SAE process and spatial information by means of HMRF based segmentation and merging of spectral and spatial information by means of maximum voting based criteria, has a significant effect on the accuracy of the HSI classification. Experiments on real diverse hyperspectral data sets with different contexts and resolutions acquired by AVIRIS and ROSIS sensors show the accuracy of the proposed method and confirms that results of the proposed classification approach are comparable to several recently proposed HSI classification techniques.

Deep learning based hyperspectral image (HSI) classification have recently shown promising performance. However, complex network architecture, tedious training process and effective utilization of spatial/contextual information in deep network limits the application and performance of deep learning. In this paper, for an effective spectral-spatial feature extraction , an improved deep network, spatial adaptive network (SANet) approach is proposed which exploits spatial contextual information and spectral characteristics to construct a more simplified deep network which leads to more powerful feature representation for effective HSI classification. SANet is established from the simple structure of a principal component analysis network. First spatial structural information is extracted and combined with informative spectral channels followed by an object-level classification using SANet based decision fusion approach. It integrates spatial-contextual outcome and spectral characteristics into a SANet framework for robust spectral-spatial HSI classification. Integration of local structural regularity and spectral similarity into simplified deep SANet has significant effect on the classification performance. Experimental results on popular standard HSI datasets reveal that proposed SANet technique produce better classification results than existing well known techniques.

Deep learning has achieved significant success in the field of hyperspectral image (HSI) classification, but challenges are still faced when the number of training samples is small. Feature fusing approaches based on multi-channel and multi-scale feature extractions are attractive for HSI classification where few samples are available. In this paper, based on feature fusion, we proposed a simple yet effective CNN-based Dual-channel Spectral Enhancement Network (DSEN) to fully exploit the features of the small labeled HSI samples for HSI classification. We worked with the observation that, in many HSI classification models, most of the incorrectly classified pixels of HSI are at the border of different classes, which is caused by feature obfuscation. Hence, in DSEN, we specially designed a spectral feature extraction channel to enhance the spectral feature representation of the specific pixel. Moreover, a spatial–spectral channel was designed using small convolution kernels to extract the spatial–spectral features of HSI. By adjusting the fusion proportion of the features extracted from the two channels, the expression of spectral features was enhanced in terms of the fused features for better HSI classification. The experimental results demonstrated that the overall accuracy (OA) of HSI classification using the proposed DSEN reached 69.47%, 80.54%, and 93.24% when only five training samples for each class were selected from the Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SA) datasets, respectively. The performance improved when the number of training samples increased. Compared with several related methods, DSEN demonstrated superior performance in HSI classification.

Hyperspectral images contain rich spatial and spectral information, which makes more and more researchers join the team of analyzing and studying them. Convolutional neural networks have been widely used in hyperspectral image classification, however, due to the high dimensionality and band correlation of the hyperspectral image data, the hyperspectral data contains a lot of redundant information, which not only adds to the arithmetic burden, but also affects the extraction of the global and local spectral and spatial features in the process of hyperspectral image classification. We design a hybrid convolutional model based on spatial and spectral channel reconstruction, which utilizes hybrid convolution to extract spatial and spectral features in hyperspectral images, and separates and reconstructs the spatial and spectral channels to suppress redundant features and reduce the computational load of the model, and introduces a global attention mechanism to enhance the global receptive field and learn the global spectral and spatial features. We conduct experiments on three widely used public datasets, IndianPines, PaviaU, and Houston 2013, and the overall accuracies reach 98.66%, 99.49%, and 99.07%, respectively, which validate the effectiveness of the model.

Hyperspectral image (HSI) classification has been one of the most important tasks in the remote sensing community over the last few decades. Due to the presence of highly correlated bands and limited training samples in HSI, discriminative feature extraction was challenging for traditional machine learning methods. Recently, deep learning based methods have been recognized as powerful feature extraction tool and have drawn a significant amount of attention in HSI classification. Among various deep learning models, convolutional neural networks (CNNs) have shown huge success and offered great potential to yield high performance in HSI classification. Motivated by this successful performance, this paper presents a systematic review of different CNN architectures for HSI classification and provides some future guidelines. To accomplish this, our study has taken a few important steps. First, we have focused on different CNN architectures, which are able to extract spectral, spatial, and joint spectral-spatial features. Then, many publications related to CNN based HSI classifications have been reviewed systematically. Further, a detailed comparative performance analysis has been presented between four CNN models namely 1D CNN, 2D CNN, 3D CNN, and feature fusion based CNN (FFCNN). Four benchmark HSI datasets have been used in our experiment for evaluating the performance. Finally, we concluded the paper with challenges on CNN based HSI classification and future guidelines that may help the researchers to work on HSI classification using CNN.

Graph convolutional neural networks (GCNs) have been widely used in hyperspectral images (HSIs) classification for their superiority in processing non-Euclidean structure data. The performance of GCNs relies on the initial graph structure. Most GCN models only utilize spectral information to construct a graph, which is inaccurate because they fail to take the relationship between adjacent nodes into consideration. In addition, due to the over-smooth phenomenon, most GCN models are shallow and unable to extract effective features. To address these issues, a dual graph u-nets is proposed by integrating spatial graph and spectral graph for HSIs classification, denoted by DGU-HSI. To integration the spectral and spatial information, two graphs are constructed for feature extraction. The spectral graph is created by spectral similarity among all pixels where multirange spectral information is retained, and the spatial graph is constructed by exploiting the neighborhood relationship of the center pixel, which describes spatial information. Then, a dual GCN is utilized to extract spatial and spectral graph features simultaneously. To relieve the over-smooth phenomenon, the graph u-nets structure is applied on constructed spectral and spatial graph to extract effective features. Finally, the extracted spectral and spatial features are fused for classification. Experiments conducted on the public datasets demonstrate the effectiveness of the proposed method on HSIs classification.

Hyperspectral image (HSI) classification has been a hot topic for decides, as hyperspectral images have rich spatial and spectral information and provide strong basis for distinguishing different land-cover objects. Benefiting from the development of deep learning technologies, deep learning based HSI classification methods have achieved promising performance. Recently, several neural architecture search (NAS) algorithms have been proposed for HSI classification, which further improve the accuracy of HSI classification to a new level. In this paper, NAS and Transformer are combined for handling HSI classification task for the first time. Compared with previous work, the proposed method has two main differences. First, we revisit the search spaces designed in previous HSI classification NAS methods and propose a novel hybrid search space, consisting of the space dominated cell and the spectrum dominated cell. Compared with search spaces proposed in previous works, the proposed hybrid search space is more aligned with the characteristic of HSI data, that is, HSIs have a relatively low spatial resolution and an extremely high spectral resolution. Second, to further improve the classification accuracy, we attempt to graft the emerging transformer module on the automatically designed convolutional neural network (CNN) to add global information to local region focused features learned by CNN. Experimental results on three public HSI datasets show that the proposed method achieves much better performance than comparison approaches, including manually designed network and NAS based HSI classification methods. Especially on the most recently captured dataset Houston University, overall accuracy is improved by nearly 6 percentage points. Code is available at: https://github.com/Cecilia-xue/HyT-NAS.

Convolutional Neural Networks (CNNs) have been extensively applied to hyperspectral (HS) image classification tasks and achieved promising performance. However, for CNN based HS image classification methods, it is hard to depict the dependencies among HS image pixels in long-range distanced positions and bands. Moreover, the limited receptive field of the convolutional layers extremely hinders the development of the CNN structure. To tackle these problems, in this paper, the novel Bottleneck Spatial-Spectral Transformer (BS2T) is proposed to depict the long-range global dependencies of HS image pixels, which can be regarded as a feature extraction module for HS image classification networks. More specifically, inspired by Bottleneck Transformer in computer vision, for HS image feature extraction, the proposed BS2T is incorporated with a feature contraction module, a multi-head spatial-spectral self-attention (MHS2A) module and a feature expansion module. In this way, convolutional operations are replaced by the MHS2A to capture the long-range dependency of HS pixels regardless of their spatial position and distance. Meanwhile, in the MHS2A module, to highlight the spectral features of HS images, we introduce the spectral information and content spatial positional information to classical multi-head self-attentions to make the attentions more positional aware and spectral aware. On this basis, a dual-branch HS image classification framework based on 3D CNN and BS2T is defined for jointly extracting the local-global features of HS images. Experimental results on three public HS image classification datasets show that the proposed classification framework achieves a significant improvement when comparing with the state-of-the-art methods. The source code of the proposed framework can be downloaded from https://github.com/srxlnnu/BS2T.

Deep spectral-spatial features fusion has become a research focus in hyperspectral image (HSI) classification. However, how to extract more robust spectral-spatial features is still a challenging problem. In this article, a novel deep multilayer fusion dense network (MFDN) is proposed to improve the performance of HSI classification. The proposed MFDN simultaneously extracts the spatial and spectral features based on different sample input sizes, which can extract abundant spectral and spatial correlation information. First, the principal component analysis algorithm is performed on hyperspectral data to extract low-dimensional HSI data, and then the spatial features are extracted from the low-dimensional 3-D HSI data through 2-D convolutional, 2-D dense block, and average-pooling layers. Second, the spectral features are extracted directly from the raw 3-D HSI data by means of 3-D convolutional, 3-D dense block, and average-pooling layers. Third, the spatial and spectral features are fused together through 3-D convolutional, 3-D dense block, and average-pooling layers. Finally, the fused spectral-spatial features are sent into two full connection layers to extract high-level abstract features. Furthermore, densely connected structures can help alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and improve the HSI classification accuracy. The proposed fusion network outperforms the other state-of-the-art methods especially with a small number of labeled samples. Experimental results demonstrate that it can achieve outstanding hyperspectral classification performance.

Deep learning has achieved great success in hyperspectral image (HSI) classification. However, its success relies on the availability of sufficient training samples. Unfortunately, the collection of training samples is expensive, time-consuming, and even impossible in some cases. Natural image datasets that are different from HSI, such as Image Net and mini-ImageNet, have abundant texture and structure information. Effective knowledge transfer between two heterogeneous datasets can significantly improve the accuracy of HSI classification. In this letter, heterogeneous few-shot learning (HFSL) for HSI classification is proposed with only a few labeled samples per class. First, few-shot learning is performed on the mini-ImageNet datasets to learn the transferable knowledge. Then, to make full use of the spatial and spectral information, a spectral&#x2013;spatial fusion network is devised. Spectral information is obtained by the residual network with pure 1-D operators. Spatial information is extracted by a convolution network with pure 2-D operators, and the weights of the spatial network are initialized by the weights of the model trained on the mini-ImageNet datasets. Finally, few-shot learning is fine-tuned on HSI to extract discriminative spectral&#x2013;spatial features and individual knowledge, which can improve the classification performance of the new classification task. Experiments conducted on two public HSI datasets demonstrate that the HFSL outperforms the existing few-shot learning methods and supervised learning methods for HSI classification with only a few labeled samples. Our source code is available at <uri>https://github.com/Li-ZK/HFSL</uri>.

Although the hyperspectral image (HSI) classification has adopted deep neural networks (DNNs) and shown remarkable performances, there is a lack of studies of the adversarial vulnerability for the HSI classifications. In this paper, we propose a novel HSI classification framework robust to adversarial attacks. To this end, we focus on the unique spectral characteristic of HSIs (<italic>i.e.,</italic> distinctive spectral patterns of materials). With the spectral characteristic, we present the random spectral sampling and spectral shape feature encoding for the robust HSI classification. For the random spectral sampling, spectral bands are randomly sampled from the entire spectrum for each pixel of the input HSI. Also, the overall spectral shape information, which is robust to adversarial attacks, is fed into the shape feature extractor to acquire the spectral shape feature. Then, the proposed framework can provide the adversarial robustness of HSI classifiers via randomization effects and spectral shape feature encoding. To the best of our knowledge, the proposed framework is the first work dealing with the adversarial robustness in the HSI classification. In experiments, we verify that our framework improves the adversarial robustness considerably under diverse adversarial attack scenarios, and outperforms the existing adversarial defense methods.