Spectral Spatial Feature Learning Research Articles

Composite Neighbor-Aware Convolutional Metric Networks for Hyperspectral Image Classification.

Supervised classification of hyperspectral image (HSI) is generally required to obtain better performance in spectral-spatial feature learning by fully using complex pixel-and superpixel-level interdependencies with small labeled samples. Limited by the local regular convolutions, convolutional neural networks (CNNs) can only exploit information from the short-range Euclidean neighbors of a target, hindering the effectiveness of feature representation. In contrast, graph convolutional networks (GCNs) can learn long-range dependencies between non-Euclidean neighbors but usually require the input of a full graph constructed from a whole HSI, making GCNs must be trained in a full-batch manner with tremendous computational consumption. In this work, we propose a composite neighbor-aware convolutional metric network (CNCMN), aiming to learn each target's representation from its composite neighbors (i.e., both Euclidean and non-Euclidean neighbors) in a batchwise manner. Specifically, for each target in an HSI, its Euclidean neighbors are the pixels in the local square region centered on itself, and its non-Euclidean neighbors are several related nodes selected from the constructed full graph. Correspondingly, a composite convolution (CoConv) is proposed by coupling an image convolution and a graph convolution, which can perform flexible convolutions on those composite neighbors and extract adaptively fused features from them. Besides, to further boost classification, we also propose a mini-batch metric classifier to dynamically optimize interclass and intraclass distances of samples batch by batch, which is then combined with the CoConv to form the mini-batch CNCMN. Extensive experiments on three real-world HSIs demonstrate the advantages of the proposed method over mini-batch deep learning algorithms and have obtained the state-of-the-art performance in these fields. The code is available at: https://github.com/qichaoliu/HSI-CNCMN.

Exploring the Relationship Between Center and Neighborhoods: Central Vector Oriented Self-Similarity Network for Hyperspectral Image Classification

To mine the spectral-spatial information of target pixel in hyperspectral image classification (HSIC), convolutional neural network (CNN)-based models widely adopt patch-based input pattern, where a patch represents its central pixel and the neighbor pixels play auxiliary roles in the classification process. However, compared to the central pixel, its neighbor pixels often have different contributions for classification. Although many existing patch-based CNNs could adaptively emphasize the spatial neighbor information, most of them ignore the latent relationship between the center pixel and its neighbor pixels. Moreover, efficient spectral-spatial feature extraction has been a difficult yet vital topic for HSIC. To address the mentioned problems, a central vector oriented self-similarity network (CVSSN) is proposed for HSIC. Specifically, based on two similarity measures, we firstly design an adaptive weight addition based spectral vector self-similarity module (AWA-SVSS) in input space and a Euclidean distance based feature vector self-similarity module (ED-FVSS) in feature space to fully mine the central vector oriented spatial relationships. Besides, a spectral-spatial information fusion module (SSIF) is formulated as a new pattern to fuse the central 1D spectral vector and the corresponding 3D patch for efficient spectral-spatial feature learning of the subsequent modules. Moreover, we implement a channel spatial separation convolution module (CSS-Conv) and a scale information complementary convolution module (SIC-Conv) for efficient spectral-spatial feature learning. Extensive experimental results on four popular HSI data sets demonstrate the effectiveness and efficiency of the proposed method compared with other state-of-the-art methods. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/lms-07/CVSSN</uri>

Grouped Multi-Attention Network for Hyperspectral Image Spectral-Spatial Classification

Deep learning has been a powerful tool for hyperspectral image (HSI) classification. However, it is still an open issue to effectively learn highly discriminative features from the HSI, due to the high-dimensionality and complex spectral-spatial characteristics. To settle this issue, we propose a new band-grouping guided multi-attention module for the performance promotion of spectral-spatial feature learning. First, based on the fact of high relevance between adjacent spectral bands and low dependencies across long-range ones, all the spectral bands are adaptively divided into multiple non-overlapping groups where relevant bands are included. The advantage is to reduce the spectral dimension and data complexity when processing and analyzing each group. Then, a multi-attention mechanism, which not only explore the intra-group salient information but also propagate the inter-group difference information, is embedded into the convolutional neural networks to learn group-specific spectral-spatial features. By emphasizing useful spectral/spatial information and squeezing useless information with attention mechanism, the severability of learned features is enhanced. Based on this module, a spectral-spatial classification network is built, named by grouped multi-attention network (GMA-Net). The GMA-Net contains a two-branch architecture, i.e., pixel-wise spectral feature learning and patch-wise spectral-spatial feature learning. Via fusing the features from two branches, the complementary and discriminative features provided by pixel-wise and patch-wise learning manner can be integrated to further boost classification performance. Experimental results demonstrate that the proposed method is superior than several state-of-the-art approaches. Codes are available at: https://github.com/luting-hnu.

Adaptive Mask Sampling and Manifold to Euclidean Subspace Learning With Distance Covariance Representation for Hyperspectral Image Classification

For the abundant spectral and spatial information recorded in hyperspectral images (HSIs), fully exploring spectral-spatial relationships has attracted widespread attention in hyperspectral image classification (HSIC) community. However, there are still some intractable obstructs. For one thing, in the patch based processing pattern, some spatial neighbor pixels are often inconsistent with the central pixel in land-cover class. For another thing, linear and nonlinear correlations between different spectral bands are vital yet tough for representing and excavating. To overcome these mentioned issues, an adaptive mask sampling and manifold to Euclidean subspace learning (AMS-M2ESL) framework is proposed for HSIC. Specifically, an adaptive mask based intra-patch sampling (AMIPS) module is firstly formulated for intra-patch sampling in an adaptive mask manner based on central spectral vector oriented spatial relationships. Then, based on distance covariance descriptor, a dual channel distance covariance representation (DC-DCR) module is proposed for modeling unified spectral-spatial feature representations and exploring spectral-spatial relationships, especially linear and nonlinear interdependence in spectral domain. Furthermore, considering that distance covariance matrix lies on the symmetric positive definite (SPD) manifold, we implement a manifold to Euclidean subspace learning (M2ESL) module respecting Riemannian geometry of SPD manifold for high-level spectral-spatial feature learning. Additionally, we introduce an approximate matrix square-root (ASQRT) layer for efficient Euclidean subspace projection. Extensive experimental results on three popular HSI data sets with limited training samples demonstrate the superior performance of the proposed method compared with other state-of-the-art methods. The source code is available at https://github.com/lms-07/AMS-M2ESL.

A dual-branch multi-feature deep fusion network framework for hyperspectral image classification

Recent research has proven that deep learning models are effective at mining the rich spectral-spatial information of hyperspectral images and attaining high classification performance. In this study, an end-to-end two-branch multi-feature deep fusion network framework (MFDFN) is proposed, combining object-oriented features of hyperspectral images after multi-scale super-pixel segmentation and original image point-image element neighbourhood block features, and designing different network modules in the upper and lower branches for different features in order to extract more distinguishable and advantageous features for classification. For the second part, a series spectral-spatial feature learning modules and augmentation modules for spatial feature learning are being developed to learn these features in a continuous manner while also effectively fusing features from various depth layers. It also uses a variety of techniques (BN, Dropout, residual join, etc.) to speed up the model’s convergence, avoid overfitting, and improve the model’s generalization performance. In addition, experiment results on the Indian Pines, University of Pavia, Tea, and Salinas datasets reveal that the proposed MFDFN framework outperforms current deep-learning-based methods in terms of comprehensive classification effectiveness evaluation.

Multiscale spectral-spatial feature learning for hyperspectral image classification

Hyperspectral image (HSI) classification is a prevalent topic in the remote sensing image processing community. Recently, deep learning has been successfully applied to this area. However, there is still room for improvement. Since HSIs provide rich spectral information while being prone to spectral heterogeneity that damages the classification accuracy, we propose a multiscale spectral-spatial feature learning network (MulNet), which aptly handles the information given by HSIs. Our model is a hybrid model combined with a 3-Dimensional Residual Network (3DResNet), a Feature Fusion Module (FFM), and a Recurrent Neural Network (RNN). 3DResNet encodes the original HSIs and learns local spectral-spatial features at multiple scales, which are upsampled by different ratios and aggregated by FFM. Afterward, the fused features are fed sequentially to the RNN, which exploits HSI’s relations and broad contexts to produce discriminative features for better classification. Experiments on five real-world datasets using random and disjointed samples demonstrate the efficacy and efficiency of the proposed networks. It outperforms several classic and newly published spectral-spatial classifiers for HSIs.

Revisiting Deep Hyperspectral Feature Extraction Networks via Gradient Centralized Convolution

The hyperspectral images are composed of a variety of textures across the different bands which increase the spectral similarity and make it difficult to predict the pixel-wise labels without inducing additional complexity at the feature level. To extract robust and discriminative features from the different regions of land cover, the hyperspectral research community is still seeking such type of convolutions which can efficiently deal with fine-grained texture information during the feature extraction phase, which often overlook this aspect by vanilla convolution. To overcome the above shortcoming, this article proposes a generalized gradient centralized 3D convolution ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G2C-Conv3D</monospace> ) operation, which is a weighted combination between the vanilla and gradient centralized 3D convolutions ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GC-Conv3D</monospace> ) to extract both the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">intensity-level</i> semantic information and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">gradient-level</i> information. This can be easily plugged into the existing HSI feature extraction networks to boost the performance of accurate prediction for land-cover types. To validate the feasibility of the proposed <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G2C-Conv3D</monospace> , we have considered the existing CNN3D, MS3DNet, ContextNet, and SSRN feature extraction models and as well as CAE3D, VAE3D, and SAE3D autoencoder (AE) networks, respectively. All these networks are embedded with <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G2C-Conv3D</monospace> convolution to implement both generalized gradient centralized feature extraction networks (G2C-FE) and generalized gradient centralized AE networks (G2C-AE) for fine-grained spectral–spatial feature learning. In addition, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G2C-Conv2D</monospace> is also considered with few networks. The extensive experiments are conducted on four most widely used hyperspectral datasets i.e., IP, KSC, UH, and UP, respectively, and compared with the nine methods. The results demonstrate that the proposed <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G2C-Conv3D</monospace> can effectively enhance the feature learning ability of the existing networks and both the qualitative and quantitative results show the superiority and effectiveness of the proposed <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G2C-Conv3D</monospace> . The source codes will be publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/danfenghong/G2C-Conv3D-HSI</uri> .

Fully Group Convolutional Neural Networks for Robust Spectral–Spatial Feature Learning

Convolutional neural network (CNN) has been widely applied in hyperspectral image (HSI) classification exhibiting excellent performance. Weak generalization of CNN models to different datasets is a common issue in this domain largely because of limited amount of labeled training samples. In this article, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fully</i> group convolutional neural network (FGCNN) method that integrates cascades of shuffled group convolutions tailored to different network stages. To our knowledge, this is the first reported full-group CNN model in general, and we design it in particular for robust spectral–spatial classification of HSI. In the primary feature extraction stage, we develop an original multiscale spectral feature extraction approach based on a novel concept of multikernel depthwise convolution that we define in terms of shuffled and importance-weighted group convolution. In the subsequent stage, we introduce a discriminative spectral–spatial feature extraction method with a novel group competition block to capture informative features with relatively few parameters. The final feature fusion stage is defined as a novel lightweight group feature fusion method that sharply reduces fusion weights compared to traditional methods with fully connected layers. Experimental results on three datasets show that the proposed FGCNN yields robust classification accuracy under the same hyperparameter settings compared to the current state-of-the-art.

A Global Spectral–Spatial Feature Learning Network for Semisupervised Hyperspectral Unmixing

Neural networks have been widely applied in hyperspectral unmixing in the past few years. However, most networks only focus on extracting the spectral information or local spectral–spatial correlation of a single pixel. In order to further explore the spectral–spatial information of hyperspectral images (HSIs) and improve the effective use of spatial information in unmixing tasks, an end-to-end semisupervised unmixing network is proposed in this article. At the encoder, the dual branch network performs convolution operations along the spatial dimension and the spectral dimension, respectively, to realize the separated extraction of global spatial and spectral information; the spectral–spatial attention residual module then fuses the separated spectral–spatial features into the joint spectral–spatial information. The encoder is to realize mapping from high-dimensional spectral data to latent abundance representations. The decoder is a simple linear fully connected layer whose weight is fixed to a known spectral library to complete the reconstruction of the HSI. Experiments on two synthetic datasets and two real datasets demonstrate the superior performance of the proposed method, and prove the effectiveness of considering global spatial information in the unmixing task.

An attention-driven convolutional neural network-based multi-level spectral–spatial feature learning for hyperspectral image classification

Recently, convolutional neural networks (CNNs) are successfully applied to extract abstract features of hyperspectral image (HSI), and they obtained competitive performances in HSI classification. However, HSI has inhomogeneous pixels or inherent spectral correlation, and the classification performance of CNN on HSI data will be degraded by modeling all information with equal importance. To address the above issues, we propose an attention mechanism-based method termed multi-level feature network with spectral–spatial attention model (MFNSAM), which consists of a multi-level feature CNN (MFCNN) and a spectral–spatial attention module (SSAM). Due to rich spectral information and spatial distribution in HSI data, MFCNN is employed as multi-scale fusion architecture to bridge the gaps between multi-level features. Specifically, the MFCNN extracts diverse information by compounding the representations generated by each tunnel of multi-scale filter group. To improve the representational capacity in spatial and spectral domains, the channel-wise attention branch is exploited to suppress redundant spectral information, and the spatial-wise attention is designed to explore the contextual information for better refinement. Thus, the SSAM is formed by merging the two branches to adaptively recalibrate the nonlinear interdependence of deep spectral–spatial features. Experiments on University of Pavia, Heihe, and Kennedy Space Center hyperspectral data sets demonstrate that the proposed model provide competitive results to state-of-the-art methods.

Adaptive hybrid attention network for hyperspectral image classification

Abstract The task of land cover classification from hyperspectral images (HSI) has recently witnessed sharp improvements with the applications of deep convolutional networks (CNN). This is mainly attributed to the hierarchical spectral-spatial feature learning capabilities of the CNN models. However, it is important to encode the short to long range spatial dependencies to predict the pixelwise labels without inducing additional redundancy at the feature level. The standard CNN based land cover classification models for HSI overlook this aspect in general. To overcome this, we propose a hybrid attention based 3D classification model for hyperspectral images. Our model comprises of 1D and 2D CNNs to individually generate the attention masks that respectively highlight the spectral and spatial characteristics of our input image. For optimal learning the spectral and spatial features in our model are combined adaptively with the original input and sent to the 3D classification module. To enhance the classification performance, we incorporate classwise Wasserstein loss alongwith the crossentropy loss. Our methods are evaluated on three widely used hyperspectral datasets: Houston datasets (DFC-2013 and DFC-2018) and Salinas dataset, and has satisfactorily outperformed all the prior benchmark methods.

Minimum class variance broad learning system for hyperspectral image classification

A new machine learning method named as a broad learning system (BLS) has been proposed recently. The advantage of simple, fast, and good generalisation ability make it attracting extensive attention. In this study, by introducing BLS to solving hyperspectral image (HSI) classification, a minimum class variance BLS (MCVBLS) was proposed. Firstly, in order to get spectral–spatial representation of original HSI, spectral–spatial feature learning has been performed to take full advantage of abundant spectral and spatial information of HSI. Then, the authors use MCVBLS to classify the extracted spectral–spatial features. MCVBLS, in contrast to BLS, fully considers the global data structure and discriminant information of the data. MCVBLS enhances the classification performance model by minimising the intra-class distribution structure while maximising the inter-class discriminant information, the measure of placing restrictions on output weights will take more discriminative information and global discriminative structure information into consideration. Conducting an experiment on three benchmark hyperspectral datasets, they demonstrate that the proposed MCVBLS methods are effective for HSI classification, better than other state-of-the-art methods.

Content-Guided Convolutional Neural Network for Hyperspectral Image Classification

Convolutional neural networks (CNNs) are of great interest and have demonstrated remarkable performance in hyperspectral images (HSIs) classification. However, due to the current configuration of the convolution layers with a fixed kernel shape, regular CNNs are inherently limited in modeling the diverse land-cover structures, particularly in the cross-classes edge regions, where irregular class boundaries would lead to high classification errors. To address this issue, we propose a content-guided CNN (CGCNN) for HSI classification. Compared with the shape-fixed kernel in the traditional CNN, the proposed content-guided convolution adaptively adjusts its kernel shape according to the spatial distribution of land covers. The content pattern is reflected by a latent guide map automatically learned from HSI. Such content-adaptive kernel with CGCNN could suppress the irregularity and unexpected features in class boundaries and, thus, improve the feature learning in cross-classes regions. Based on the content-guided convolution, a novel guided feature extraction unit (GFEU) is constructed for spectral–spatial feature learning of HSI. Finally, the CGCNN classification framework is established by stacking multiple GFEUs with dense connection, which is helpful for mitigating the gradient vanishing and increasing the robustness to overfitting. Extensive experiments on several HSIs demonstrate that the proposed approach possesses great details’ preserving ability and its performance outperforms other state-of-the-art methods.

Residual Spectral–Spatial Attention Network for Hyperspectral Image Classification

In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral-spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral-spatial feature learning. Third, a sequential spectral-spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC).

Three-Dimensional ResNeXt Network Using Feature Fusion and Label Smoothing for Hyperspectral Image Classification.

In recent years, deep learning methods have been widely used in the hyperspectral image (HSI) classification tasks. Among them, spectral-spatial combined methods based on the three-dimensional (3-D) convolution have shown good performance. However, because of the three-dimensional convolution, increasing network depth will result in a dramatic rise in the number of parameters. In addition, the previous methods do not make full use of spectral information. They mostly use the data after dimensionality reduction directly as the input of networks, which result in poor classification ability in some categories with small numbers of samples. To address the above two issues, in this paper, we designed an end-to-end 3D-ResNeXt network which adopts feature fusion and label smoothing strategy further. On the one hand, the residual connections and split-transform-merge strategy can alleviate the declining-accuracy phenomenon and decrease the number of parameters. We can adjust the hyperparameter cardinality instead of the network depth to extract more discriminative features of HSIs and improve the classification accuracy. On the other hand, in order to improve the classification accuracies of classes with small numbers of samples, we enrich the input of the 3D-ResNeXt spectral-spatial feature learning network by additional spectral feature learning, and finally use a loss function modified by label smoothing strategy to solve the imbalance of classes. The experimental results on three popular HSI datasets demonstrate the superiority of our proposed network and an effective improvement in the accuracies especially for the classes with small numbers of training samples.

A Superpixel-Based Relational Auto-Encoder for Feature Extraction of Hyperspectral Images

Filter banks transferred from a pre-trained deep convolutional network exhibit significant performance in heightening the inter-class separability for hyperspectral image feature extraction, but weakening the intra-class consistency simultaneously. In this paper, we propose a new superpixel-based relational auto-encoder for cohesive spectral–spatial feature learning. Firstly, multiscale local spatial information and global semantic features of hyperspectral images are extracted by filter banks transferred from the pre-trained VGG-16. Meanwhile, we utilize superpixel segmentation to construct the low-dimensional manifold embedded in the spectral domain. Then, representational consistency constraint among each superpixel is added in the objective function of sparse auto-encoder, which iteratively assist and supervisedly learn hidden representation of deep spatial feature with greater cohesiveness. Superpixel-based local consistency constraint in this work not only reduces the computational complexity, but builds the neighborhood relationships adaptively. The final feature extraction is accomplished by collaborative encoder of spectral–spatial feature and weighting fusion of multiscale features. A large number of experimental results demonstrate that our proposed method achieves expected results in discriminant feature extraction and has certain advantages over some existing methods, especially on extremely limited sample conditions.

Active Transfer Learning Network: A Unified Deep Joint Spectral–Spatial Feature Learning Model for Hyperspectral Image Classification

Deep learning has recently attracted significant attention in the field of hyperspectral images (HSIs) classification. However, the construction of an efficient deep neural network (DNN) mostly relies on a large number of labeled samples being available. To address this problem, this paper proposes a unified deep network, combined with active transfer learning that can be well-trained for HSIs classification using only minimally labeled training data. More specifically, deep joint spectral-spatial feature is first extracted through hierarchical stacked sparse autoencoder (SSAE) networks. Active transfer learning is then exploited to transfer the pre-trained SSAE network and the limited training samples from the source domain to the target domain, where the SSAE network is subsequently fine-tuned using the limited labeled samples selected from both source and target domain by corresponding active learning strategies. The advantages of our proposed method are threefold: 1) the network can be effectively trained using only limited labeled samples with the help of novel active learning strategies; 2) the network is flexible and scalable enough to function across various transfer situations, including cross-dataset and intra-image; 3) the learned deep joint spectral-spatial feature representation is more generic and robust than many joint spectral-spatial feature representation. Extensive comparative evaluations demonstrate that our proposed method significantly outperforms many state-of-the-art approaches, including both traditional and deep network-based methods, on three popular datasets.

Hyperspectral image classification using multi-feature fusion

Traditional hyperspectral image (HSI) classification methods typically use the spectral features and do not make full use of the spatial or other features of the HSI. To address this problem, this paper proposes a novel HSI classification method based on a multi-feature fusion strategy. The spectral-spatial features are first extracted by spectral-spatial feature learning (SSFL), which is a deep hierarchical architecture. Additionally, the texture features of the local binary pattern (LBP) image are applied and fused with the spectral-spatial features. Then, the kernel extreme learning machine (KELM) is used to classify the hyperspectral images. The results of a number of experiments show that the proposed method effectively improves the classification accuracy of hyperspectral images.

Unsupervised Spectral–Spatial Feature Learning via Deep Residual Conv–Deconv Network for Hyperspectral Image Classification

Supervised approaches classify input data using a set of representative samples for each class, known as training samples . The collection of such samples is expensive and time demanding. Hence, unsupervised feature learning, which has a quick access to arbitrary amounts of unlabeled data, is conceptually of high interest. In this paper, we propose a novel network architecture, fully Conv–Deconv network, for unsupervised spectral–spatial feature learning of hyperspectral images, which is able to be trained in an end-to-end manner. Specifically, our network is based on the so-called encoder–decoder paradigm, i.e., the input 3-D hyperspectral patch is first transformed into a typically lower dimensional space via a convolutional subnetwork (encoder), and then expanded to reproduce the initial data by a deconvolutional subnetwork (decoder). However, during the experiment, we found that such a network is not easy to be optimized. To address this problem, we refine the proposed network architecture by incorporating: 1) residual learning and 2) a new unpooling operation that can use memorized max-pooling indexes. Moreover, to understand the “black box,” we make an in-depth study of the learned feature maps in the experimental analysis. A very interesting discovery is that some specific “neurons” in the first residual block of the proposed network own good description power for semantic visual patterns in the object level, which provide an opportunity to achieve “free” object detection. This paper, for the first time in the remote sensing community, proposes an end-to-end fully Conv–Deconv network for unsupervised spectral–spatial feature learning. Moreover, this paper also introduces an in-depth investigation of learned features. Experimental results on two widely used hyperspectral data, Indian Pines and Pavia University, demonstrate competitive performance obtained by the proposed methodology compared with other studied approaches.

Bidirectional-Convolutional LSTM Based Spectral-Spatial Feature Learning for Hyperspectral Image Classification

This paper proposes a novel deep learning framework named bidirectional-convolutional long short term memory (Bi-CLSTM) network to automatically learn the spectral-spatial features from hyperspectral images (HSIs). In the network, the issue of spectral feature extraction is considered as a sequence learning problem, and a recurrent connection operator across the spectral domain is used to address it. Meanwhile, inspired from the widely used convolutional neural network (CNN), a convolution operator across the spatial domain is incorporated into the network to extract the spatial feature. In addition, to sufficiently capture the spectral information, a bidirectional recurrent connection is proposed. In the classification phase, the learned features are concatenated into a vector and fed to a Softmax classifier via a fully-connected operator. To validate the effectiveness of the proposed Bi-CLSTM framework, we compare it with six state-of-the-art methods, including the popular 3D-CNN model, on three widely used HSIs (i.e., Indian Pines, Pavia University, and Kennedy Space Center). The obtained results show that Bi-CLSTM can improve the classification performance by almost 1.5 % as compared to 3D-CNN.