Hyperspectral Image Classification Method Research Articles

Deep CNN-based hyperspectral image classification using discriminative multiple spatial-spectral feature fusion

ABSTRACT Convolutional Neural Networks (CNNs) are widely used in various fields, and have shown good performance in hyperspectral image (HSI) classification. Recently, utilizing deep networks to learn spatial-spectral features has become of great interest. However, excessively increasing the depth of network may result in overfitting. Moreover, in HSI classification, the existing network models ignore the strong complementary yet correlated spatial-spectral information among different hierarchical layers. In order to address these two problems, a novel CNN-based method for HSI classification is proposed. Firstly, it considers fusing the outputs of recurrent two layers in each large convolutional block and thereby using the fusion result as the input of next layer, which facilitates the extraction of discriminative features. Next, the spectral-spatial features are extracted by cascading spectral features to four-scale spatial features from shallow to deep layers. Finally, a 11 convolution layer is used to interact and integrate information across channels. Without increasing the number of training samples and the size of pixel patches at the training stage, the proposed approach achieves the state-of-the-art results in the experiment on three well-known hyperspectral images.

Composite Clustering Sampling Strategy for Multiscale Spectral-Spatial Classification of Hyperspectral Images

In recent years, many high-performance spectral-spatial classification methods were proposed in the field of hyperspectral image classification. At present, a great quantity of studies has focused on developing methods to improve classification accuracy. However, some research has shown that the widely adopted pixel-based random sampling strategy is not suitable for spectral-spatial hyperspectral image classification algorithms. Therefore, a composite clustering sampling strategy is proposed, which can greatly reduce the overlap between the training set and the test set, while making sample points in the training set sufficiently representative in the spectral domain. At the same time, in order to solve problems of a three-dimensional Convolutional Neural Network which is commonly used in spectral-spatial hyperspectral image classification methods, such as long training time and large computing resource requirements, a multiscale spectral-spatial hyperspectral image classification model based on a two-dimensional Convolutional Neural Network is proposed, which effectively reduces the training time and computing resource requirements.

Spectral-spatial hyperspectral image classification based on superpixel and multi-classifier fusion

ABSTRACT Hyperspectral image classification is a challenging problem for machine learning methods due to the small number of labelled samples and high spectral variability. In this paper, to solve this problem, a novel superpixel and multi-classifier fusion (SMCF)-based classification method for hyperspectral images is proposed. This method takes full advantage of the spectral information of superpixels and the spatial information of hyperspectral images and includes the following three steps. First, superpixels are used to increase the number of training samples and their spectral diversity. Second, label propagation (LP) is used to classify the hyperspectral images. Although LP is an efficient semi-supervised classification method, the corresponding performance is poor for certain land cover types with dispersed spatial distributions. Thus, a support vector machine (SVM) classifier is introduced to classify the hyperspectral images. Finally, the results of the SVM and LP classifiers are combined using our new class-specific weighted fusion algorithm. In the experiments, we selected three widely used and real hyperspectral data sets for evaluation. The final classification performance was evaluated based on two common metrics: the overall accuracy (OA) and the Kappa coefficient. The experimental results show that the proposed SMCF method is superior to six well-known classification methods, even when only 1% or less of the labelled samples are used.

Dual unsupervised features fusion for hyperspectral image classification

ABSTRACT An unsupervised feature extraction method could effectively address the phenomenon where using a limited number of labelled samples leads to poor hyperspectral image (HSI) classification. Inspired by this method, we have developed a new method based on fusing dual unsupervised features for HSI classification in this paper. First, we used principal component analysis (PCA) to reduce the dimensions of HSI data and obtain the first set of unsupervised features. Then, we got the second set of unsupervised features by adopting the self-taught learning method, which could learn more discriminatory features by making full use of the information of unlabelled samples in the same HSI. Next, we used the correlation canonical analysis (CCA) algorithm to fuse linearly dual unsupervised learning features. Finally, we proved the effectiveness of the features extracted by the proposed algorithm by applying the extreme learning machine (ELM) classifier to evaluate the purposes. Experiments with three widely used real HSI datasets showed good classification performance when the number of training samples was quite limited. This demonstrates that the proposed feature learning method was indeed an effective method for HSI classification.

Spectral–Spatial Hyperspectral Classification via Structural-Kernel Collaborative Representation

This letter introduces a novel spatial-spectral classification method for hyperspectral images (HSIs) based on a structural-kernel collaborative representation (SKCR), which considers one weak assumption of spatial neighborhood that of the pixels in a superpixel belong to the same class when exploiting contextual information in HSI. The proposed method consists of the following steps. First, a superpixel segmentation strategy is used to construct self-adaptive regions for the HSI. Then, the structural information within each superpixel block is extracted based on the density peak and K nearest neighbors. Next, dual kernels are separately utilized for the exploitation of the spectral and the spatial information. Finally, the dual kernels are combined and incorporated into a support-vector-machine classifier. Since the weak assumption of spatial neighborhood is well considered in the collaborative representation, the proposed method showed excellent classification performance for two widely used real hyperspectral data sets even when the number of training samples was relatively small.

Hyperspectral Image Classification With Context-Aware Dynamic Graph Convolutional Network

In hyperspectral image (HSI) classification, spatial context has demonstrated its significance in achieving promising performance. However, conventional spatial context-based methods simply assume that spatially neighboring pixels should correspond to the same land-cover class, so they often fail to correctly discover the contextual relations among pixels in complex situations, and thus leading to imperfect classification results on some irregular or inhomogeneous regions such as class boundaries. To address this deficiency, we develop a new HSI classification method based on the recently proposed Graph Convolutional Network (GCN), as it can flexibly encode the relations among arbitrarily structured non-Euclidean data. Different from traditional GCN, there are two novel strategies adopted by our method to further exploit the contextual relations for accurate HSI classification. First, since the receptive field of traditional GCN is often limited to fairly small neighborhood, we proposed to capture long range contextual relations in HSI by performing successive graph convolutions on a learned region-induced graph which is transformed from the original 2D image grids. Second, we refine the graph edge weight and the connective relationships among image regions by learning the improved adjacency matrix and the 'edge filter', so that the graph can be gradually refined to adapt to the representations generated by each graph convolutional layer. Such updated graph will in turn result in accurate region representations, and vice versa. The experiments carried out on three real-world benchmark datasets demonstrate that the proposed method yields significant improvement in the classification performance when compared with some state-of-the-art approaches.

A deep feature manifold embedding method for hyperspectral image classification

ABSTRACT In this letter, we proposed a novel deep feature manifold embedding method to improve feature extraction ability of traditional deep learning methods. This method first obtains deep features of hyperspectral image (HSI) from a trained autoencoder. Then, an intrinsic graph and a penalty graph are constructed to discover the discriminant manifold structure of deep features. Finally, the deep features are mapped into a low-dimensional embedding space, in which samples in intraclass manifold are compacted and samples from interclass manifolds are separated. Experiments on Pavia University, Indian Pines and Urban datasets demonstrate that the proposed method effectively improves the classification performance of HSI compared with other state-of-the-art approaches.

Residual Spectral–Spatial Attention Network for Hyperspectral Image Classification

In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral-spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral-spatial feature learning. Third, a sequential spectral-spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC).

An Enhanced 3-D Discrete Wavelet Transform for Hyperspectral Image Classification

In the classification of hyperspectral image (HSI), there exists a common issue that the collected HSI data set is always contaminated by various noise (e.g., Gaussian, stripe, and deadline), degrading the classification results. To tackle this issue, we modify the 3-dimensional discrete wavelet transform (3DDWT) method by considering the noise effect on feature quality and propose an enhanced 3DDWT (E-3DDWT) approach to extract the feature and meanwhile alleviate the noise. Specifically, the proposed E-3DDWT method first applies classical 3DDWT method to the HSI data cube and thus can generate eight subcubes in each level. Then, the stripe noise is concentrated into several subcubes due to its spatial vertical property. Finally, we abandon these subcubes and obtain the feature cube by stacking the remaining ones. After acquiring the feature, we then adopt the convolutional neural network (CNN) model with an active learning strategy for classification since CNN has been verified to be a state-of-the-art feature extraction method for HSI classification, and active learning strategy can alleviate the insufficient labeled sample issue to some extent. In addition, we apply the Markov random field to enhance the final categorized results. Experiments on two synthetically striped data sets show that our proposed approach achieves better categorized results than other advanced methods.

A Lightweight Spectral–Spatial Feature Extraction and Fusion Network for Hyperspectral Image Classification

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

A Two-stage Deep Domain Adaptation Method for Hyperspectral Image Classification

Deep learning has attracted extensive attention in the field of hyperspectral images (HSIs) classification. However, supervised deep learning methods heavily rely on a large amount of label information. To address this problem, in this paper, we propose a two-stage deep domain adaptation method for hyperspectral image classification, which can minimize the data shift between two domains and learn a more discriminative deep embedding space with very few labeled target samples. A deep embedding space is first learned by minimizing the distance between the source domain and the target domain based on Maximum Mean Discrepancy (MMD) criterion. The Spatial–Spectral Siamese Network is then exploited to reduce the data shift and learn a more discriminative deep embedding space by minimizing the distance between samples from different domains but the same class label and maximizes the distance between samples from different domains and class labels based on pairwise loss. For the classification task, the softmax layer is replaced with a linear support vector machine, in which learning minimizes a margin-based loss instead of the cross-entropy loss. The experimental results on two sets of hyperspectral remote sensing images show that the proposed method can outperform several state-of-the-art methods.

Hyperspectral image classification method based on space-spectral fusion conditional random field

Hyperspectral image classification method based on space-spectral fusion conditional random field

Spatial-spectral hyperspectral image classification based on information measurement and CNN

In order to construct virtual land environment for virtual test, we propose a construction method of virtual land environment using multi-satellite remote sensing data, the key step of which is accurate recognition of ground object. In this paper, a method of ground object recognition based on hyperspectral image (HSI) was proposed, i.e., a HSI classification method based on information measure and convolutional neural networks (CNN) combined with spatial-spectral information. Firstly, the most important three spectra of the hyperspectral image was selected based on information measure. Specifically, the entropy and color-matching functions were applied to determine the candidate spectra sets from all the spectra of the hyperspectral image. Then three spectra with the largest amount of information were selected through the minimum mutual information. Through the above two steps, the dimensionality reduction for hyperspectral images was effectively achieved. Based on the three selected spectra, the CNN network input combined with the spatial-spectral information was designed. Two input strategies were designed: (1) The patch surrounding the pixel to be classified was directly intercepted from the grayscale images of the three selected spectra. (2) In order to highlight the effect of the spectrum of the pixel to be classified, all the spectral components of this pixel were superimposed on the patch obtained by the previous strategy. As a result, a new patch with more prominent spectral components of the pixel to be classified was obtained. Using the two public hyperspectral datasets, Salinas and Pavia Center, the experiments of on both parameter selection and classification performance were performed to verify that the proposed methods had better classification performance.

Multiple Spatial Features Extraction and Fusion for Hyperspectral Images Classification

In recent decades, spatial feature extraction has greatly improved the performance of hyperspectral image (HSI) classification. This paper presents an HSI classification method based on multiple spatial features extraction and fusion (MSFs-EF). The method consists of five sequential steps. 1- Principal component analysis is applied for HSI dimensionality reduction. 2- A mean curvature filter is used to extract the spatial texture features from the HSI. 3- The spatial correlation features are obtained using a domain transform normalized convolution filter. 4- Spatial texture features and spatial correlation features are combined. 5- The multiple spatial features are classified using the Large Margin Distribution Machine. Three hyperspectral data sets are used to verify the performance. This method improves the accuracy of HSI classification compared with SVM method, edge-preserving filter, recursive filter method, and deep learning method. In the case of ratios of training samples of 5%, 0.6%, and 5%, the overall accuracy of three data sets reaches 98.23%, 99.17%, and 98.21% respectively, and are about 1.3%∼19%, 0.2%∼13%, and 0.4%∼13% higher than other fourteen algorithms. In the case of ratios of training samples of 10%, 1%, and 10%, the overall accuracy of the three data sets reaches 98.63%, 99.53%, and 98.99%, respectively and still outperform other methods.

Combining Spectral Unmixing and 3D/2D Dense Networks with Early-Exiting Strategy for Hyperspectral Image Classification

Recently, Hyperspectral Image (HSI) classification methods based on deep learning models have shown encouraging performance. However, the limited numbers of training samples, as well as the mixed pixels due to low spatial resolution, have become major obstacles for HSI classification. To tackle these problems, we propose a resource-efficient HSI classification framework which introduces adaptive spectral unmixing into a 3D/2D dense network with early-exiting strategy. More specifically, on one hand, our framework uses a cascade of intermediate classifiers throughout the 3D/2D dense network that is trained end-to-end. The proposed 3D/2D dense network that integrates 3D convolutions with 2D convolutions is more capable of handling spectral-spatial features, while containing fewer parameters compared with the conventional 3D convolutions, and further boosts the network performance with limited training samples. On another hand, considering the existence of mixed pixels in HSI data, the pixels in HSI classification are divided into hard samples and easy samples. With the early-exiting strategy in these intermediate classifiers, the average accuracy can be improved by reducing the amount of computation cost for easy samples, thus focusing on classifying hard samples. Furthermore, for hard samples, an adaptive spectral unmixing method is proposed as a complementary source of information for classification, which brings considerable benefits to the final performance. Experimental results on four HSI benchmark datasets demonstrate that the proposed method can achieve better performance than state-of-the-art deep learning-based methods and other traditional HSI classification methods.

Multiscale Densely-Connected Fusion Networks for Hyperspectral Images Classification

Convolutional neural network (CNN) has demonstrated to be a powerful tool for hyperspectral images (HSIs) classification. Previous CNN-based HSI classification methods only adopt the fixed-size patches to train the CNN model, and such single scale patches may not reflect the complex spatial structural information in the HSIs. In addition, although different layers of CNN can extract features of multiple scales, the traditional CNN model can only utilize features from the highest level for the classification task. These features, however, do not fully consider the strong complementary yet correlated information among different layers. To address these issues, in this paper, a multiscale densely-connected convolutional network (MS-DenseNet) framework is proposed to sufficiently exploit multiple scales information for the HSIs classification. Specifically, for each pixel, the MS-DenseNet, first, extracts its surrounding patches of multiple scales. These patches can separately constitute multiple scale training and testing samples. Within each specific scale sample, instead of using the forward convolutional layers, the MS-DenseNet adopts the dense blocks, which can connect each layer to other layers in a feed-forward fashion and thus can exploit the information among different layers for training and testing. Furthermore, since high correlations exist in patches of different scales, the MS-DenseNet introduces several dense blocks to fuse the multiscale information among different layers for the final HSI classification. Experimental results on several real HSIs demonstrate the superiority of the proposed MS-DenseNet over single scale-based CNN classification model and several well-known classification methods.

A Sparse Representation-Based Sample Pseudo-Labeling Method for Hyperspectral Image Classification

Hyperspectral image classification methods may not achieve good performance when a limited number of training samples are provided. However, labeling sufficient samples of hyperspectral images to achieve adequate training is quite expensive and difficult. In this paper, we propose a novel sample pseudo-labeling method based on sparse representation (SRSPL) for hyperspectral image classification, in which sparse representation is used to select the purest samples to extend the training set. The proposed method consists of the following three steps. First, intrinsic image decomposition is used to obtain the reflectance components of hyperspectral images. Second, hyperspectral pixels are sparsely represented using an overcomplete dictionary composed of all training samples. Finally, information entropy is defined for the vectorized sparse representation, and then the pixels with low information entropy are selected as pseudo-labeled samples to augment the training set. The quality of the generated pseudo-labeled samples is evaluated based on classification accuracy, i.e., overall accuracy, average accuracy, and Kappa coefficient. Experimental results on four real hyperspectral data sets demonstrate excellent classification performance using the new added pseudo-labeled samples, which indicates that the generated samples are of high confidence.

Collaborative learning of lightweight convolutional neural network and deep clustering for hyperspectral image semi-supervised classification with limited training samples

Deep learning provides excellent potentials for hyperspectral images (HSIs) classification, but it is infamous for requiring large amount of labeled samples while the collection of high-quality labels for HSIs is extremely expensive and time-consuming. Furthermore, when the limited training samples are available, deep learning methods may suffer from over-fitting. In this work, we propose a novel collaborative learning framework for semi-supervised HSI classification with joint deep convolutional neural networks and deep clustering. Specifically, a lightweight 3D convolutional neural network (CNN) with much less parameters compared with classical 3D CNNs is designed for deep discriminative feature learning and classification. Then a deep clustering method, that is approximate rank-order clustering (AROC) algorithm, is applied to cluster deep features to generate pseudo labels for abundant unlabeled samples. Finally, we fine-tune the lightweight 3D CNN by minimizing a dual-loss (softmax loss and center loss) using both true and pseudo labels. Experimental results on three challenging HSI datasets demonstrate that the proposed method can achieve better performance than other state-of-the-art deep learning based methods and traditional HSI classification methods methods.

Multiscale CNNs Ensemble Based Self-Learning for Hyperspectral Image Classification

Fully supervised methods for hyperspectral image (HSI) classification usually require a considerable number of training samples to obtain high classification accuracy. However, it is time-consuming and difficult to collect the training samples. Under this context, semisupervised learning, which can effectively augment the number of training samples and extract the underlying information among the unlabeled samples, gained much attention. In this letter, we propose a Multiscale convolutional neural networks (CNNs) Ensemble Based Self-Learning (MCE-SL) method for semisupervised HSI classification. Generally, the proposed MCE-SL method consists of the following two stages. In the first stage, the spatial information of different scales from limited labeled training samples are extracted to train several CNN models. In the second stage, the trained multiscale CNNs are used to classify the unlabeled samples. After error correction, the problem of label partially incorrect is alleviated, and unlabeled samples with high confidence will be added to the original training data set for the next training iteration. We conduct comprehensive experiments on two real HSI data sets, and the experimental results show that the proposed MCE-SL can obtain better classification performance compared with several traditional semisupervised methods in few iterations.

Multiscale 3-D-CNN based on spatial–spectral joint feature extraction for hyperspectral remote sensing images classification

In order to make full use of the spatial–spectral information of hyperspectral images (HSIs), a multiscale three-dimensional (3-D) convolutional neural network (CNN) model is proposed to extract the features of HSIs and classify them. First, the principal component analysis method is used to reduce the dimension of the original HSI, reduce the spectrum size on the basis of retaining the majority of spectrum information, and improve the training speed of the model. Second, at present, most of the HSI classification methods are based on spectral features and ignore the pixels’ neighborhood information on each band, thus a 3-D-CNN model is proposed for convolution feature extraction using a spatial–spectral joint method, which makes full use of the spatial–spectral information of HSI and greatly improves the classification effect of the model. Finally, traditional CNN models mostly use single-size convolution kernels. Several convolution kernels of different sizes are added to the convolution layer to extract the spatial features of different sizes, which effectively improves the classification ability of the model. The experimental results show that the multiscale 3-D-CNN model designed, when used on three datasets: Indian Pines, Pavia University, and Salinas, reached 97.54%, 99.78%, and 99.24% overall classification accuracy, respectively. Compared with the traditional single-scale two-dimensional-CNN model, the overall classification accuracy is improved by about 1% to 2%, which can better complete the classification task of HSIs.