Features Of Hyperspectral Image Research Articles

Fusing Multiple Deep Models for In Vivo Human Brain Hyperspectral Image Classification to Identify Glioblastoma Tumor

Glioblastoma (GBM) tumor is the most common primary brain malignant tumor. The precise identification of GBM tumors is very important for diagnosis and treatment. Hyperspectral imaging is a fast, noncontact, accurate, and safe modern medical detection technology, which is expected to be a new tool of intraoperative diagnosis. In order to make full use of the spectral and spatial information of hyperspectral images (HSIs) to achieve accurate GBM tumor identification, a method based on the fusion of multiple deep models is proposed for in vivo human brain HSI classification. The proposed method includes the following major steps: 1) spectral phasor analysis and data oversampling; 2) 1-D deep neural network (1D-DNN)-based spectral HSI feature extraction and classification; 3) 2-D convolution neural network (2D-CNN)-based spectral–spatial HSI feature extraction and classification; 4) edge-preserving filtering-based classification result fusion and optimization; and 5) fully convolutional network (FCN)-based background segmentation. To verify the capabilities of the proposed method, experiments are performed on two real human brain hyperspectral datasets, including 36 in vivo HSIs captured from 16 different patients. The proposed method can achieve an overall accuracy of 96.69% for four-class classification and overall accuracy of 96.34% for GBM tumor identification. Experimental results demonstrate that the proposed method exhibits competitive classification performance and can generate satisfactory thematic maps of the location of the GBM tumor, which can provide the surgeon with guidance on successful and precise tumor resection.

Segment-Based Clustering of Hyperspectral Images Using Tree-Based Data Partitioning Structures

Hyperspectral image classification has been increasingly used in the field of remote sensing. In this study, a new clustering framework for large-scale hyperspectral image (HSI) classification is proposed. The proposed four-step classification scheme explores how to effectively use the global spectral information and local spatial structure of hyperspectral data for HSI classification. Initially, multidimensional Watershed is used for pre-segmentation. Region-based hierarchical hyperspectral image segmentation is based on the construction of Binary partition trees (BPT). Each segmented region is modeled while using first-order parametric modelling, which is then followed by a region merging stage using HSI regional spectral properties in order to obtain a BPT representation. The tree is then pruned to obtain a more compact representation. In addition, principal component analysis (PCA) is utilized for HSI feature extraction, so that the extracted features are further incorporated into the BPT. Finally, an efficient variant of k-means clustering algorithm, called filtering algorithm, is deployed on the created BPT structure, producing the final cluster map. The proposed method is tested over eight publicly available hyperspectral scenes with ground truth data and it is further compared with other clustering frameworks. The extensive experimental analysis demonstrates the efficacy of the proposed method.

A multi-strategy integrated multi-objective artificial bee colony for unsupervised band selection of hyperspectral images

Abstract As the spectral dimension of hyperspectral images increases, band selection becomes more and more important when using hyperspectral data. Evolutionary algorithms have been applied to the band selection problem of hyperspectral images, but most of existing methods focus only on single indicator, ignoring the whole characteristics of hyperspectral image. In this paper we study a multi-objective artificial bee colony approach for the band selection problem of hyperspectral images. Firstly, a new multi-objective unsupervised band selection model is proposed by using both band correlation and information amount. Secondly, a multi-strategy integrated multi-objective artificial bee colony algorithm (MABC-BS) is proposed to deal with the band selection model above. Several new operators, including the multi-direction search strategy, the x-space crowing degree-based search strategy, and the adaptive mutation, are developed to enhance the proposed algorithm. Compared with eight representative algorithms on three typical test problems, experimental results on the classification performance of four classifiers (i.e., Random Forest, SVM, KNN, ObRaF) show that the proposed algorithm is a powerful approach for tackling the problem of hyperspectral band selection.

TDSSC: A Three-Directions Spectral–Spatial Convolution Neural Network for Hyperspectral Image Change Detection

Change detection (CD) is a hot issue in the research of remote sensing technology. Hyperspectral images (HSIs) greatly promote the development of CD technology because of their high resolution in the spectral domain. However, some traditional CD methods currently applied to low-dimensional and multispectral images cannot adapt to the complex high-dimensional features of the HSIs. In addition, the spectral measurements of the HSI contain a lot of noise and redundancy, which greatly contaminates spectral-only information for CD. In order to fully extract the discriminant features of HSI to improve the accuracy of CD, this article proposes a three-directions spectral–spatial convolution neural network (TDSSC). A novel method for three-direction decomposition of hyperspectral change tensors is proposed—change tensor is decomposed along the spectral direction and two spatial directions to get a single tensor containing the spectral information and two kinds of tensors containing the spectral–spatial information. TDSSC uses 1-D convolution to extract spectral features from the spectral direction as well as reducing the tensor dimension, which helps the latter network to be lightweight and significantly improves the speed of change detection. Also, it uses 2-D convolution to extract spectral–spatial features from two spatial directions of the reduced tensor, and to extract features from different directions to improve the accuracy and Kappa value of CD. The experimental results of three real hyperspectral datasets show that TDSSC is superior to most existing CD methods.

Research on vision navigation and position system of agricultural unmanned aerial vehicle

ABSTRACT Precision agriculture requires the allocation of the minimum amount of resources to each piece of land in order to maximize the benefit while having a minimal economic and environmental impact. Based on this principle, precision pesticide spraying for farmland pests requires precise identification of plant features and in-flight management. The spraying rate spray needs to be accurate and uniform and targeted at the plant regions that suffer from disease and pests. The intelligent spraying technology is also used. In this paper, the navigation needs of a UAV (unmanned aerial vehicle) are studied. Hyperspectral imaging is used for field crops to combine plant location data with visual data for precision management of the crop spraying process. GPS (Global Positioning System) positioning and hyperspectral image features such as crop growth status and nutritional status are combined to achieve human-computer intelligent spraying visual servo control.

Burn wound assessment system using near-infrared hyperspectral imaging and deep transfer features

The instant and accurate burn assessment is very important for burn wound treatment. But due to the limited burn wound samples, the accurate classification of burn wounds is difficult. In this work, a full-field burn wound classification system is proposed using the near-infrared hyperspectral imaging (NIHSI) with the deep transfer features. The convolutional neural network (CNN) is applied to establish a nonlinear relationship between the hyperspectral image data and the burn severity. And the transfer learning strategy is used to boost the high classification accuracy. The burn wounds are created on bilateral sides of the abdomen of a bama miniature pig. The burn depth model (BdasNet) is pre-trained based on side A, and then fine-tuned with data of side B. The proposed model BdasNet can get the classification accuracy more than 96% for burn depth, which demonstrated the proposed method can perform an accurate classification of burn wounds and provide more practical references for clinicians.

Hyperspectral Image Classification Using Feature Relations Map Learning

Recently, deep learning has been reported to be an effective method for improving hyperspectral image classification and convolutional neural networks (CNNs) are, in particular, gaining more and more attention in this field. CNNs provide automatic approaches that can learn more abstract features of hyperspectral images from spectral, spatial, or spectral-spatial domains. However, CNN applications are focused on learning features directly from image data—while the intrinsic relations between original features, which may provide more information for classification, are not fully considered. In order to make full use of the relations between hyperspectral features and to explore more objective features for improving classification accuracy, we proposed feature relations map learning (FRML) in this paper. FRML can automatically enhance the separability of different objects in an image, using a segmented feature relations map (SFRM) that reflects the relations between spectral features through a normalized difference index (NDI), and it can then learn new features from SFRM using a CNN-based feature extractor. Finally, based on these features, a classifier was designed for the classification. With FRML, our experimental results from four popular hyperspectral datasets indicate that the proposed method can achieve more representative and objective features to improve classification accuracy, outperforming classifications using the comparative methods.

Adoption of Machine Learning in Intelligent Terrain Classification of Hyperspectral Remote Sensing Images

To overcome the difficulty of automating and intelligently classifying the ground features in remote-sensing hyperspectral images, machine learning methods are gradually introduced into the process of remote-sensing imaging. First, the PaviaU, Botswana, and Cuprite hyperspectral datasets are selected as research subjects in this study, and the objective is to process remote-sensing hyperspectral images via machine learning to realize the automatic and intelligent classification of features. Then, the basic principles of the support vector machine (SVM) and extreme learning machine (ELM) classification algorithms are introduced, and they are applied to the datasets. Next, by adjusting the parameter estimates using a restricted Boltzmann machine (RBM), a new terrain classification model of hyperspectral images that is based on a deep belief network (DBN) is constructed. Next, the SVM, ELM, and DBN classification algorithms for hyperspectral image terrain classification are analysed and compared in terms of accuracy and consistency. The results demonstrate that the average detection accuracies of ELM on the three datasets are 89.54%, 96.14%, and 96.28%, and the Kappa coefficient values are 0.832, 0.963, and 0.924; the average detection accuracies of SVM are 88.90%, 92.11%, and 91.68%, and the Kappa coefficient values are 0.768, 0.913, and 0.944; the average detection accuracies of the DBN classification model are 92.36%, 97.31%, and 98.84%, and the Kappa coefficient values are 0.883, 0.944, and 0.972. The results also demonstrate that the classification accuracy of the DBN algorithm exceeds those of the previous two methods because it fully utilizes the spatial and spectral information of hyperspectral remote-sensing images. In summary, the DBN algorithm that is proposed in this study has high application value in object classification for remote-sensing hyperspectral images.

Semisupervised collaborative representation graph embedding for hyperspectral imagery

Graph embedding (GE) frameworks are used for extracting the discriminative features of hyperspectral images (HSIs). However, it is difficult to select a proper neighborhood size for graph construction. To overcome this difficulty, a semisupervised feature extraction (FE) method, called semisupervised collaborative representation graph embedding (SCRGE), is proposed. The proposed algorithm utilizes collaborative representation (CR) to obtain the collaborative coefficients of labeled and unlabeled samples. Then, a semisupervised graph is constructed using the collaborative coefficients of the labeled samples within the same class and the collaborative coefficients of the unlabeled samples, and an interclass graph is constructed using the collaborative coefficients of the labeled samples in different classes. Finally, a projection matrix for FE is obtained by embedding these graphs into a low-dimensional space. SCRGE not only inherits the merits of CR to reveal the collaborative reconstructive properties of data but also enhances intraclass compactness and interclass separability to improve the discriminating power for classification. Experimental results on three real HSIs datasets demonstrate that SCRGE outperforms other state-of-the-art FE methods in terms of classification accuracy.

Fusion of Hyperspectral CASI and Airborne LiDAR Data for Ground Object Classification through Residual Network.

Modern satellite and aerial imagery outcomes exhibit increasingly complex types of ground objects with continuous developments and changes in land resources. Single remote-sensing modality is not sufficient for the accurate and satisfactory extraction and classification of ground objects. Hyperspectral imaging has been widely used in the classification of ground objects because of its high resolution, multiple bands, and abundant spatial and spectral information. Moreover, the airborne light detection and ranging (LiDAR) point-cloud data contains unique high-precision three-dimensional (3D) spatial information, which can enrich ground object classifiers with height features that hyperspectral images do not have. Therefore, the fusion of hyperspectral image data with airborne LiDAR point-cloud data is an effective approach for ground object classification. In this paper, the effectiveness of such a fusion scheme is investigated and confirmed on an observation area in the middle parts of the Heihe River in China. By combining the characteristics of hyperspectral compact airborne spectrographic imager (CASI) data and airborne LiDAR data, we extracted a variety of features for data fusion and ground object classification. Firstly, we used the minimum noise fraction transform to reduce the dimensionality of hyperspectral CASI images. Then, spatio-spectral and textural features of these images were extracted based on the normalized vegetation index and the gray-level co-occurrence matrices. Further, canopy height features were extracted from airborne LiDAR data. Finally, a hierarchical fusion scheme was applied to the hyperspectral CASI and airborne LiDAR features, and the fused features were used to train a residual network for high-accuracy ground object classification. The experimental results showed that the overall classification accuracy was based on the proposed hierarchical-fusion multiscale dilated residual network (M-DRN), which reached an accuracy of 97.89%. This result was found to be 10.13% and 5.68% higher than those of the convolutional neural network (CNN) and the dilated residual network (DRN), respectively. Spatio-spectral and textural features of hyperspectral CASI images can complement the canopy height features of airborne LiDAR data. These complementary features can provide richer and more accurate information than individual features for ground object classification and can thus outperform features based on a single remote-sensing modality.

Unsupervised noise-robust feature extraction for aerial image classification

The rich data provided by satellites and unmanned aerial vehicles bring opportunities to directly model aerial image features by extracting their spatial and structural patterns. Although convolutional autoencoders (CAEs) have been attained a remarkable performance in ideal aerial image feature extraction, they are still challenging to extract information from noisy images which are generated from capture and transmission. In this paper, a novel CAE-based noise-robust unsupervised learning method is proposed for extracting high-level features accurately from aerial images and mitigating the effect of noise. Different from conventional CAEs, the proposed method introduces the noise-robust module between the encoder and the decoder. Besides, several pooling layers in CAEs are replaced by convolutional layers with stride=2. The performance of feature extraction is evaluated by the prediction accuracy and the accuracy loss in image classification experiments. A 5-classes aerial optical scene and a 9-classes hyperspectral image (HSI) data set are utilized for optical image and HSI feature extraction, respectively. High-level features extracted from aerial images are utilized for image classification by a linear support vector machine (SVM) classifier. Experimental results indicate that the proposed method improves the classification accuracy for noisy images (Gaussian noise 2D σ =0.1, 3D σ =60) in both optical images (2D 87.5%) and HSIs (3D 85.6%) compared with the traditional CAE (2D 78.6%, 3D 84.2%). The accuracy loss in classification experiments increases with the increment of noise. Compared with the traditional CAE (2D 15.7%, 3D 11.8%), the proposed method shows the lower classification accuracy loss in experiments (2D 0.3%, 3D 6.3%). The proposed unsupervised noise-robust feature extraction method attains desirable classification accuracy in ideal input and enhances the feature extraction capability from noisy input.

Subpixel-Pixel-Superpixel-Based Multiview Active Learning for Hyperspectral Images Classification

Active learning (AL) attempts to actively select the most representative or useful training samples in an iterative manner. The aim is to simultaneously improve the classification performance and reduce the manual labeling effort. In this article, a novel subpixel-pixel-superpixel-based multiview AL (MAL) (SPS-MAL) method is proposed for hyperspectral image (HSI) classification. Here, the multiple views are generated via extracting the subpixel-level, pixel-level, and superpixel-level information. The multiple views can reflect various characteristics of HSI, i.e., spectral mixture, spectral discrimination, and spectral–spatial structure. Therefore, the joint use of diverse and complementary information in multiple views will contribute to a better identification ability of different classes. In addition, a coarse-to-fine MAL algorithm is introduced to effectively select the most representative samples with the most uncertainty. Specifically, a disagreement analysis on multiple views and joint posterior probability estimation is used to query unlabeled samples. Along with the expansion of training samples, view-specific confidence scores are estimated to adaptively integrate the classification results of multiple views, according to their discrimination performance. In this way, the classification accuracy will be further boosted while the number of necessary training samples can be significantly reduced. The experimental classification results on three well-known HSIs demonstrate the effectiveness of the proposed SPS-MAL method.

Hyperspectral Image Classification of Convolutional Neural Network Combined with Valuable Samples

Aiming at the problem that the manual labeling of samples in the hyperspectral image classification is expensive and laborious, a large number of unlabeled samples are not effectively utilized and the classification results are not ideal. A method which can provide valuable samples and employ convolutional neural network to extract spectral spatial features for classification is proposed. Active learning method is used to construct a valuable training sample set by iteratively selecting the most uncertain samples through support vector machine which performs well in small sample classification, and labeling them. Then the 3D convolutional neural network is used to extract the spectral spatial features of hyperspectral image. The experimental results of the hyperspectral classification on Indian Pines and PaviaU datasets show that the proposed method (3D VS-CNN) is better than traditional classification methods.

Multiscale dual-level network for hyperspectral image classification

We propose a new dual-level convolutional neural network model based on Inception modules and residual connections. First, Inception has filters with different kernel sizes, and its output feature maps contain different scales of receptive fields. The feature map with the wide receptive field receives the global information, while the feature map with the small receptive field contains some local information. The multiscale feature provides more comprehensive information. Second, with the help of residual connections, the training process is simple and can avoid overfitting. Third, the proposed network adopts two levels, i.e., a low level and a high level, and uses the feature fusion operation to take full advantage of the complementary and correlated information of the two levels. Fourth, we combine the spatial features and the spectral features of hyperspectral image (HSI). The pixels to be classified with their neighborhood information serve as the input of the neural network to realize spectral–spatial classification for HSI. Experimental results show that our model performs better than other state-of-the-art methods.

A deep feature manifold embedding method for hyperspectral image classification

ABSTRACT In this letter, we proposed a novel deep feature manifold embedding method to improve feature extraction ability of traditional deep learning methods. This method first obtains deep features of hyperspectral image (HSI) from a trained autoencoder. Then, an intrinsic graph and a penalty graph are constructed to discover the discriminant manifold structure of deep features. Finally, the deep features are mapped into a low-dimensional embedding space, in which samples in intraclass manifold are compacted and samples from interclass manifolds are separated. Experiments on Pavia University, Indian Pines and Urban datasets demonstrate that the proposed method effectively improves the classification performance of HSI compared with other state-of-the-art approaches.

Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution

Deep learning-based hyperspectral image super-resolution (SR) methods have achieved great success recently. However, there are two main problems in the previous works. One is to use the typical three-dimensional convolution analysis, resulting in more parameters of the network. The other is not to pay more attention to the mining of hyperspectral image spatial information, when the spectral information can be extracted. To address these issues, in this paper, we propose a mixed convolutional network (MCNet) for hyperspectral image super-resolution. We design a novel mixed convolutional module (MCM) to extract the potential features by 2D/3D convolution instead of one convolution, which enables the network to more mine spatial features of hyperspectral image. To explore the effective features from 2D unit, we design the local feature fusion to adaptively analyze from all the hierarchical features in 2D units. In 3D unit, we employ spatial and spectral separable 3D convolution to extract spatial and spectral information, which reduces unaffordable memory usage and training time. Extensive evaluations and comparisons on three benchmark datasets demonstrate that the proposed approach achieves superior performance in comparison to existing state-of-the-art methods.

Transferring CNN Ensemble for Hyperspectral Image Classification

In recent years, deep convolutional neural networks (CNNs) have been widely investigated for hyperspectral image (HSI) classification. The CNN-based HSI classifiers obtained good performance under the condition of sufficient training samples. In order to address the problem of limited training samples, in this letter, transfer learning is combined with CNN to address the issue of HSI classification. Pretrained models on large-scale data sets (e.g., ImageNet) can extract the general and discriminative features. Due to the fact that the extracted low-level and mid-level features can be reused for the HSI feature extraction, the CNN-based methods usually obtain good classification performance with insufficient training samples. The ImageNet data set has three channels, while the HSI data set contains hundreds of channels. Therefore, three channels of the HSI are randomly selected to formulate a transferring CNN. Then, several transferring CNNs are combined to establish an ensemble classification system with diversity. Moreover, an improved label smoothing technique is proposed to further improve the classification accuracy of the HSI. Experimental results on two popular hyperspectral data sets [i.e., Indian Pines and Kennedy Space Center (KSC)] show that the transferring CNN ensemble obtains good classification performance compared to the state-of-the-art methods.

A multi-scale 3D convolution neural network for spectral-spatial classification of hyperspectral imagery

Along with the remarkable achievements of various neural networks in the field of computer vision, deep learning methods have gradually been applied to hyperspectral image (HSI) classification. Traditional classification methods have several outstanding issues, including the insufficient hand-craft features and time-consuming and laborious feature extraction. We therefore propose a multi-scale, 3D convolutional neural network (CNN) framework (MSCNN), trained in an end-to-end manner for hyperspectral image classification. Using the original hyperspectral image as an input, the MSCNN framework leverages two branches of a 3D residual neural network to extract deep and abstract HSI features. Then, the HSI spectral-spatial features of different scales are fused and fed into the SoftMax layer to achieve an end-to-end hyperspectral image classification. Besides, data augmentation, dynamic learning rate, and regularization methods are used to promote the rapid convergence of MSCNN and avoid model over-fitting. Three well-known hyperspectral datasets (Indian Pines, University of Pavia and Pavia Center) were used to evaluate the classification performance of the proposed MSCNN method. The results indicate that, compared with the existing deep learning methods, the proposed MSCNN method achieves the best classification performance within the 20 training epochs. The overall accuracy (OA), average accuracy (AA), and kappa statistic (K) of MSCNN are 97.16%, 98.69% and 0.9665, respectively, for Indian Pines; 99.17%, 98.97% and 0.9888 for University of Pavia; and 99.87%, 99.70% and 0.9981 for Pavia Center.

Identification of Wheat Yellow Rust Using Spectral and Texture Features of Hyperspectral Images

Wheat yellow rust is one of the most destructive diseases in wheat production and significantly affects wheat quality and yield. Accurate and non-destructive identification of yellow rust is critical to wheat production management. Hyperspectral imaging technology has proven to be effective in identifying plant diseases. We investigated the feasibility of identifying yellow rust on wheat leaves using spectral features and textural features (TFs) of hyperspectral images. First, the hyperspectral images were preprocessed, and healthy and yellow rust-infected samples were obtained by creating regions of interest. Second, the extraction of spectral reflectance characteristics and vegetation indices (VIs) were performed from the preprocessed hyperspectral images, and the TFs were extracted using the grey-level co-occurrence matrix from the images transformed by principal component analysis. Third, the successive projections algorithm was employed to choose the optimum wavebands (OWs), and correlation-based feature selection was employed to select the optimal VIs and TFs (those most sensitive to yellow rust and having minimal redundancy between features). Finally, identification models of wheat yellow rust were established using a support vector machine and different features. Six OWs (538, 598, 689, 702, 751, and 895 nm), four VIs (nitrogen reflectance index, photochemical reflectance index, greenness index, and anthocyanin reflectance index), and four TFs (correlation 1, correlation 2, entropy 2, and second moment 3) were selected. The identification models based on the OWs, VIs, and TFs provided overall accuracies of 83.3%, 89.5%, and 86.5%, respectively. The TF results were especially encouraging. The models with the combination of spectral features and TFs exhibited better performance than those using the spectral features or TFs alone. The accuracies of the models with the combined features (OWs and TFs, Vis, and TFs) were 90.6% and 95.8%, respectively. These values were 7.3% and 6.3% higher, respectively, than those of the models using only the OWs or VIs. The model with the combined feature (VIs and TFs) had the highest accuracy (95.8%) and was used to map the yellow rust lesions on wheat leaves with different damage levels. The results showed that the yellow rust lesions on the leaves could be identified accurately. Overall, the combination of spectral features and TFs of hyperspectral images significantly improved the identification accuracy of wheat yellow rust.

Unsupervised and Supervised Feature Extraction Methods for Hyperspectral Images Based on Mixtures of Factor Analyzers

This paper proposes three feature extraction (FE) methods based on density estimation for hyperspectral images (HSIs). The methods are a mixture of factor analyzers (MFA), deep MFA (DMFA), and supervised MFA (SMFA). The MFA extends the Gaussian mixture model to allow a low-dimensionality representation of the Gaussians. DMFA is a deep version of MFA and consists of a two-layer MFA, i.e, samples from the posterior distribution at the first layer are input to an MFA model at the second layer. SMFA consists of single-layer MFA and exploits labeled information to extract features of HSI effectively. Based on these three FE methods, the paper also proposes a framework that automatically extracts the most important features for classification from an HSI. The overall accuracy of a classifier is used to automatically choose the optimal number of features and hence performs dimensionality reduction (DR) before HSI classification. The performance of MFA, DMFA, and SMFA FE methods are evaluated and compared to five different types of unsupervised and supervised FE methods by using four real HSIs datasets.