Two Dimensional Linear Discriminant Analyses for Hyperspectral Data

Feature extraction based on ridge regression (FERR) is proposed in this article. In FERR, a feature vector is defined in each spectral band using the mean of all classes in that dimension. Then, it is modelled using a linear combination of its farthest neighbours from among other defined feature vectors. The representation coefficients obtained by solving the ridge regression model compose the projection matrix for feature extraction. FERR can extract each desired number of features while the other methods such as linear discriminant analysis (LDA) and generalized discriminant analysis (GDA) have limitations in the number of extracted features. Experimental results on four popular real hyperspectral images show that the efficiency of FERR is superior to those of other supervised feature extraction methods in small sample-size situations. For example, for the Indian Pines dataset, the comparison between the highest average classification accuracies achieved by different feature extraction methods using a support vector machine (SVM) classifier and 16 training samples per class shows that FERR is 7% more accurate than nonparametric weighted feature extraction and is also 9% better than GDA. LDA, having the singularity problem in the small sample-size situations, has 40% less accuracy than FERR. The experiments show that generally the performance of FERR using the SVM classifier is better than when using the maximum likelihood classifier.

ABSTRACTThe feature extraction is an important preprocessing step of the classification procedure particularly in high-dimensional data with limited number of training samples. Conventional supervised feature extraction methods, for example, linear discriminant analysis (LDA), generalized discriminant analysis, and non-parametric weighted feature extraction ones, need to calculate scatter matrices. In these methods, within-class and between-class scatter matrices are used to formulate the criterion of class separability. Because of the limited number of training samples, the accurate estimation of these matrices is not possible. So the classification accuracy of these methods falls in a small sample size situation. To cope with this problem, a new supervised feature extraction method namely, feature extraction using attraction points (FEUAP) has been recently proposed in which no statistical moments are used. Thus, it works well using limited training samples. To take advantage of this method and LDA one, this article combines them by a dyadic scheme. In the proposed scheme, the similar classes are grouped hierarchically by the k-means algorithm so that a tree with some nodes is constructed. Then the class of each pixel is determined from this scheme. To determine the class of each pixel, depending on the node of the tree, we use FEUAP or LDA for a limited or large number of training samples, respectively. The experimental results demonstrate the better performance of the proposed hybrid method in comparison with other supervised feature extraction methods in a small sample size situation.

A new supervised feature extraction method appropriate for small sample size situations is proposed in this work. The proposed method is based on the first-order statistics, in which there is no need to estimate the scatter matrices. Thus, the presented method not only can avoid the singularity problem in small sample size situations but also can achieve high performance in such situations. In addition, due to the fact that the proposed algorithm only exploits the first order statistical moments, it is very fast making it suitable for real-time hyperspectral scene analysis. The proposed method makes a matrix whose columns are obtained by averaging training samples of different classes. Then, a new transform is used to map the features from the original space into a new low-dimensional space such that the new features are as different from each other as possible. Subsequently, to capture the inherent nonlinearity of the original data, the algorithm is improved using the kernel trick. In experiments, four widely-used hyperspectral datasets, namely, Indian Pines, University of Pavia, Salinas, and Botswana are classified. The experimental results show that the proposed algorithm achieves state-of-the-art results in small sample size situations.

Feature extraction is a very important preprocessing step for classification of hyperspectral images. The linear discriminant analysis (LDA) method fails to work in small sample size situations. Moreover, LDA has poor efficiency for non-Gaussian data. LDA is optimized by a global criterion. Thus, it is not sufficiently flexible to cope with the multi-modal distributed data. We propose a new feature extraction method in this paper, which uses the boundary semi-labeled samples for solving small sample size problem. The proposed method, which called hybrid feature extraction based on boundary semi-labeled samples (HFE-BSL), uses a hybrid criterion that integrates both the local and global criteria for feature extraction. Thus, it is robust and flexible. The experimental results with three real hyperspectral images show the good efficiency of HFE-BSL compared to some popular and state-of-the-art feature extraction methods.

For classification of hyperspectral images, particularly using limited training samples, supervised feature extraction is an approach for reduction of dimensionality, overcoming the Hughes phenomenon and increasing the classification accuracies. Classic and popular feature extraction methods such as linear discriminant analysis (LDA) have not good efficiency in small sample size situation because of the singularity problem. Another supervised method, nonparametric weighted feature extraction (NWFE) is efficient for solving some problems of LDA and works well using limited training samples. We propose an efficient approach for improving of discriminant analysis (DA) method. The proposed method, named boundary based discriminant analysis (BBDA), uses only the boundary training samples for DA to increases the classification accuracy. Moreover, with using a regularization technique, it overcomes the singularity problem in DA. The experimental results obtained on two popular real hyperspectral data sets (one agriculture image and one urban image) show the improvement of BBDA respect to some conventional supervised feature extraction methods in small sample size situation.

Due to the great benefit of rich spectral information, hyperspectral images (HSIs) have been successfully applied in many fields. However, some problems of concern also limit their further applications, such as high dimension and expensive labeling. To address these issues, an unsupervised latent low-rank projection learning with graph regularization (LatLRPL) method is presented for feature extraction and classification of HSIs in this paper, in which discriminative features can be extracted from the view of latent space by decomposing the latent low-rank matrix into two different matrices, also benefiting from the preservation of intrinsic subspace structures by the graph regularization. Different from the graph embedding-based methods that need two phases to obtain the low-dimensional projections, one step is enough for LatLRPL by constructing the integrated projection learning model, reducing the complexity and simultaneously improving the robustness. To improve the performance, a simple but effective strategy is exploited by conducting the local weighted average on the pixels in a sliding window for HSIs. Experiments on the Indian Pines and Pavia University datasets demonstrate the superiority of the proposed LatLRPL method.

Hyperspectral images (HSI) contain powerful spectral characterization capabilities and are widely used especially for classification applications. However, the rich spectrum contained in HSI also increases the difficulty of extracting useful information, which makes the feature extraction method significant as it enables effective expression and utilization of the spectrum. Traditional HSI feature extraction methods design spectral features manually, which is likely to be limited by the complex spectral information within HSI. Recently, data-driven methods, especially the use of convolutional neural networks (CNNs), have shown great improvements in performance when processing image data owing to their powerful automatic feature learning and extraction abilities and are also widely used for HSI feature extraction and classification. The CNN extracts features based on the convolution operation. Nevertheless, the local perception of the convolution operation makes CNN focus on the local spectral features (LSF) and weakens the description of features between long-distance spectral ranges, which will be referred to as global spectral features (GSF) in this study. LSF and GSF describe the spectral features from two different perspectives and are both essential for determining the spectrum. Thus, in this study, a local-global spectral feature (LGSF) extraction and optimization method is proposed to jointly consider the LSF and GSF for HSI classification. To increase the relationship between spectra and the possibility to obtain features with more forms, we first transformed the 1D spectral vector into a 2D spectral image. Based on the spectral image, the local spectral feature extraction module (LSFEM) and the global spectral feature extraction module (GSFEM) are proposed to automatically extract the LGSF. The loss function for spectral feature optimization is proposed to optimize the LGSF and obtain improved class separability inspired by contrastive learning. We further enhanced the LGSF by introducing spatial relation and designed a CNN constructed using dilated convolution for classification. The proposed method was evaluated on four widely used HSI datasets, and the results highlighted its comprehensive utilization of spectral information as well as its effectiveness in HSI classification.

Linear discriminant analysis (LDA) is a very important approach to selecting features in classification such as facial recognition. However it suffers from the small sample size (SSS) problem where LDA cannot be solved numerically. The SSS problem occurs when the number of training samples is less than the number of dimensions, which is often the case in practice. Researchers have proposed several modified versions of LDA to deal with this problem. However, a solid theoretical analysis is missing. In this paper, we analyze LDA and the SSS problem based on learning theory. LDA is derived from Fisher’s criterion. However, when formulated as a least square approximation problem, LDA has a direct connection to regularization network (RN) algorithms. Many learning algorithms such as support vector machines (SVMs) can be viewed as regularization networks. LDA turns out to be an RN without the regularization term, which is in general an ill-posed problem. This explains why LDA suffers from the SSS problem. In order to transform the ill-posed problem into a well-posed one, the regularization term is necessary. Thus, based on statistical learning theory, we derive a new approach to discriminant analysis. We call it discriminant learning analysis (DLA). DLA is wellposed and behaves well in the SSS situation. Experimental results are presented to validate our proposal.

The small sample size (SSS) problem has been an essential issue for high-dimensional data classification because the cost of collecting the samples is generally expensive and difficult. The SSS problem often results in unsatisfied classification results. Feature extraction (or selection) and classifier enhancement are two commonly-used approaches for overcoming the SSS problem to achieve better classification results. The former method is to reduce the data dimensionality, and then applying the reduced-dimensionality data set to train a classifier. The latter is to modify the classifier design to be suitable for the SSS problem. The recent literature manifests that a classifier performs better on the data set transformed by nonparametric feature extraction than that by the well-known parametric method, linear discriminant analysis (LDA). In this paper, we propose a novel K-nearest neighbor (KNN) classifier, namely adaptive KNN (AKNN), which is a KNN-type classifier embedding the merits of a well-performed nonparametric feature extraction method, namely NLDA. In AKNN, the distance metrics are formed by NLDA features. In the training phase of AKNN, a metric is estimated and assigned to each training sample and then, in the classification phase, a weighted metric with respect to the test sample is computed by its KNN training samples' metrics. Based on the weighted metric, the test sample is classified to the majority category of its KNNs. One remotely sensed hyperspectral benchmark image is included for investigating the effectiveness of AKNN. Experimental results demonstrate that the proposed AKNN can perform better than the classic KNN and support vector machine (SVM) classifier.

Hyperspectral image (HSI) classification (HIC) has attracted much attention in the last decade. Spectral-spatial HIC methods have been the state-of-the-art methods in recent years. Small labeled training sample size (SLTSS) problem is still an important issue in HIC. This paper presents a spectral-spatial HIC method that is based on superpixel (SP) segmentation and distance-weighted linear regression classifier to tackle the SLTSS problem. First, SP segmentation is applied to the original HSI. Then, those SPs that contain training samples belonging to only one class are first searched out, and all the pixels of each of these SPs are assigned to the class of the training samples it contains. Next, with the identified labels, all these classified pixels are added to the initial training sample set for training sample set enlargement. Later, using this enlarged training sample set, the distance-weighted linear regression classifier is applied to classify each mean vector of each SP. Finally, the last classification map is obtained by assigning each SP with the same label as its mean vector. Experimental results on three HSI data sets demonstrate that the proposed approach can solve SLTSS problem very well and outperforms several state-of-the-art algorithms in classification accuracy under different training samples sizes.

The classification of Hyperspectral images (HSIs) has been the focus of many recent research efforts, where feature extraction plays an important role. Discriminative feature extraction methods aim to reduce the data dimension of HSIs, retain effective image information to the greatest extent, and suppress noises at the same time. Besides, according to the characteristics of pixel-by-pixel-multi-band of HSIs and data redundancy between bands, the processing of HSIs in the classifier will bring huge computational overhead. In this paper, we present a parallel implementation of the improved noise adaptive principal component algorithm (INAPC) for feature extraction of hyperspectral images on commodity graphics processing units (GPUs). Aiming at maximizing the signal-to-noise ratio (SNR) instead of the variance, we firstly deploy two SVDs and more comprehensive noise estimation in the INAPC transform and constructed a complete feature extraction process. Then we deploy a complete CPU-GPU collaborative computing solution, and use several GPU programming optimization methods to achieve the maximum acceleration effect. Through the experiments on three real hyperspectral datasets, Experimental results show that the proposed INAPC has stable superiority and provides a significant speedup compared to the CPU implementation.

For hyperspectral image classification, feature extraction is a crucial pre-process for avoiding the Hughes phenomena. Some feature extraction methods such as linear discriminant analysis (LDA), nonparametric weighted feature extraction (NWFE), and their kernel versions, generalized discriminant analysis (GDA) and kernel nonparametric weighted feature extraction method (KNWFE) have been shown that they can improve the classification performance. However, for GDA and KNWFE, it is hard to find the suitable kernel parameters. Hence, although they have been published about 14 or 6 years, respectively, researchers rarely implement them for dealing with hyperspectral image classification problem. An automatic kernel parameter selection method (APS) was proposed to predetermine the appropriate radial basis function (RBF) kernel for support vector machine (SVM) and GDA. In this study, APS was applied to find the suitable RBF kernel function for KNWFE. From the experiment results on PAVIA data set, the classification performance of KNWFE still outperforms those of GDA [10] and SVM [10]. The most important of this research, the kernel parameters of GDA and KNWFE based on RBF kernel can be “automatically” determined and the researcher can implement them directly without tuning the kernel parameter.

Despite the successful applications of probabilistic collaborative representation classification (PCRC) in pattern classification, it still suffers from two challenges when being applied on hyperspectral images (HSIs) classification: 1) ineffective feature extraction in HSIs under noisy situation; and 2) lack of prior information for HSIs classification. To tackle the first problem existed in PCRC, we impose the sparse representation to PCRC, i.e., to replace the 2-norm with 1-norm for effective feature extraction under noisy condition. In order to utilize the prior information in HSIs, we first introduce the Euclidean distance (ED) between the training samples and the testing samples for the PCRC to improve the performance of PCRC. Then, we bring the coordinate information (CI) of the HSIs into the proposed model, which finally leads to the proposed locality regularized robust PCRC (LRR-PCRC). Experimental results show the proposed LRR-PCRC outperformed PCRC and other state-of-the-art pattern recognition and machine learning algorithms.

Most studies showed that most hyperspectral image classification encountered the Hughes phenomenon due to the redundant features, especially in the small sample size problem. Feature extraction method such as linear discriminant analysis (LDA), nonparametric weighted feature extraction (NWFE) is a preprocessing step before classification and used to combine and reduce the original features into a new feature space based on the between-class and with-class separability. Then, the classifier such as the nonlinear support vector machine (SVM) is trained and classifies the unknown samples. However, the separability measurement of LDA and NWFE is for the original space not the kernel-induced feature space. In this study, a kernel-based feature extraction method is proposed. The corresponding transformation matrix for dimension reduction is based on the class separability in the kernel-induced feature space which was proposed in our previous study. Experimental results on the Indian Pine Site dataset show that the proposed method improves the classification performance of the SVM on the small sample size problem.

Hyperspectral images provide a large volume of spectral bands. Feature extraction (FE) is an important preprocessing step for classification of high-dimensional data. Supervised FE methods such as linear discriminant analysis, generalized discriminant analysis, and nonparametric weighted FE use the criteria of class separability. Theses methods maximize the between-class scatter matrix and minimize the within-class scatter matrix. We propose a supervised FE method in this letter, which uses no statistical moments. Thus, it works well using limited training samples. The proposed FE method consists of two important phases. In the first phase, an attraction point for each class is found. In the second phase, by using an appropriate transformation, the samples of each class move toward the attraction point of their class. The experimental results on two real hyperspectral images demonstrate that FE using attraction points has better performance in comparison with some other supervised FE methods in a small sample size situation.