Traditional Linear Discriminant Analysis Research Articles

Comparison of incremental linear dimension reduction methods for streaming data

Traditional linear dimension reduction methods such as Principal component analysis (PCA) and linear discriminant analysis (LDA) have been used in many application areas due to simplicity and high performance. However, in data streams where data instances are generated continuously over time, it is difficult to apply traditional PCA or LDA. Moreover, data streams can have drifting concepts over time. In this paper, we compared several incremental linear dimension reduction algorithms which can be applied for classification in streaming data. Also, the performance comparison for prediction accuracy and time complexity was conducted in various streaming environments such as low dimensional data streams, high dimensional data streams, and data streams with concept drifts. Experimental results showed that incremental least squares formulation (ILS) combined with incremental PCA can be used effectively for classification in streaming data.

Tensor Discriminant Analysis via Compact Feature Representation for Hyperspectral Images Dimensionality Reduction

Dimensionality reduction is of great importance which aims at reducing the spectral dimensionality while keeping the desirable intrinsic structure information of hyperspectral images. Tensor analysis which can retain both spatial and spectral information of hyperspectral images has caused more and more concern in the field of hyperspectral images processing. In general, a desirable low dimensionality feature representation should be discriminative and compact. To achieve this, a tensor discriminant analysis model via compact feature representation (TDA-CFR) was proposed in this paper. In TDA-CFR, the traditional linear discriminant analysis was extended to tensor space to make the resulting feature representation more informative and discriminative. Furthermore, TDA-CFR redefines the feature representation of each spectral band by employing the tensor low rank decomposition framework which leads to a more compact representation.

Unsupervised emotion recognition algorithm based on improved deep belief model in combination with probabilistic linear discriminant analysis

Since the initial weight matrix between the last hidden layer of the network and the classification layer is usually generated randomly, the weight matrix does not have the discrimination ability to accurately classify the facial expression recognition, which results in that the features obtained by the weight matrix mapping cannot be guaranteed to be suitable for classification tasks. To solve this problem, a novel linear discriminant deep belief network is proposed in this paper. Firstly, the traditional linear discriminant analysis method is improved, and a new type of inter-class dispersion matrix is designed to solve the rank limitation problem in the traditional Linear Discriminant Analysis Method (LDA). Then, the weight matrix between the last hidden layer and the classification layer of the deep belief network is initialized by the improved linear discriminant analysis method, so that the network is more suitable for the classification task. In the experiments, our proposed deep network obtains respectively the recognition rates of 78.26% and 94.48% on the JAFFE database and the Extended Cohn-Kanade database. In addition, using our proposed algorithm for aggregating linear discriminant analysis into a deep belief network, we were able to produce an accuracy of 81.03% on the challenge test set.

Robust bilateral Lp-norm two-dimensional linear discriminant analysis

Traditional linear discriminant analysis (LDA) may suffer from a sensitivity to outliers and the small sample size (SSS) problem, while the Lp-norm measure for 0 < p ≤ 1 is robust in a sense. In this paper, based on the criterion of the Bayes optimality, we propose a matrix-based bilateral Lp-norm two-dimensional linear discriminant analysis (BLp2DLDA) with robust performance, where 0 < p ≤ 1. We prove that the BLp2DLDA criterion is equivalent to an upper bound of the theoretical framework of the Bayes optimality. Compared with the L2-norm 2-directional 2-dimensional LDA ((2D)2LDA), our BLp2DLDA is more robust to outliers and noise. Moreover, unilateral Lp2DLDA can also be easily derived from BLp2DLDA. In addition, a simple but effective iterative technique is introduced to solve BLp2DLDA and the unilateral Lp2DLDA. Experimental results on different types of contaminated human face databases show that the proposed BLp2DLDA outperforms (2D)2LDA.

Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers.

In this paper, we propose a robust linear discriminant analysis (RLDA) through Bhattacharyya error bound optimization. RLDA considers a nonconvex problem with the L1 -norm operation that makes it less sensitive to outliers and noise than the L2 -norm linear discriminant analysis (LDA). In addition, we extend our RLDA to a sparse model (RSLDA). Both RLDA and RSLDA can extract unbounded numbers of features and avoid the small sample size (SSS) problem, and an alternating direction method of multipliers (ADMM) is used to cope with the nonconvexity in the proposed formulations. Compared with the traditional LDA, our RLDA and RSLDA are more robust to outliers and noise, and RSLDA can obtain sparse discriminant directions. These findings are supported by experiments on artificial data sets as well as human face databases.

Convenient and accurate method for the identification of Chinese teas by an electronic nose

A convenient, accurate, and effective approach for the identification of Chinese teas and their production area has been developed. For this, Chinese tea samples from different regions were collected and their odours were analysed by an electronic nose (E-nose). An unambiguous identification of the Chinese teas could not be achieved by means of traditional principal component analysis or linear discriminant analysis methods. Thus, multiple logistic regression (MLR), support vector machines (SVM), and random forests (RF) were employed as alternative to build identification models. The experimental results show that the method aiming within scope based on the RF performs very well, with prediction accuracies and computation times being superior to the two others (MLR and SVM). The results were demonstrated that E-nose could be used in the classification of Chinese teas, when an optimal pattern recognition algorithm is selected. The present study provides a critical outlook on the developments of Chinese teas identification, authenticity control and against adulteration in the Chinese circulation market.

Rapid and Automatic Classification of Tobacco Leaves Using a Hand-Held DLP‑Based NIR Spectroscopy Device

A hand-held near infrared (NIR) spectroscopy device is much more convenient than a traditional desktop NIR instrument. Thus, it is more suitable for the practical application. An automatic and rapid tool for grading tobacco leaves on the spot using a hand-held digital light processing (DLP)-based NIR spectroscopy device is proposed in this paper. Firstly, the spectral data of the samples is scanned with a hand-held NIR device directly from the tobacco leaves without any samples preparation procedures. Then, the training model of different classes is built and the class of each test sample is predicted by using sparse representation classification (SRC) algorithm. Comparing with the traditional linear discriminant analysis (LDA) and support vector machine (SVM) algorithms, the classification accuracy of SRC method is the highest and has the least computation time. The results show that hand-held NIR spectroscopy technology could be a novel classification tool for grading tobacco leaves in the purchasing on the spot.

Discriminant Analysis with Graph Learning for Hyperspectral Image Classification

Linear Discriminant Analysis (LDA) is a widely-used technique for dimensionality reduction, and has been applied in many practical applications, such as hyperspectral image classification. Traditional LDA assumes that the data obeys the Gaussian distribution. However, in real-world situations, the high-dimensional data may be with various kinds of distributions, which restricts the performance of LDA. To reduce this problem, we propose the Discriminant Analysis with Graph Learning (DAGL) method in this paper. Without any assumption on the data distribution, the proposed method learns the local data relationship adaptively during the optimization. The main contributions of this research are threefold: (1) the local data manifold is captured by learning the data graph adaptively in the subspace; (2) the spatial information within the hyperspectral image is utilized with a regularization term; and (3) an efficient algorithm is designed to optimize the proposed problem with proved convergence. Experimental results on hyperspectral image datasets show that promising performance of the proposed method, and validates its superiority over the state-of-the-art.

Research and Implementation of ECG-Based Biological Recognition Parallelization

Nowadays, biological recognition technologies attract more attention in electrocardiograph (ECG) signals, which vary among different people and are difficult to counterfeit. However, the robustness of recognition cannot be well sustained in the case of diversified application scenarios and huge human crowds. In order to tackle this problem, this paper puts forward a fiducial and non-fiducial mixed feature extraction method, which can effectively complete the multidimensional feature modeling of ECG signal. In addition, this paper proposes a linear discriminant analysis (LDA) based on multiple features (LOMF) algorithm based on ECG mixed feature to solve time-overhead problem of big data training. LOMF includes ECG signal preprocessing, sub-block division, and block training. By combining the MapReduce distributed computing framework and the secondary retrieval method based on the multi-dimensional feature space, LOMF is parallelized to improve recognition rate and computing efficiency at the same time. The experiment results show that, in the diversified scenarios, utilizing ECG mixed feature can return a higher recognition rate than the traditional ECG 1-D feature. Moreover, compared with the traditional LDA and support vector machine algorithms, the precision of LOMF increases by 7%–8%, which depends on the most competitive advantage of using LOMF. LOMF fits MapReduce parallel framework well so it is more effective than traditional algorithms, especially on diversified application scenarios (such as Internet) where the amount of data grows rapidly.

Locality Adaptive Discriminant Analysis for Spectral–Spatial Classification of Hyperspectral Images

Linear discriminant analysis (LDA) is a popular technique for supervised dimensionality reduction, but with less concern about a local data structure. This makes LDA inapplicable to many real-world situations, such as hyperspectral image (HSI) classification. In this letter, we propose a novel dimensionality reduction algorithm, locality adaptive discriminant analysis (LADA) for HSI classification. The proposed algorithm aims to learn a representative subspace of data, and focuses on the data points with close relationship in spectral and spatial domains. An intuitive motivation is that data points of the same class have similar spectral feature and the data points among spatial neighborhood are usually associated with the same class. Compared with traditional LDA and its variants, LADA is able to adaptively exploit the local manifold structure of data. Experiments carried out on several real hyperspectral data sets demonstrate the effectiveness of the proposed method.

Graph embedding and extensions: a general framework for dimensionality reduction.

Over the past few decades, a large family of algorithms - supervised or unsupervised; stemming from statistics or geometry theory - has been designed to provide different solutions to the problem of dimensionality reduction. Despite the different motivations of these algorithms, we present in this paper a general formulation known as graph embedding to unify them within a common framework. In graph embedding, each algorithm can be considered as the direct graph embedding or its linear/kernel/tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set, with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided. Furthermore, the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms. By utilizing this framework as a tool, we propose a new supervised dimensionality reduction algorithm called Marginal Fisher Analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class, while the penalty graph connects the marginal points and characterizes the interclass separability. We show that MFA effectively overcomes the limitations of the traditional Linear Discriminant Analysis algorithm due to data distribution assumptions and available projection directions. Real face recognition experiments show the superiority of our proposed MFA in comparison to LDA, also for corresponding kernel and tensor extensions.

Classification of Big Data With Application to Imaging Genetics

Big data applications, such as medical imaging and genetics, typically generate datasets that consist of few observations ${\boldsymbol n}$ on many more variables ${\boldsymbol p}$ , a scenario that we denote as ${\boldsymbol p}\gg{\boldsymbol n}$ . Traditional data processing methods are often insufficient for extracting information out of big data. This calls for the development of new algorithms that can deal with the size, complexity, and the special structure of such datasets. In this paper, we consider the problem of classifying ${\boldsymbol p}\gg{\boldsymbol n}$ data and propose a classification method based on linear discriminant analysis (LDA). Traditional LDA depends on the covariance estimate of the data, but when ${\boldsymbol p}\gg{\boldsymbol n}$ , the sample covariance estimate is singular. The proposed method estimates the covariance by using a sparse version of noisy principal component analysis (nPCA). The use of sparsity in this setting aims at automatically selecting variables that are relevant for classification. In experiments, the new method is compared to state-of-the art methods for big data problems using both simulated datasets and imaging genetics datasets.

Generalized linear discriminant analysis based on euclidean norm for gait recognition

One of the key issues in gait recognition is how to extract the low dimensional feature. Linear Discriminant Analysis (LDA) is a commonly used method for linear dimension reduction. This paper proposed a generalized LDA based on Euclidean norm (ELDA) for gait recognition. By redefining a better between-class scatter matrix to separate the neighboring samples that overcome the drawbacks existing in the traditional LDA method. Firstly, the contour is unwrapped counterclockwise by the distance from the uppermost pixel to transformed 2D features into 1D. Secondly, we use ELDA to obtain more discriminative feature space. Finally, multi-class Support Vector Machine (SVM) is applied to implement gait classification. Experimental results show that this algorithm achieves better results in terms of accuracy and efficiency than other gait recognition methods at present.

Semi-supervised linear discriminant analysis for dimension reduction and classification

When facing high dimensional data, dimension reduction is necessary before classification. Among dimension reduction methods, linear discriminant analysis (LDA) is a popular one that has been widely used. LDA aims to maximize the ratio of the between-class scatter and total data scatter in projected space, and the label of each data is necessary. However, in real applications, the labeled data are scarce and unlabeled data are in large quantity, so LDA is hard to be used under such case. In this paper, we propose a novel method named semi-supervised linear discriminant analysis (SLDA), which can use limited number of labeled data and a quantity of the unlabeled ones for training so that LDA can accommodate to the situation of a few labeled data available. Assuming that F represents the calculated class indicator matrix of the training data and Y denotes the true label of the labeled data, the objective function contains two parts: one is the criterion of LDA (which is a function of projection W, and a class indicator matrix F), the other is the difference between the true data label and calculated label of these labeled data. As far as we know, there is no closed-form solution to the objective function. To solve such problem, we develop an iterative algorithm which calculates the class indicator matrix and the projection alternatively. The convergence of the proposed iterative algorithm is proved and confirmed by experiments. The experimental results on eight datasets show that the performance of SLDA is superior to that of traditional LDA and some state-of-the-art semi-supervised algorithms.

Phone classification via manifold learning based dimensionality reduction algorithms

Mechanical limitations imposed on the articulators during speech production lead to a limitation of the intrinsic dimensionality of speech signals. This limitation leads to a specific neighborhood structure of speech sounds when they are represented in a high-dimensional feature space. We investigate whether phone classification can be improved by exploiting this neighborhood structure, by means of extended variants of the conventional Linear Discriminant Analysis (LDA) based on manifold learning.In this extended LDA approach, the within-class and between-class scatter matrices are defined in terms of adjacency graphs. We compare extensions of LDA that use either a full adjacency graph or an adjacency graph defined in the neighborhood of the training observations. In addition, we apply different kernels for weighing the distances in the graphs via different kernels, of which the Adaptive Kernel is proposed in this paper.Experiments with TIMIT show that while LDA algorithms that use the full adjacency graph do not outperform traditional LDA, the algorithms that exploit only local information provide significantly better results than traditional LDA. These improvements are not uniform across different broad phonetic classes, which suggests that the added value of the neighborhood structure is phone class dependent. The structure is represented by locally different densities in the neighborhood of feature vectors that are representative of a specific phone in a specific context.

Multi-class Fukunaga Koontz discriminant analysis for enhanced face recognition

Linear subspace learning methods such as Fisher׳s Linear Discriminant Analysis (LDA), Unsupervised Discriminant Projection (UDP), and Locality Preserving Projections (LPP) have been widely used in face recognition applications as a tool to capture low dimensional discriminant information. However, when these methods are applied in the context of face recognition, they often encounter the small-sample-size problem. In order to overcome this problem, a separate Principal Component Analysis (PCA) step is usually adopted to reduce the dimensionality of the data. However, such a step may discard dimensions that contain important discriminative information that can aid classification performance. In this work, we propose a new idea which we named Multi-class Fukunaga Koontz Discriminant Analysis (FKDA) by incorporating the Fukunaga Koontz Transform within the optimization for maximizing class separation criteria in LDA, UDP, and LPP. In contrast to traditional LDA, UDP, and LPP, our approach can work with very high dimensional data as input, without requiring a separate dimensionality reduction step to make the scatter matrices full rank. In addition, the FKDA formulation seeks optimal projection direction vectors that are orthogonal which the existing methods cannot guarantee, and it has the capability of finding the exact solutions to the “trace ratio” objective in discriminant analysis problems while traditional methods can only deal with a relaxed and inexact “ratio trace” objective. We have shown using six face database, in the context of large scale unconstrained face recognition, face recognition with occlusions, and illumination invariant face recognition, under “closed set”, “semi-open set”, and “open set” recognition scenarios, that our proposed FKDA significantly outperforms traditional linear discriminant subspace learning methods as well as five other competing algorithms.

Sparse matrix transform-based linear discriminant analysis for hyperspectral image classification

Due to the high dimensionality of hyperspectral image (HSI), dimension reduction or feature extraction is usually needed before the HSI classification. Traditional linear discriminant analysis (LDA) method for feature extraction usually encounters difficulty because the available training samples in HSI classification are limited, which causes the singularity of data scatter matrix. In this paper, we propose a sparse matrix transform-based LDA (SMT-LDA) algorithm for the HSI classification. By using SMT, the total scatter matrix used in LDA can be constrained to have an eigen-decomposition where the eigenvectors can be sparsely parametrized by a limited number of Givens rotations. In this way, the estimated scatter matrix is always positive definite and well conditioned even in the case of limited training samples. The proposed SMT-LDA method is compared with regularized LDA and PCA-LDA methods on two benchmark hyperspectral data sets. Experimental results indicate that the performance of the proposed method is overall superior to these methods, especially for small-sample-size classification.

On the linear discriminant analysis for large number of classes

We study the challenging problem to classify samples into a large number of classes, and propose the idea of using different Dimensionality-Reduction (DR) projections for different classes of samples. Based on this intuitive idea, the traditional Linear Discriminant Analysis (LDA) and the trace-ratio LDA are formulated to their corresponding new multi-subspace objectives. We justify that certain effects of class-adaptive feature selection are naturally achieved via our multi-subspace DR methods. Experiments on seven datasets show that, our multi-subspace trace-ratio LDA outperform its ratio-trace and single-subspace counterparts, and its advantage is more apparent when the number of classes to be classified is large.

Face Recognition Invariant to Partial Occlusions

Face recognition is considered a complex biometrics in the field of image processing mainly due to the constraints imposed by variation in the appearance of facial images. These variations in appearance are affected by differences in expressions and/or occlusions (sunglasses, scarf etc.). This paper discusses incremental Kernel Fisher Discriminate Analysis on sub-classes for dealing with partial occlusions and variant expressions. This framework focuses on the division of classes into fixed size sub-classes for effective feature extraction. For this purpose, it modifies the traditional Linear Discriminant Analysis into incremental approach in the kernel space. Experiments are performed on AR, ORL, Yale B and MIT-CBCL face databases. The results show a significant improvement in face recognition.

Sparse Graph-Based Discriminant Analysis for Hyperspectral Imagery

Sparsity-preserving graph construction is investigated for the dimensionality reduction of hyperspectral imagery. In particular, a sparse graph-based discriminant analysis is proposed when labeled samples are available. By forcing the projection to be along the direction where a sample is clustered with within-class samples that best represented it, the discriminative power can be enhanced. The proposed method has no requirement on the number of labeled samples as in traditional linear discriminant analysis, and it can be solved by a simple generalized eigenproblem. The quality of the dimensionality reduction is evaluated by a support vector machine with a composite spatial-spectral kernel. Experimental results demonstrate that the proposed sparse graph-based discriminant analysis can yield superior classification performance with much lower dimensionality as compared to performance on the original data or on data transformed with other dimensionality-reduction approaches.