Problem Of Discriminant Analysis Research Articles

Kernel Locality Preserving Symmetrical Weighted Fisher Discriminant Analysis based subspace approach for expression recognition

This paper mainly focuses on dimensional reduction of fused dataset of holistic and geometrical face features vectors by solving singularity problem of linear discriminant analysis and maximizing the Fisher ratio in nonlinear subspace region with the preservation of local discriminative features. The combinational feature vector space is projected into low dimensional subspace using proposed Kernel Locality Preserving Symmetrical Weighted Fisher Discriminant Analysis (KLSWFDA) method. Matching score level fusion technique has been applied on projected subspace and combinational entire Gabor subspace is framed. Euclidean distance metric (L2) and support vector machine (SVM) classifier has been implemented to recognize and classify the expressions. Performance of proposed approach is evaluated and compared with state of art approaches. Experimental results on JAFFE, YALE and FD expression database demonstrate the effectiveness of the proposed approach.

Multi-class Fukunaga Koontz discriminant analysis for enhanced face recognition

Linear subspace learning methods such as Fisher׳s Linear Discriminant Analysis (LDA), Unsupervised Discriminant Projection (UDP), and Locality Preserving Projections (LPP) have been widely used in face recognition applications as a tool to capture low dimensional discriminant information. However, when these methods are applied in the context of face recognition, they often encounter the small-sample-size problem. In order to overcome this problem, a separate Principal Component Analysis (PCA) step is usually adopted to reduce the dimensionality of the data. However, such a step may discard dimensions that contain important discriminative information that can aid classification performance. In this work, we propose a new idea which we named Multi-class Fukunaga Koontz Discriminant Analysis (FKDA) by incorporating the Fukunaga Koontz Transform within the optimization for maximizing class separation criteria in LDA, UDP, and LPP. In contrast to traditional LDA, UDP, and LPP, our approach can work with very high dimensional data as input, without requiring a separate dimensionality reduction step to make the scatter matrices full rank. In addition, the FKDA formulation seeks optimal projection direction vectors that are orthogonal which the existing methods cannot guarantee, and it has the capability of finding the exact solutions to the “trace ratio” objective in discriminant analysis problems while traditional methods can only deal with a relaxed and inexact “ratio trace” objective. We have shown using six face database, in the context of large scale unconstrained face recognition, face recognition with occlusions, and illumination invariant face recognition, under “closed set”, “semi-open set”, and “open set” recognition scenarios, that our proposed FKDA significantly outperforms traditional linear discriminant subspace learning methods as well as five other competing algorithms.

Noisy discriminant analysis with boundary assumptions

We address the problem of smooth discriminant analysis when data are collected from two samples with measurement errors. This problem turns to be an inverse problem and requires a specific treatment. In this context, we investigate consistency rates of convergence using both a margin assumption, and a complexity assumption in terms of entropy. In particular, we concentrate our attention on a boundary condition on the Bayes set and exhibits two distinct scenarii of convergence for the excess risk. For mildly ill-posed inverse problems, fast rates (i.e. faster than ) may occur whereas in the presence of one ‘supersmooth’ component for measurement errors, the excess risk is a negative power of .

Optimal Feature Selection in High-Dimensional Discriminant Analysis.

We consider the high-dimensional discriminant analysis problem. For this problem, different methods have been proposed and justified by establishing exact convergence rates for the classification risk, as well as the ℓ2 convergence results to the discriminative rule. However, sharp theoretical analysis for the variable selection performance of these procedures have not been established, even though model interpretation is of fundamental importance in scientific data analysis. This paper bridges the gap by providing sharp sufficient conditions for consistent variable selection using the sparse discriminant analysis (Mai et al., 2012). Through careful analysis, we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size n, dimensionality p, and sparsity level s in the high-dimensional setting. Sufficient conditions are complemented by the necessary information theoretic limits on the variable selection problem in the context of high-dimensional discriminant analysis. Exploiting a numerical equivalence result, our method also establish the optimal results for the ROAD estimator (Fan et al., 2012) and the sparse optimal scaling estimator (Clemmensen et al., 2011). Furthermore, we analyze an exhaustive search procedure, whose performance serves as a benchmark, and show that it is variable selection consistent under weaker conditions. Extensive simulations demonstrating the sharpness of the bounds are also provided.

Pareto models for discriminative multiclass linear dimensionality reduction

We address the class masking problem in multiclass linear discriminant analysis (LDA). In the multiclass setting, LDA does not maximize each pairwise distance between classes, but rather maximizes the sum of all pairwise distances. This results in serious overlaps between classes that are close to each other in the input space, and degrades classification performance. Our research proposes Pareto Discriminant Analysis (PARDA); an approach for multiclass discriminative analysis that builds over multiobjective optimizing models. PARDA decomposes the multiclass problem to a set of objective functions, each representing the distance between every pair of classes. Unlike existing LDA extensions that maximize the sum of all distances, PARDA maximizes each pairwise distance to maximally separate all class means, while minimizing the class overlap in the lower dimensional space. Experimental results on various data sets show consistent and promising performance of PARDA when compared with well-known multiclass LDA extensions.

Fault diagnosis of rolling element bearings via discriminative subspace learning: Visualization and classification

Rolling element bearings play an important role in ensuring the availability of industrial machines. Unexpected bearing failures in such machines during field operation can lead to machine breakdown, which may have some pretty severe implications. To address such concern, we extend our algorithm for solving trace ratio problem in linear discriminant analysis to diagnose faulty bearings in this paper. Our algorithm is validated by comparison with other state-of art methods based on a UCI data set, and then be extended to rolling element bearing data. Through the construction of feature data set from sensor-based vibration signals of bearing, the fault diagnosis problem is solved as a pattern classification and recognition way. The two-dimensional visualization and classification accuracy of bearing data show that our algorithm is able to recognize different bearing fault categories effectively. Thus, it can be considered as a promising method for fault diagnosis.

SAR target recognition method based on weighted two-directional and two-dimensional linear discriminant analysis

To solve the Small Sample Size(SSS) problem and the "inferior" problem of traditional Fisher Linear Discriminant Analysis(FLDA) when it is applied to Synthetic Aperture Radar(SAR) image recognition tasks,a new image feature extraction technique was proposed based on weighted two-directional and two-dimensional linear discriminant analysis(W(2D)2LDA).First,the scatter matrices in the two-directional and two-dimensional linear discriminant analysis criterion were modified by adding weights.Then,feature matrix was extracted by W(2D)2LDA.The experimental results with MSTAR dataset verify the effectiveness of the proposed method,and it can strengthen the feature's discrimination and obtain better recognition performance with fewer memory requirements simultaneously.

A misclassification cost risk bound based on hybrid particle swarm optimization heuristic

Linear discriminant analysis models to minimize misclassification cost have recently gained popularity. It is well known that the misclassification cost minimizing linear discriminant analysis problem is an ▪-complete problem that is difficult to solve to optimality for large scale datasets. As a result, heuristic techniques have gained popularity but it is difficult to assess how well these heuristic techniques perform. One way to aid assessment of the performance of heuristic techniques is to establish a lower-bound on the optimal value of misclassification cost. In this paper, we propose and use a hybrid particle swarm optimization (PSO) and Lagrangian relaxation (LR) based heuristic to establish a misclassification cost lower bound (MCLB) for two-group linear classifiers. We use the subgradient optimization procedure to tighten the MCLB. Using simulated and real-world datasets, we test a misclassification cost minimizing linear genetic algorithm classifier and two commercial non-linear classifiers (C5.0 and C&RT) to compare their performances with the MCLB. Our holdout sample tests indicate that the proposed MCLB works well for both linear and non-linear classifiers when class data distributions are normal. Additionally, as misclassification cost asymmetry increases, the proposed MCLB appears to provide better results.

Kernel analysis on Grassmann manifolds for action recognition

Modelling video sequences by subspaces has recently shown promise for recognising human actions. Subspaces are able to accommodate the effects of various image variations and can capture the dynamic properties of actions. Subspaces form a non-Euclidean and curved Riemannian manifold known as a Grassmann manifold. Inference on manifold spaces usually is achieved by embedding the manifolds in higher dimensional Euclidean spaces. In this paper, we instead propose to embed the Grassmann manifolds into reproducing kernel Hilbert spaces and then tackle the problem of discriminant analysis on such manifolds. To achieve efficient machinery, we propose graph-based local discriminant analysis that utilises within-class and between-class similarity graphs to characterise intra-class compactness and inter-class separability, respectively. Experiments on KTH, UCF Sports, and Ballet datasets show that the proposed approach obtains marked improvements in discrimination accuracy in comparison to several state-of-the-art methods, such as the kernel version of affine hull image-set distance, tensor canonical correlation analysis, spatial-temporal words and hierarchy of discriminative space-time neighbourhood features.

Efficient optimally regularized discriminant analysis

Regularized discriminant analysis (RDA) and its special case uncorrelated linear discriminant analysis (ULDA) are important subspace learning methods proposed recently to handle the small sample size (SSS) problem of linear discriminant analysis (LDA). One important unsolved issue of RDA is how to automatically determine an appropriate regularization parameter without resorting to unscalable procedures like cross-validation (CV). In this paper, we develop a novel efficient algorithm to automatically estimate the regularization parameter based on a geometric interpretation of RDA. We further provide a formal analysis of the proposed method, and show that it is robust to the perturbation in the feature space of the training data. The extensive experiments on various benchmark datasets verify the scalability and effectiveness of our approach, compared with the state-of-the-art algorithms.

Characterization of All Solutions for Undersampled Uncorrelated Linear Discriminant Analysis Problems

In this paper the uncorrelated linear discriminant analysis (ULDA) for undersampled problems is studied. The main contributions of the present work include the following: (i) all solutions of the optimization problem used for establishing the ULDA are parameterized explicitly; (ii) the optimal solutions among all solutions of the corresponding optimization problem are characterized in terms of both the ratio of between-class distance to within-class distance and the maximum likelihood classification, and it is proved that these optimal solutions are exactly the solutions of the corresponding optimization problem with minimum Frobenius norm, also minimum nuclear norm; these properties provide a good mathematical justification for preferring the minimum-norm transformation over other possible solutions as the optimal transformation in ULDA; (iii) explicit necessary and sufficient conditions are provided to ensure that these minimal solutions lead to a larger ratio of between-class distance to within-class distance, thereby achieving larger discrimination in the reduced subspace than that in the original data space, and our numerical experiments show that these necessary and sufficient conditions hold true generally. Furthermore, a new and fast ULDA algorithm is developed, which is eigendecomposition-free and SVD-free, and its effectiveness is demonstrated by some real-world data sets.

A New Face Recognition Framework: Symmetrical Bilateral 2DPLS plus LDA

A novel face recognition framework is proposed in this paper to alleviate Small Sample Size (SSS) problem of the conventional Linear Discriminant Analysis (LDA). This method is based on the feature extraction of global odd and even face image representation, and a dimension reduction process via Symmetrical Bilateral 2D Partial Least Square Analysis (2DPLS). The low-dimensional features are then used to train a LDA classifier which uses Frobenius -norm classification measure, and uses pseudo inverse to make sure between-class matrix S w be full rank. Experimental results on Yale Face Database B, ORL, and FERET Face Database demonstrate that our framework is highly efficient and gives the state-of-the-art recognition rate.

Incremental two-dimensional linear discriminant analysis with applications to face recognition

Two dimensional linear discriminant analysis (2DLDA) has been verified as an effective method to solve the small sample size (SSS) problem in linear discriminant analysis (LDA). However, most of the existing 2DLDA techniques do not support incremental subspace analysis for updating the discriminant eigenspace. Incremental learning has proven to enable efficient training if large amounts of training data have to be processed or if not all data are available in advance as, for example, in on-line situations. Instead of having to re-training across the entire training data whenever a new sample is added, this paper proposed an incremental two-dimensional linear discriminant analysis (I2DLDA) algorithm with closed-form solution to extract facial features of the appearance image on-line. The proposed I2DLDA inherits the advantages of the 2DLDA and the Incremental LDA (ILDA) and overcomes the number of the classes or chunk size limitation in the ILDA because the size of the between-class scatter matrix and the size of the within-class scatter matrix in the I2DLDA are much smaller than the ones in the ILDA. The results on experiments using the ORL and XM2VTS databases show that the I2DLDA is computationally more efficient than the batch 2DLDA and can achieve better recognition results than the ILDA.

A tabu search with an oscillation strategy for the discriminant analysis problem

This article proposes a tabu search approach to solve a mathematical programming formulation of the linear classification problem, which consists of determining an hyperplane that separates two groups of points as well as possible in ℜ m . The tabu search approach proposed is based on a non-standard formulation using linear system infeasibility. The search space is the set of bases defined on the matrix that describes the linear system. The moves are performed by pivoting on a specified row and column. On real machine learning databases, our approach compares favorably with implementations based on parametric programming and irreducible infeasible constraint sets. Additional computational results for randomly generated instances confirm that our method provides a suitable alternative to the mixed integer programming formulation that is solved by a commercial code when the number of attributes m increases.

자료별 분류분석(DDA)에 의한 특징추출

본 논문은 선형차원감소(Linear Dimensionality Reduction)을 위해 널리 이용되고 있는 특징추출 알고리듬인 선형판별분석(Linear Discriminant Analysis)의 문제점을 해결할 수 있는 새로운 특징추출 알고리듬을 제안한다. 선형판별분석에 포함되는 평균-자료 간 거리 및 평균-평균 간의 거리에 기반한 분산행렬은 역행렬 연산, 계수의 제한 등으로 인하여 계산상의 문제와 추출되는 특징의 수가 제한되는 한계를 가지고 있다. 또한 자료의 집단이 단일 모드의 정규 분포로부터 얻어진 것으로 가정되며 그렇지 않은 경우에 대해서는 적절한 결과를 얻을 수 없다. 본 논문에서는 자료-자료 간의 거리에 기반하고 적절하게 가중치가 추가된 새로운 행렬을 정의하였으며. 이에 기반하여 특징을 추출하는 방법을 제안하였다. 그럼으로써 앞서 선형판별분석의 여러 문제를 해결하고자 시도하였다. 제안된 방법의 성능을 실험을 통해 확인하였다. This paper presents a new feature extraction algorithm which can deal with the problems of linear discriminant analysis, widely used for linear dimensionality reduction. The scatter matrices included in linear discriminant analysis are defined by the distances between each datum and its class mean, and those between class means and mean of whole data. Use of these scatter matrices can cause computational problems and the limitation on the number of features. In addition, these definition assumes that the data distribution is unimodal and normal, for the cases not satisfying this assumption the appropriate features are not achieved. In this paper we define a new scatter matrix which is based on the differently weighted distances between individual data, and presents a feature extraction algorithm using this scatter matrix. With this new method. the mentioned problems of linear discriminant analysis can be avoided, and the features appropriate for discriminating data can be achieved. The performance of this new method is shown by experiments.

Optimization of Polarimetric Contrast Enhancement Based on Fisher Criterion

The optimization of polarimetric contrast enhancement (OPCE) is a widely used method for maximizing the received power ratio of a desired target versus an undesired target (clutter). In this letter, a new model of the OPCE is proposed based on the Fisher criterion. By introducing the well known two-class problem of linear discriminant analysis (LDA), the proposed model is to enlarge the normalized distance of mean value between the target and the clutter. In addition, a cross-iterative numerical method is proposed for solving the optimization with a quadratic constraint. Experimental results with the polarimetric SAR (POLSAR) data demonstrate the effectiveness of the proposed method.

FAST CLUSTERING-BASED KERNEL FOLEY–SAMMON TRANSFORM

The Kernel Foley–Sammon Transform (KFST) performs well in solving nonlinear discriminant analysis problems. However, as a kernel method, KFST also faces with large kernel matrix calculation problems O(n3) with the sample size n. KFST will be very costly and even intractable to compute when n is large. In this paper, we propose a Fast Clustering-based Kernel Foley–Sammon Transform (FCKFST) approach to tackle this problem. FCKFST solves KFST over a reductive l × l matrix representing clustering data instead of an n × n matrix of the original data, where l is exceedingly smaller than n. This paper also proves that FCKFST improving the calculating efficiency does not decrease the classification precision with comparison to KFST. We apply our method to digit and image recognition problems, and we obtain good experimental results.

Kernel Uncorrelated and Regularized Discriminant Analysis: A Theoretical and Computational Study

Linear and kernel discriminant analyses are popular approaches for supervised dimensionality reduction. Uncorrelated and regularized discriminant analyses have been proposed to overcome the singularity problem encountered by classical discriminant analysis. In this paper, we study the properties of kernel uncorrelated and regularized discriminant analyses, called KUDA and KRDA, respectively. In particular, we show that under a mild condition, both linear and kernel uncorrelated discriminant analysis project samples in the same class to a common vector in the dimensionality-reduced space. This implies that uncorrelated discriminant analysis may suffer from the overfitting problem if there are a large number of samples in each class. We show that as the regularization parameter in KRDA tends to zero, KRDA approaches KUDA. This shows that KUDA is a special case of KRDA and that regularization can be applied to overcome the overfitting problem in uncorrelated discriminant analysis. As the performance of KRDA depends on the value of the regularization parameter, we show that the matrix computations involved in KRDA can be simplified, so that a large number of candidate values can be cross-validated efficiently. Finally, we conduct experiments to evaluate the proposed theories and algorithms.

Solving the Problem of Discriminant Analysis under the Condition of Structural Uncertainty on the Basis of Group Method of Data Handling

Solving the Problem of Discriminant Analysis under the Condition of Structural Uncertainty on the Basis of Group Method of Data Handling

Graphical Tools for Quadratic Discriminant Analysis

Sufficient dimension-reduction methods provide effective ways to visualize discriminant analysis problems. For example, Cook and Yin showed that the dimension-reduction method of sliced average variance estimation (SAVE) identifies variates that are equivalent to a quadratic discriminant analysis (QDA) solution. This article makes this connection explicit to motivate the use of SAVE variates in exploratory graphics for discriminant analysis. Classification can then be based on the SAVE variates using a suitable distance measure. If the chosen measure is Mahalanobis distance, then classification is identical to QDA using the original variables. Just as canonical variates provide a useful way to visualize linear discriminant analysis (LDA), so do SAVE variates help visualize QDA. This would appear to be particularly useful given the lack of graphical tools for QDA in current software. Furthermore, whereas LDA and QDA can be sensitive to nonnormality, SAVE is more robust.