Approximation-guided Fairness Testing through Discriminatory Space Analysis

In this paper, a framework of Discriminant Low-dimensional Subspace Analysis (DLSA) method is proposed to deal with the Small Sample Size (SSS) problem in face recognition area. Firstly, it is rigorously proven that the null space of the total covariance matrix, S t , is useless for recognition. Therefore, a framework of Fisher discriminant analysis in a low-dimensional space is developed by projecting all the samples onto the range space of S t . Two algorithms are proposed in this framework, i.e., Unified Linear Discriminant Analysis (ULDA) and Modified Linear Discriminant Analysis (MLDA). The ULDA extracts discriminant information from three subspaces of this lowdimensional space. The MLDA adopts a modified Fisher criterion which can avoid the singularity problem in conventional LDA. Experimental results on a large combined database have demonstrated that the proposed ULDA and MLDA can both achieve better performance than the other state-of-the-art LDA-based algorithms in recognition accuracy.

A generalized nonlinear discriminant analysis (GNDA) method is proposed, which implements Fisher discriminant analysis in a nonlinear mapping space. Linear discriminant analysis in the nonlinear mapping space corresponds to nonlinear discriminant analysis in an input space. GNDA suggests a unified framework of nonlinear discriminant analysis which includes the kernel Fisher discriminant analysis as a specific case. Experimental results on UCI data sets demonstrate the validity of our method.

In general, a two-dimensional display is defined by two orthogonal unit vectors. In developing the display, discriminant analysis has the shortcoming that, in general, the extracted axes are not orthogonal. In order to overcome this shortcoming, the authors propose a discriminant analysis which provides an orthonormal system in the transformed space. The transformation preserves the discriminatory ability in terms of the Fisher ratio. The authors present a necessary and sufficient condition under which discriminant analysis in the original space provides an orthonormal system. Relationships between orthogonal discriminant analysis and the Karhunen-Loeve expansion in the original space are investigated.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Linear discriminant analysis (LDA) has been widely applied for hyperspectral image (HSI) analysis as a popular method for feature extraction and dimensionality reduction. Linear methods such as LDA work well for unimodal Gaussian class-conditional distributions. However, when data samples between classes are nonlinearly separated in the input space, linear methods such as LDA are expected to fail. The kernel discriminant analysis (KDA) attempts to address this issue by mapping data in the input space onto a subspace such that Fisher's ratio in an intermediate (higher-dimensional) kernel-induced space is maximized. In recent studies with HSI data, KDA has been shown to outperform LDA, particularly when the data distributions are non-Gaussian and multimodal, such as when pixels represent target classes severely mixed with background classes. In this letter, a modified KDA algorithm, i.e., kernel local Fisher discriminant analysis (KLFDA), is studied for HSI analysis. Unlike KDA, KLFDA imposes an additional constraint on the mapping-it ensures that neighboring points in the input space stay close-by in the projected subspace and vice versa. Classification experiments with a challenging HSI task demonstrate that this approach outperforms current state-of-the-art HSI-classification methods.

In this paper, the 3D head model classification problem is addressed by use of a newly developed subspace analysis method: kernel clustering-based discriminant analysis or KCDA as an abbreviation. This method works by first mapping the original data into another high-dimensional space, and then performing clustering-based discriminant analysis in the feature space. The main idea of clustering-based discriminant analysis is to overcome the Gaussian assumption limitation of the traditional linear discriminant analysis by using a new criterion that takes into account the multiple cluster structure possibly embedded within some classes. As a result, Kernel CDA tries to get through the limitations of both Gaussian assumption and linearity facing the traditional linear discriminant analysis simultaneously. A novel application of this method in 3D head model classification is presented in this paper. A group of tests of our method on 3D head model dataset have been carried out, reporting very promising experimental results.Keywords3D head model classificationKernel Linear Discriminant Analysis (KLDA)Clustering-based Discriminant Analysis (CDA)Kernel Fuzzy c-meansKernel Clustering-based Discriminant Analysis (KCDA)

Partial robust M regression (PRM), as well as its sparse counterpart sparse PRM, have been reported to be regression methods that foster a partial least squares‐alike interpretation while having good robustness and efficiency properties, as well as a low computational cost. In this paper, the partial robust M discriminant analysis classifier is introduced, which consists of dimension reduction through an algorithm closely related to PRM and a consecutive robust discriminant analysis in the latent variable space. The method is further generalized to sparse partial robust M discriminant analysis by introducing a sparsity penalty on the estimated direction vectors. Thereby, an intrinsic variable selection is achieved, which yields a better graphical interpretation of the results, as well as more precise coefficient estimates, in case the data contain uninformative variables. Both methods are robust against leverage points within each class, as well as against adherence outliers (points that have been assigned a wrong class label). A simulation study investigates the effect of outliers, wrong class labels, and uninformative variables on the proposed methods and its classical PLS counterparts and corroborates the robustness and sparsity claims. The utility of the methods is demonstrated on data from mass spectrometry analysis (time‐of‐flight secondary ion mass spectrometry) of meteorite samples. Copyright © 2016 John Wiley &amp; Sons, Ltd.

This paper develops a supervised dimensionality reduction method, Lorentzian Discriminant Projection (LDP), for discriminant analysis and classification. Our method represents the structures of sample data by a manifold, which is furnished with a Lorentzian metric tensor. Different from classic discriminant analysis techniques, LDP uses distances from points to their within-class neighbors and global geometric centroid to model a new manifold to detect the intrinsic local and global geometric structures of data set. In this way, both the geometry of a group of classes and global data structures can be learnt from the Lorentzian metric tensor. Thus discriminant analysis in the original sample space reduces to metric learning on a Lorentzian manifold. The experimental results on benchmark databases demonstrate the effectiveness of our proposed method.KeywordsDimensionality reductionLorentzian geometrymetric learningdiscriminant analysis

In multivariate data analysis, Fisher linear discriminant analysis is useful to optimally separate two classes of observations by finding a linear combination of p variables. Functional data analysis deals with the analysis of continuous functions and thus can be seen as a generalisation of multivariate analysis where the dimension of the analysis space p strives to infinity. Several authors propose methods to perform discriminant analysis in this infinite dimensional space. Here, the methodology is introduced to perform discriminant analysis, not on single infinite dimensional functions, but to find a linear combination of p infinite dimensional continuous functions, providing a set of continuous canonical functions which are optimally separated in the canonical space.

Improved discriminate analysis for high-dimensional data and its application to face recognition

In our previous work, we have developed sparse least squares support vector machines (sparse LS SVMs) trained in the reduced empirical feature space, spanned by the independent training data selected by the Cholesky factorization. In this paper, we propose selecting the independent training data by forward selection based on linear discriminant analysis in the empirical feature space. Namely, starting from the empty set, we add a training datum that maximally separates two classes in the empirical feature space. To calculate the separability in the empirical feature space we use linear discriminant analysis (LDA), which is equivalent to kernel discriminant analysis in the feature space. If the matrix associated with the LDA is singular, we consider that the datum does not contribute to the class separation and permanently delete it from the candidates of addition. We stop the addition of data when the objective function of LDA does not increase more than the prescribed value. By computer experiments for two-class and multi-class problems we show that in most cases we can reduce the number of support vectors more than with the previous method.

This paper presents a method for recognizing sign language using hand movement trajectory. By applying Kernel Principal Component Analysis (KPCA), the motion trajectory data are firstly mapped into a higher-dimensional feature space for analysis. The advantage of using high dimensionality is that it allows a more flexible decision boundary and thus helps us to achieve better classification accuracy. Then we perform the Nonparametric Discriminant Analysis (NDA) in the higher-dimensional feature space, so that the most helpful information can be extracted. We have tested the proposed method using the Australian Sign Language (ASL) data set. The results demonstrate that our approach outperforms the current state-of-the-art for trajectory-based sign language recognition.

In Kernel Fisher discriminant analysis (KFDA), we carry out Fisher linear discriminant analysis in a high dimensional feature space defined implicitly by a kernel. The performance of KFDA depends on the choice of the kernel; in this paper, we consider the problem of finding the optimal kernel, over a given convex set of kernels. We show that this optimal kernel selection problem can be reformulated as a tractable convex optimization problem which interior-point methods can solve globally and efficiently. The kernel selection method is demonstrated with some UCI machine learning benchmark examples.

Correlation is one of the most widely used similarity measures in machine learning like Euclidean and Mahalanobis distances. However, compared with proposed numerous discriminant learning algorithms in distance metric space, only a very little work has been conducted on this topic using correlation similarity measure. In this paper, we propose a novel discriminant learning algorithm in correlation measure space, Correlation Discriminant Analysis (CDA). In this framework, based on the definitions of within-class correlation and between-class correlation, the optimum transformation can be sought for to maximize the difference between them, which is in accordance with good classification performance empirically. Under different cases of the transformation, different implementations of the algorithm are given. Extensive empirical evaluations of CDA demonstrate its advantage over alternative methods.

The kernel function plays a central role in kernel methods. In this paper, we consider the automated learning of the kernel matrix over a convex combination of pre-specified kernel matrices in Regularized Kernel Discriminant Analysis (RKDA), which performs lineardiscriminant analysis in the feature space via the kernel trick. Previous studies have shown that this kernel learning problem can be formulated as a semidefinite program (SDP), which is however computationally expensive, even with the recent advances in interior point methods. Based on the equivalence relationship between RKDA and least square problems in the binary-class case, we propose a Quadratically Constrained Quadratic Programming (QCQP) formulation for the kernel learning problem, which can be solved more efficiently than SDP. While most existing work on kernel learning deal with binary-class problems only, we show that our QCQP formulation can be extended naturally to the multi-class case. Experimental results on both binary-class and multi-class benchmarkdata sets show the efficacy of the proposed QCQP formulations.

We present a comprehensive approach to address three challenging problems in face recognition: modelling faces across multi-views, extracting the nonlinear discriminating features, and recognising moving faces dynamically in image sequences. A multi-view dynamic face model is designed to extract the shape-and-pose-free facial texture patterns. Kernel discriminant analysis, which employs the kernel technique to perform linear discriminant analysis in a high-dimensional feature space, is developed to extract the significant nonlinear features which maximise the between-class variance and minimise the within-class variance. Finally, an identity surface based face recognition is performed dynamically from video input by matching object and model trajectories. q 2003 Elsevier B.V. All rights reserved.