Discriminant Analysis Framework Research Articles

Discriminative sparse subspace learning with manifold regularization

Common subspace learning methods only utilize local or global structure in feature extraction, and cannot obtain the global optimal discriminative projection matrix. For this reason, this paper proposes a discriminative sparse subspace learning method based on the manifold regularization framework (DSSL-MR), which introduces the graph Laplacian matrix that reflects the intrinsic geometric structure of the sample as a penalty term. DSSL-MR simultaneously uses both sub-manifold and multi-manifold information of samples for obtaining optimal projection to enhance the discriminability of different classes in subspace. DSSL-MR uses the sparse property of the L2,1-norm to constrain the projection matrix, which can eliminate redundant features and select features that are significant for classification. It is a linear supervised method, which belongs to the Fisher discriminant analysis framework. Experimental results on multiple real-world datasets show that the algorithm is very effective in classification and has high recognition rates.

ML-FGAT: Identification of multi-label protein subcellular localization by interpretable graph attention networks and feature-generative adversarial networks

The prediction of multi-label protein subcellular localization (SCL) is a pivotal area in bioinformatics research. Recent advancements in protein structure research have facilitated the application of graph neural networks. This paper introduces a novel approach termed ML-FGAT. The approach begins by extracting node information of proteins from sequence data, physical-chemical properties, evolutionary insights, and structural details. Subsequently, various evolutionary techniques are integrated to consolidate multi-view information. A linear discriminant analysis framework, grounded on entropy weight, is then employed to reduce the dimensionality of the merged features. To enhance the robustness of the model, the training dataset is augmented using feature-generative adversarial networks. For the primary prediction step, graph attention networks are employed to determine multi-label protein SCL, leveraging both node and neighboring information. The interpretability is enhanced by analyzing the attention weight parameters. The training is based on the Gram-positive bacteria dataset, while validation employs newly constructed datasets: human, virus, Gram-negative bacteria, plant, and SARS-CoV-2. Following a leave-one-out cross-validation procedure, ML-FGAT demonstrates noteworthy superiority in this domain.

Evaluating brain modularity benefits of an acting intervention: a discriminant-analysis framework.

Aging is associated with a reduction in brain modularity as well as aspects of executive function, namely, updating, shifting, and inhibition. Previous research has suggested that the aging brain exhibits plasticity. Further, it has been hypothesized that broad-based intervention models may be more effective in eliciting overall gains in executive function than interventions targeted at specific executive skills (e.g., computer-based training). To this end, we designed a 4-week theater-based acting intervention in older adults within an RCT framework. We hypothesized that older adults would show improvements in brain modularity and aspects of executive function, ascribed to the acting intervention. The participants were 179 adults from the community, aged 60-89 years and on average, college educated. They completed a battery of executive function tasks and resting state functional MRI scans to measure brain network modularity pre- and post-intervention. Participants in the active intervention group (n = 93) enacted scenes with a partner that involved executive function, whereas the active control group (n = 86) learned about the history and styles of acting. Both groups met two times/week for 75-min for 4 weeks. A mixed model was used to evaluate intervention effects related to brain modularity. Discriminant-analysis was used to determine the role of seven executive functioning tasks in discriminating the two groups. These tasks indexed subdomains of updating, switching, and inhibition. Discriminant tasks were subject to a logistic regression analysis to determine how post-intervention executive function performance interacted with changes in modularity to predict group membership. We noted an increase in brain modularity in the acting group, relative to pre-intervention and controls. Performance on updating tasks were representative of the intervention group. However, post-intervention performance on updating did not interact with the observed increase in brain modularity to distinguish groups. An acting intervention can facilitate improvements in modularity and updating, both of which are sensitive to aging and may confer benefits to daily functioning and the ability to learn.

Fast Locality Discriminant Analysis With Adaptive Manifold Embedding.

Linear discriminant analysis (LDA) has been proven to be effective in dimensionality reduction. However, the performance of LDA depends on the consistency assumption of the global structure and the local structure. Some work extended LDA along this line of research and proposed local formulations of LDA. Unfortunately, the learning scheme of these algorithms is suboptimal in that the intrinsic relationship between data points is pre-learned in the original space, which is usually affected by the noise and redundant features. Besides, the time cost is relatively high. To alleviate these drawbacks, we propose a Fast Locality Discriminant Analysis framework (FLDA), which has three advantages: (1) It can divide a non-Gaussian distribution class into many sub-blocks that obey Gaussian distributions by using the anchor-based strategy. (2) It captures the manifold structure of data by learning the fuzzy membership relationship between data points and the corresponding anchor points, which can reduce computation time. (3) The weights between data points and anchor points are adaptively updated in the subspace where the irrelevant information and the noise in high-dimensional space have been effectively suppressed. Extensive experiments on toy data sets, UCI benchmark data sets and imbalanced data sets demonstrate the efficiency and effectiveness of the proposed method.

Nonparametric discriminant analysis with network structures in predictor

ABSTRACT Multiclassification, known as classification for multi-label responses, has been an important problem in supervised learning and has attracted our attention. Discriminant analysis (DA) is a popular method to deal with multiclassification. With the increasing availability of complex data, it becomes more challenging to analyse them. One of the important features in complex data is the network structure, which is ubiquitous in high-dimensional data because of strong or weak correlations among variables. In addition, in the framework of DA, an assumption of normal distributions is imposed on the predictors, but it is usually invalid in applications. To relax the normality assumption, we propose a nonparametric discriminant function to address multiclassification. In addition, to incorporate the network structure and improve the accuracy of classification, we develop three different network-based surrogate predictors to replace conventional predictors. The key features of the proposed method include the incorporation of network structures in predictors and allowance of predictors to follow exponential family distributions. Finally, numerical studies, including simulation and real data analysis, are conducted to assess the performance of the proposed method.

Feature reduction based transfer structural subspace learning for small-footprint cross-domain keyword spotting via linear discriminant analysis

Small-footprint keyword spotting has received considerable attention in recent years, which is often conducted on the assumption that the predefined keywords in training and testing data are obtained under the same condition. However, in practical situations, this assumption does not hold due to the widespread existence of various speech scenarios or voice-embedded devices. To tackle this problem, in this paper, we propose a new transfer subspace learning method called feature reduction based transfer structural subspace learning (FRTSSL) for small-footprint cross-domain keyword spotting. FRTSSL aims to learn a domain-invariant and discriminative subspace by which (1) feature reduction is used to high dimensional features to avoid unnecessary computation; and (2) transfer structural subspace learning jointly exploits the statistical properties and geometric structure to reduce the distribution discrepancy of the source and target domains based on a joint linear discriminant analysis (LDA) framework; and (3) the feedback term is constructed for improving the discrimination of the subspace based on source labels and pseudo-target labels. To preserve the intrinsic geometric structure of samples in the projection subspace, we first preserve the global subspace structure by imposing the reconstruction constraints on the reconstruction coefficient matrix, and then we preserve the space relationship of samples using a graph regularization method. Furthermore, we formulate a minimization problem that integrates marginal and conditional distribution alignment, reconstruction constraints, and graph regularization into the joint LDA framework, giving an effective optimization algorithm. Experimental results on four cross-domain keyword datasets show that our method outperforms some state-of-the-art conventional transfer learning methods and no transfer learning methods.

Self-weighted learning framework for adaptive locality discriminant analysis

Linear discriminant analysis (LDA) is one of the most important dimensionality reduction techniques and applied in many areas. However, traditional LDA algorithms aim to capture the global structure from data and ignore the local information. That may lead to the failure of LDA in some real-world datasets which have a complex geometry distribution. Although there are many previous works that focus on preserving the local information, they are all stuck in the same problem that the neighbor relationships of pairwise data points obtained from the original space may not be reliable, especially in the case of heavy noise. Therefore, we proposed a novel self-weighted learning framework, named Self-Weighted Adaptive Locality Discriminant Analysis (SALDA), for locality-aware based dimensionality reduction. The proposed framework can adaptively learn an intrinsic low-dimensional subspace, so that we can explore the better neighbor relationships for samples under the ideal subspace. In addition, our model can automatically learn to assign the weights to data pairwise points within the same class and takes no extra parameters compared to other classical locality-aware methods. At last, the experimental results on both synthetic and real-world benchmark datasets demonstrate the effectiveness and superiority of the proposed algorithm.

Speaker-Phrase-Specific Adaptation of PLDA Model for Improved Performance in Text-Dependent Speaker Verification

The i-vector/probabilistic linear discriminant analysis (PLDA) framework has been popularly used in the field of speaker verification for a long time. Lately, the introduction of online i-vectors and its integration with dynamic time warping template matching technique have significantly improved the performance of text-dependent speaker verification system. The PLDA model learns to discriminate among instances of different speaker-phrase classes and also compensates for channel and session variability. However, when exposed to unseen speakers and text, the variability compensation model turns less than optimal, leading to substantial verification error. In this paper, PLDA adaptation, in order to incorporate the idea of speaker-phrase-dependent variability in the ivector/PLDA technique, has been proposed. The adapted model gets specifically tuned to particular speaker-phrase class, leading to a more optimal solution. Two adaptation techniques, namely interpolation and weighted likelihood, have been explored in this work. Experiments have been performed on Part 1 of the RSR2015 database, and relative equal error rate (EER) reductions of up to 58.22% and 45% have been observed for interpolation and weighted likelihood techniques, respectively. The use of speaker-phrase-specific mean and whitening parameters has led to further improvement, resulting in EER reduction of up to 20% relative to that of the adapted models.

Accurate prediction of multi-label protein subcellular localization through multi-view feature learning with RBRL classifier.

Multi-label proteins can participate in carrier transportation, enzyme catalysis, hormone regulation and other life activities. Meanwhile, they play a key role in the fields of biopharmaceuticals, gene and cell therapy. This article proposes a prediction method called Mps-mvRBRL to predict the subcellular localization (SCL) of multi-label protein. Firstly, pseudo position-specific scoring matrix, dipeptide composition, position specific scoring matrix-transition probability composition, gene ontology and pseudo amino acid composition algorithms are used to obtain numerical information from different views. Based on the contribution of five individual feature extraction methods, differential evolution is used for the first time to learn the weight of single feature, and then these original features use a weighted combination method to fuse multi-view information. Secondly, the fused high-dimensional features use a weighted linear discriminant analysis framework based on binary weight form to eliminate irrelevant information. Finally, the best feature vector is input into the joint ranking support vector machine and binary relevance with robust low-rank learning classifier to predict the SCL. After applying leave-one-out cross-validation, the overall actual accuracy (OAA) and overall location accuracy (OLA) of Mps-mvRBRL on the training set of Gram-positive bacteria are both 99.81%. The OAA on the test sets of plant, virus and Gram-negative bacteria datasets are 97.24%, 98.55% and 98.20%, respectively, and the OLA are 97.16%, 97.62% and 98.28%, respectively. The results show that the model achieves good prediction performance for predicting the SCL of multi-label protein.

MpsLDA-ProSVM: Predicting multi-label protein subcellular localization by wMLDAe dimensionality reduction and ProSVM classifier

Multi-label proteins play a significant role in life processes such as cell growth, development, and reproduction. Exploring protein subcellular localization (SCL) is a direct way to better understand the function of multi-label proteins in cells. This paper firstly presents a new prediction model named MpsLDA-ProSVM which predicts the SCL of multi-label proteins. Firstly, we utilize four coding algorithms including pseudo position-specific scoring matrix (PsePSSM), gene ontology (GO), conjoint triad (CT) and pseudo amino acid composition (PseAAC) to draw the feature information from protein sequences. Then, for the first time, we use a weighted multi-label linear discriminant analysis framework based on entropy weight form (wMLDAe) to refine and purify features. Finally, we input the optimal feature subset into the multi-label learning with label-specific features (LIFT) and multi-label k-nearest neighbor (ML-KNN) algorithms to obtain a synthetic ranking of relevant labels, and then use Prediction and Relevance Ordering based SVM (ProSVM) classifier to predict the SCLs. Tested by leave-one-out cross-validation (LOOCV), the overall actual accuracy on virus, plant, Gram-positive bacteria and Gram-negative bacteria datasets are 98.06%, 98.97%, 99.81% and 98.49%, which are 0.56%–9.16%, 1.07%–30.87%, 0.21%–6.91% and 3.99%–8.59% higher than other advanced methods respectively. By comparison, the model MpsLDA-ProSVM can effectively predict the specific location of multi-label proteins in cells.

Attribute np Control Charts using Resampling Systems for Monitoring Non-Conforming Items using Exponentiated Half Logistic Distribution

This paper is centred on a binary classification problem in which it is desired to assign a new object with multivariate features to one of two distinct populations as based on historical sets of samples from two populations. A linear discriminant analysis framework has been proposed, called the minimised sum of deviations by proportion (MSDP) to model the binary classification problem. In the MSDP formulation, the sum of the proportion of exterior deviations is minimised subject to the group separation constraints, the normalisation constraint, the upper bound constraints on proportions of exterior deviations and the sign unrestriction vis-à-vis the non-negativity constraints. The two-phase method in linear programming is adopted as a solution technique to generate the discriminant function. The decision rule on group-membership prediction is constructed using the apparent error rate. The performance of the MSDP has been compared with some existing linear discriminant models using a previously published dataset on road casualties. The MSDP model was more promising and well suited for the imbalanced dataset on road casualties.

On the binary classification problem in discriminant analysis using linear programming methods

This paper is centred on a binary classification problem in which it is desired to assign a new object with multivariate features to one of two distinct populations as based on historical sets of samples from two populations. A linear discriminant analysis framework has been proposed, called the minimised sum of deviations by proportion (MSDP) to model the binary classification problem. In the MSDP formulation, the sum of the proportion of exterior deviations is minimised subject to the group separation constraints, the normalisation constraint, the upper bound constraints on proportions of exterior deviations and the sign unrestriction vis-à-vis the non-negativity constraints. The two-phase method in linear programming is adopted as a solution technique to generate the discriminant function. The decision rule on group-membership prediction is constructed using the apparent error rate. The performance of the MSDP has been compared with some existing linear discriminant models using a previously published dataset on road casualties. The MSDP model was more promising and well suited for the imbalanced dataset on road casualties.

Task-Driven Variability Model for Speaker Verification

The total variability model (TVM)/probabilistic linear discriminant analysis (PLDA) framework is one of the most popular methods for speaker verification. In this framework, the i-vector representations are first extracted from utterances via an estimated TVM and then employed to estimate the PLDA parameters for classification. The TVM and PLDA are estimated serially, so the information loss in the TVM is inherited by the i-vectors, and then passed into the PLDA classifier. More seriously, this loss cannot be compensated by the PLDA. To solve this problem, we propose a task-driven variability model (TDVM) to jointly estimate the TVM and PLDA classifier. In this method, the feedback from the PLDA can supervise the optimal solution of the TVM to move toward the space that has the maximum between-class separation and minimum within-class variation. Meanwhile, this space is suitable for open-set test which can deal with unenrolled speakers. Unlike most embedding methods which extract the embedding representations via the stack of network structures, the TDVM contains the assumptions about latent variables, which can enhance the interpretation of speaker representation extraction. The proposed method is evaluated on the King-ASR-010 and VoxCeleb databases, and the experimental results show that the TDVM method can achieve better performance than the traditional TVM/PLDA and VGG-M network with different cost functions.

Gaussian Process Graph-Based Discriminant Analysis for Hyperspectral Images Classification

Dimensionality Reduction (DR) models are highly useful for tackling Hyperspectral Images (HSIs) classification tasks. They mainly address two issues: the curse of dimensionality with respect to spectral features, and the limited number of labeled training samples. Among these DR techniques, the Graph-Embedding Discriminant Analysis (GEDA) framework has demonstrated its effectiveness for HSIs feature extraction. However, most of the existing GEDA-based DR methods largely rely on manually tuning the parameters so as to obtain the optimal model, which proves to be troublesome and inefficient. Motivated by the nonparametric Gaussian Process (GP) model, we propose a novel supervised DR algorithm, namely Gaussian Process Graph-based Discriminate Analysis (GPGDA). Our algorithm takes full advantage of the covariance matrix in GP to constructing the graph similarity matrix in GEDA framework. In this way, more superior performance can be provided with the model parameters tuned automatically. Experiments on three real HSIs datasets demonstrate that the proposed GPGDA outperforms some classic and state-of-the-art DR methods.

Enhanced Grassmann discriminant analysis with randomized time warping for motion recognition

This paper proposes a framework for classifying motion sequences, by extending the framework of Grassmann discriminant analysis (GDA). A problem of GDA is that its discriminant space is not necessarily optimal. This limitation becomes even more prominent when utilizing the subspace representation of randomized time warping (RTW). RTW is a sequence representation that can effectively model a motion’s temporal information by a low-dimensional subspace, simplifying the problem of comparing two sequences to that of comparing two subspaces. The key idea of the proposed enhanced GDA is projecting class subspaces onto a generalized difference subspace before mapping them on a Grassmann manifold. The GDS projection can remove overlapping components of the subspaces in the vector space, nearly orthogonalizing them. Consequently, a dictionary of orthogonalized class subspaces produces a set of more discriminant data points in the Grassmann manifold, in comparison with the original set. This set of data points can further enhance the discriminant ability of GDA. We demonstrate the validity of the proposed framework, RTW+eGDA, through experiments on motion recognition using the publicly available Cambridge gesture, KTH action, and UCF sports datasets.

Multiclass linear discriminant analysis with ultrahigh-dimensional features.

Within the framework of Fisher's discriminant analysis, we propose a multiclass classification method which embeds variable screening for ultrahigh-dimensional predictors. Leveraging interfeature correlations, we show that the proposed linear classifier recovers informative features with probability tending to one and can asymptotically achieve a zero misclassification rate. We evaluate the finite sample performance of the method via extensive simulations and use this method to classify posttransplantation rejection types based on patients' gene expressions.

Laplacian Regularized Spatial-Aware Collaborative Graph for Discriminant Analysis of Hyperspectral Imagery

Dimensionality Reduction (DR) models are of significance to extract low-dimensional features for Hyperspectral Images (HSIs) data analysis where there exist lots of noisy and redundant spectral features. Among many DR techniques, the Graph-Embedding Discriminant Analysis framework has demonstrated its effectiveness for HSI feature reduction. Based on this framework, many representation based models are developed to learn the similarity graphs, but most of these methods ignore the spatial information, resulting in unsatisfactory performance of DR models. In this paper, we firstly propose a novel supervised DR algorithm termed Spatial-aware Collaborative Graph for Discriminant Analysis (SaCGDA) by introducing a simple but efficient spatial constraint into Collaborative Graph-based Discriminate Analysis (CGDA) which is inspired by recently developed Spatial-aware Collaborative Representation (SaCR). In order to make the representation of samples on the data manifold smoother, i.e., similar pixels share similar representations, we further add the spectral Laplacian regularization and propose the Laplacian regularized SaCGDA (LapSaCGDA), where the two spectral and spatial constraints can exploit the intrinsic geometric structures embedded in HSIs efficiently. Experiments on three HSIs data sets verify that the proposed SaCGDA and LapSaCGDA outperform other state-of-the-art methods.

Sparse graphical representation based discriminant analysis for heterogeneous face recognition

Face images captured in heterogeneous environments, e.g., sketches generated by the artists or composite-generation software, photos taken by common cameras and infrared images captured by corresponding infrared imaging devices, usually subject to large texture (i.e., style) differences. This results in heavily degraded performance of conventional face recognition methods in comparison with the performance on images captured in homogeneous environments. In this paper, we propose a novel sparse graphical representation based discriminant analysis (SGR-DA) approach to address aforementioned face recognition in heterogeneous scenarios. An adaptive sparse graphical representation scheme is designed to represent heterogeneous face images, where a Markov networks model is constructed to generate adaptive sparse vectors. To handle the complex facial structure and further improve the discriminability, a spatial partition-based discriminant analysis framework is presented to refine the adaptive sparse vectors for face matching. We conducted experiments on six commonly used heterogeneous face datasets and experimental results illustrate that our proposed SGR-DA approach achieves superior performance in comparison with state-of-the-art methods.

Framework for dynamic hand gesture recognition using Grassmann manifold for intelligent vehicles

In this study, the authors propose a novel and robust approach to control auxiliary tasks in vehicles using hand gestures. First, they create a three-dimensional video volume by appending one frame to other that captures the motion history of frames. Then, they extract features using histogram of oriented gradients on each video volume. These features are represented in the form of subspaces on Grassmann manifold. To improve the recognition accuracy, they map the data from one manifold to another manifold with the help of a Grassmann kernel. Grassmann graph embedding discriminant analysis framework is used to classify the gestures. They perform experiments on two datasets: LISA and Cambridge Hand Gesture in three different testing methods such as 1/3-subject, 2/3-subject and cross-subject. Experimental results show that their proposed model outperforms and is comparable with the state-of-the-art methods.

A comparison of machine learning techniques for taxonomic classification of teeth from the Family Bovidae

This study explores the performance of machine learning algorithms on the classification of fossil teeth in the Family Bovidae. Isolated bovid teeth are typically the most common fossils found in southern Africa and they often constitute the basis for paleoenvironmental reconstructions. Taxonomic identification of fossil bovid teeth, however, is often imprecise and subjective. Using modern teeth with known taxons, machine learning algorithms can be trained to classify fossils. Previous work by Brophy et al. [Quantitative morphological analysis of bovid teeth and implications for paleoenvironmental reconstruction of plovers lake, Gauteng Province, South Africa, J. Archaeol. Sci. 41 (2014), pp. 376–388] uses elliptical Fourier analysis of the form (size and shape) of the outline of the occlusal surface of each tooth as features in a linear discriminant analysis (LDA) framework. This manuscript expands on that previous work by exploring how different machine learning approaches classify the teeth and testing which technique is best for classification. In addition to LDA, four other machine learning techniques were considered (neural networks, nuclear penalized multinomial regression,random forests, and support vector machines) with support vector machines and random forests performing the best in terms of log loss and classification rate.