Discriminant Subspace Learning Research Articles

Discriminative Subspace Learning With Adaptive Graph Regularization

Abstract Many subspace learning methods based on low-rank representation employ the nearest neighborhood graph to preserve the local structure. However, in these methods, the nearest neighborhood graph is a binary matrix, which fails to precisely capture the similarity between distinct samples. Additionally, these methods need to manually select an appropriate number of neighbors, and they cannot adaptively update the similarity graph during projection learning. To tackle these issues, we introduce Discriminative Subspace Learning with Adaptive Graph Regularization (DSL_AGR), an innovative unsupervised subspace learning method that integrates low-rank representation, adaptive graph learning and nonnegative representation into a framework. DSL_AGR introduces a low-rank constraint to capture the global structure of the data and extract more discriminative information. Furthermore, a novel graph regularization term in DSL_AGR is guided by nonnegative representations to enhance the capability of capturing the local structure. Since closed-form solutions for the proposed method are not easily obtained, we devise an iterative optimization algorithm for its resolution. We also analyze the computational complexity and convergence of DSL_AGR. Extensive experiments on real-world datasets demonstrate that the proposed method achieves competitive performance compared with other state-of-the-art methods.

Cross-dataset discriminant subspace learning algorithm for apple leaf diseases identification

Apple has a large planting area and high yield in China, but it is easily affected by diseases. Artificial intelligence technology has achieved good results in apple leaf disease identification. However, the training and testing data in complex environments often come from different collection scenarios. There are significant differences in the training and testing datasets, which leads to a significant decline in the identification performance. To improve the generalization ability of apple leaf disease identification, we develop a cross-dataset discriminant subspace learning (CDDSL) algorithm by utilizing the idea of transfer learning, low-rank sparse representation, and maximum margin neighborhood preserving embedding (NPE) to cross-dataset scenarios. Firstly, based on the transfer learning and low-rank sparse representation, each target sample can be represented by a linear combination of source domain samples. The sparse constraint on the noise matrix weakens the influence of noise in the sample data. Then, CDDSL progressively projects the subspace features into the semantic space and establishes a discriminant classifier. Meanwhile, CDDSL utilizes the maximal margin NPE to improve the intraclass compactness and interclass separability of projection features, which can enhance the image identification performance. The experiments conducted on real-world apple leaf datasets verify the effectiveness of the CDDSL algorithm in cross-dataset scenario.

Discriminative sparse subspace learning with manifold regularization

Common subspace learning methods only utilize local or global structure in feature extraction, and cannot obtain the global optimal discriminative projection matrix. For this reason, this paper proposes a discriminative sparse subspace learning method based on the manifold regularization framework (DSSL-MR), which introduces the graph Laplacian matrix that reflects the intrinsic geometric structure of the sample as a penalty term. DSSL-MR simultaneously uses both sub-manifold and multi-manifold information of samples for obtaining optimal projection to enhance the discriminability of different classes in subspace. DSSL-MR uses the sparse property of the L2,1-norm to constrain the projection matrix, which can eliminate redundant features and select features that are significant for classification. It is a linear supervised method, which belongs to the Fisher discriminant analysis framework. Experimental results on multiple real-world datasets show that the algorithm is very effective in classification and has high recognition rates.

Discriminative subspace learning using generalized mean

Linear discriminant analysis (LDA) is one of the most popular methods to extract discriminative features because it is simple and powerful. However, LDA fails to learn a discriminative subspace in some cases. This study deals with a problem of LDA, the so-called class separation (CS) problem, which means that some classes located close to each other in the original input space tend to overlap in a learned subspace. This problem can also happen in a heteroscedastic extension of LDA, the oriented discriminant analysis (ODA). To alleviate the problem, we propose two methods to maximize the generalized mean instead of the arithmetic mean in the objective functions. Experimental results show that the proposed methods can obtain better discriminative subspaces than LDA, ODA, and other alternatives designed to solve the CS problem.

Numerically stable locality-preserving partial least squares discriminant analysis for efficient dimensionality reduction and classification of high-dimensional data

Dimensionality reduction plays a pivotal role in preparing high-dimensional data for classification and discrimination tasks by eliminating redundant features and enhancing the efficiency of classifiers. The effectiveness of a dimensionality reduction algorithm hinges on its numerical stability. When data projections are numerically stable, they lead to enhanced class separability in the lower-dimensional embedding, consequently yielding higher classification accuracy. This paper investigates the numerical attributes of dimensionality reduction and discriminant subspace learning, with a specific focus on Locality-Preserving Partial Least Squares Discriminant Analysis (LPPLS-DA). High-dimensional data frequently introduce singularity in the scatter matrices, posing a significant challenge. To tackle this issue, the paper explores two robust implementations of LPPLS-DA. These approaches not only optimize data projections but also capture more discriminative features, resulting in a marked improvement in classification accuracy. Empirical evidence supports these findings through numerical experiments conducted on synthetic and spectral datasets. The results demonstrate the superior performance of the proposed methods when compared to several state-of-the-art dimensionality reduction techniques in terms of both classification accuracy and dimension reduction.

Pseudo-Label Guided Structural Discriminative Subspace Learning for Unsupervised Feature Selection.

In this article, we propose a new unsupervised feature selection method named pseudo-label guided structural discriminative subspace learning (PSDSL). Unlike the previous methods that perform the two stages independently, it introduces the construction of probability graph into the feature selection learning process as a unified general framework, and therefore the probability graph can be learned adaptively. Moreover, we design a pseudo-label guided learning mechanism, and combine the graph-based method and the idea of maximizing the between-class scatter matrix with the trace ratio to construct an objective function that can improve the discrimination of the selected features. Besides, the main existing strategies of selecting features are to employ l2,1 -norm for feature selection, but this faces the challenges of sparsity limitations and parameter tuning. For addressing this issue, we employ the l2,0 -norm constraint on the learned subspace to ensure the row sparsity of the model and make the selected feature more stable. Effective optimization strategy is given to solve such NP-hard problem with the determination of parameters and complexity analysis in theory. Ultimately, extensive experiments conducted on nine real-world datasets and three biological ScRNA-seq genes datasets verify the effectiveness of the proposed method on the data clustering downstream task.

Discriminative latent subspace learning with adaptive metric learning

Discriminative latent subspace learning with adaptive metric learning

Graph adaptive semi-supervised discriminative subspace learning for EEG emotion recognition

Since Electroencephalogram (EEG) is resistant to camouflage and contains abundant neurophysiological information, it shows significant superiorities in objective emotion recognition, making EEG-based emotion recognition become a hot research field in brain-computer interface research. However, EEG is generally non-stationary and has a low signal-to-noise ratio, which is difficult to analyze. Inspired by the consensus that exploring a discriminative subspace representation usually helps to capture the semantic information of EEG data, in this paper we propose a Graph Adaptive Semi-supervised Discriminative Subspace Learning (GASDSL) model for EEG-based emotion recognition. GASDSL aims to explore a discriminative subspace in which the intra-class scatter decreases while the inter-class separability increases. The adaptive maximum entropy graph construction and semi-supervised subspace emotional state prediction are adopted to mediate the discriminative subspace learning. Extensive comparative studies on the SEED-IV and SEED-V datasets depict that 1) GASDSL achieved satisfactory emotion recognition accuracy compared with other semi-supervised learning models, 2) the discriminative abilities of both the learned maximum entropy graph and subspace are improved as the model iterates, and 3) the features extracted from the Gamma band, the left/right temporal, prefrontal, and (central) parietal lobes contributed more to emotion recognition based on the spatial-frequency pattern analysis results.

Age invariant face recognition using discriminative tensor subspace learning with fuzzy based classification

AbstractAging is a complex process that affects both the shape and texture of a face. In the aging domain, various methods have been proposed for age estimation or simulation of the face based on an image at a given age. Face recognition methods that are designed for automatic face recognition are still a challenging issue. We present a novel age invariant face recognition method using tensor subspace learning with fuzzy synthetic classification. Local Binary Pattern (LBP) processed face images are used to learn the tensor sub‐space, which converts high dimensional feature space to age‐invariants while retaining key elements of the local geometrical structure of the face. Tensor sub‐space is better in terms of sub‐space learning and feature extraction than Principal Component Analysis (PCA)/Linear Discriminant Analysis (LDA) that been used in many techniques in the literature. Tensor Normalized Face images are generated which exhibit maximum inter‐class distances and minimum intra‐class distances. Further local patches are used for synthesizing global Fuzzy membership scores to classify the test face images. Experiments performed on standard face‐aging datasets, namely FG‐NET, AGEDB, and MORPH‐Album‐II, and received accuracy of 99.15%, 99.20%, and 99.8%, respectively. Experimental results outperform the current state‐of‐the‐art techniques, and results show the promise of the proposed system for personal identification despite the aging process. It also proved that the local descriptor gives better performance over the global descriptor like PCA for the aging process. The method also demonstrated improved performance as compared with compute‐intensive methods that required training on deep networks.

Discriminative subspace learning via optimization on Riemannian manifold

Discriminative subspace learning is an important problem in machine learning, which aims to find the maximum separable decision subspace. Traditional Euclidean-based methods usually use Fisher discriminant criterion for finding an optimal linear mapping from a high-dimensional data space to a lower-dimensional subspace, which hardly guarantee a quadratic rate of global convergence and suffers from the singularity problem. Here, we propose the manifold optimization-based discriminant analysis (MODA) which is constructed by using the latent subspace alignment and the geometry of objective function with orthogonality constraint. MODA is solved by using Riemannian version of trust-region algorithm. Experimental results on various image datasets and electroencephalogram (EEG) datasets show that MODA achieves the best separability and is significantly superior to the competing algorithms. Especially for the time series of EEG signals, the accuracy of MODA is 20–30% higher than existing algorithms. The code for MODA is available at https://github.com/ncclabsustech/MODA-algorithm.

Multiview nonlinear discriminant structure learning for emotion recognition

Multiview subspace learning (MSL) has been widely used in various practical applications including emotion recognition. Despite the recent progress in MSL, two challenges remain to address. First, most existing MSL methods indiscriminately utilize both helpful and defective information contained in different views. Second, the most recent methods are linear approaches that do not perform well on emotion datasets with weak linear separability. Therefore, in this study, we introduce a framework for emotion recognition: multiview nonlinear discriminant structure learning (MNDSL). MNDSL fully exploits useful information in each input through local information preservation and discriminant reconstruction (LPDR) and obtains latent subspaces using multiview discriminant latent subspace learning (MDLSL). In addition, an out-of-sample extension was introduced to satisfy the requirements of large-scale applications and obtain the projections of new samples. The proposed framework constructs interviews and intra-view-weighted connections to explore discriminant structures and preserve locality under complementarity and correlation principles. The results demonstrate the superiority of the proposed framework compared with state-of-the-art methods.

Discriminative aging subspace learning for age estimation

Human age estimation from facial images has become an active research topic in computer vision field because of various real-world applications. Temporal property of facial aging display sequential patterns that lie on the low-dimensional aging manifold. In this paper, we propose hidden factor analysis (HFA) model-based discriminative manifold learning method for age estimation. The hidden factor analysis decomposes facial features into independent age factor and identity factor. Various age invariant face recognition systems in the literature utilize identity factor for face recognition; however, the age factor remains unutilized. The age component of the hidden factor analysis model depends on the subject’s age. Thus it carries significant age-related information. In this paper, we demonstrate that such aging patterns can be effectively extracted from the HFA-based discriminant subspace learning algorithm. Next, we have applied multiple regression methods on low-dimensional aging features learned from the HFA model. Effect of reduced dimensionality on the accuracy has been evaluated by extensive experiments and compared with the state-of-the-art methods. Effectiveness and robustness in terms of MAE and CS of the proposed framework are demonstrated using experimental analysis on a large-scale aging database MORPH II. The accuracy of our method is found superior to the current state-of-the-art methods.

Neighborhood Preserving and Weighted Subspace Learning Method for Drift Compensation in Gas Sensor

This article presents a novel discriminative subspace-learning-based unsupervised domain adaptation (DA) method for the gas sensor drift problem. Many existing subspace learning approaches assume that the gas sensor data follow a certain distribution such as Gaussian, which often does not exist in real-world applications. In this article, we address this issue by proposing a novel discriminative subspace learning method for DA with neighborhood preserving (DANP). We introduce two novel terms, including the intraclass graph term and the interclass graph term, to embed the graphs into DA. Besides, most existing methods ignore the influence of the subspace learning on the classifier design. To tackle this issue, we present a novel classifier design method (DANP+) that incorporates the DA ability of the subspace into the learning of the classifier. The weighting function is introduced to assign different weights to different dimensions of the subspace. We have verified the effectiveness of the proposed methods by conducting experiments on two public gas sensor datasets in comparison with the state-of-the-art DA methods.

A Cross-Media Advertising Design and Communication Model Based on Feature Subspace Learning.

This paper uses feature subspace learning and cross-media retrieval analysis to construct an advertising design and communication model. To address the problems of the traditional feature subspace learning model and make the samples effectively maintain their local structure and discriminative properties after projection into the feature space, this paper proposes a discriminative feature subspace learning model based on Low-Rank Representation (LRR), which explores the local structure of samples through Low-Rank Representation and uses the representation coefficients as similarity constraints of samples in the projection space so that the projection subspace can better maintain the local nearest-neighbor relationship of samples. Based on the common subspace learning, this paper uses the extreme learning machine method to improve the cross-modal retrieval accuracy, mining deeper data features and maximizing the correlation between different modalities, so that the learned shared subspace is more discriminative; meanwhile, it proposes realizing cross-modal retrieval by the deep convolutional generative adversarial network, using unlabeled samples to further explore the correlation of different modal data and improve the cross-modal performance. The clustering quality of images and audios is corrected in the feature subspace obtained by dimensionality reduction through an optimization algorithm based on similarity transfer. Three active learning strategies are designed to calculate the conditional probability of unannotated samples around user-annotated samples in the correlation feedback process, thus improving the efficiency of cross-media retrieval in the case of limited feedback samples. The experimental results show that the method accurately measures the cross-media relevance and effectively achieves mutual retrieval between image and audio data. Through the study of cross-media advertising design and communication models based on feature subspace learning, it is of positive significance to advance commercial advertising design by guiding designers and artists to better utilize digital media technology for artistic design activities at the level of theoretical research and applied practice.

Locality-Aware Discriminative Subspace Learning for Image Classification

Projection learning is an effective and widely used technique for extracting discriminative features for pattern recognition and classification. In projection learning, it is essential to preserve the global and local structure of the data while extracting discriminative features. However, transforming the source data directly to a target i.e., the strict binary label matrix, using a projection matrix may result in the loss of some intrinsic information. We propose a locality-aware discriminative subspace learning (LADSL) method to address these limitations. In LADSL, the original data is transformed into a latent space instead of a restrictive label space. The latent space seamlessly integrates the original visual features and the class labels to improve the classification performance. The projection matrix and classification parameters are jointly optimized to supervise the discriminative subspace learning. Additionally, LADSL exploits the adaptive local structure to preserve the nearest neighbor relationship among the data samples while learning more projections to achieve superior classification performance. Experiments have been carried out on various data sets for face and object recognition, and the results achieved are compared with the state-of-the-art methods to validate the effectiveness of the proposed LADSL method.

Learning Robust Discriminant Subspace Based on Joint L₂,ₚ- and L₂,ₛ-Norm Distance Metrics.

Recently, there are many works on discriminant analysis, which promote the robustness of models against outliers by using L1- or L2,1-norm as the distance metric. However, both of their robustness and discriminant power are limited. In this article, we present a new robust discriminant subspace (RDS) learning method for feature extraction, with an objective function formulated in a different form. To guarantee the subspace to be robust and discriminative, we measure the within-class distances based on [Formula: see text]-norm and use [Formula: see text]-norm to measure the between-class distances. This also makes our method include rotational invariance. Since the proposed model involves both [Formula: see text]-norm maximization and [Formula: see text]-norm minimization, it is very challenging to solve. To address this problem, we present an efficient nongreedy iterative algorithm. Besides, motivated by trace ratio criterion, a mechanism of automatically balancing the contributions of different terms in our objective is found. RDS is very flexible, as it can be extended to other existing feature extraction techniques. An in-depth theoretical analysis of the algorithm's convergence is presented in this article. Experiments are conducted on several typical databases for image classification, and the promising results indicate the effectiveness of RDS.

Graph-based adaptive and discriminative subspace learning for face image clustering

Current graph-based subspace clustering methods have achieved some results for the clustering of face images. The core of those methods lies in graph learning. However, they still have the following problems when learning the graph. Firstly, the graph learning processes of those methods do not consider the alignment of the images. It is well known that the obtained images of the same category may not be aligned due to different devices and shooting angles. The unaligned images used for graph learning directly affect the accuracy of the resulting graph. Hence, the graphs obtained by these methods are not accurate. We know that the inaccuracy of the learned graph will directly reduce the clustering performance of those methods. Secondly, they believe that important features, redundant features, and noise play the same contribution in the process of graph construction and feature representation. Redundant features and noise are not beneficial to graph reconstruction and feature representation, and even cause the learned graph to be inaccurate. Thirdly, the intrinsic structural correlation between samples is rarely considered for graph learning, which makes it difficult for the learned graph to reflect the structural correlation, then a good clustering performance cannot be obtained. To address those problems, this paper proposes a graph-based adaptive and discriminative subspace learning method (GADSL). In GADSL, image alignment is introduced and unified with subspace learning under the same graph learning framework which helps reduce the impact of different shooting equipment. Besides, GADSL can adaptively assign large weights to the important features and small weights to the unimportant features by introducing the weighting matrix. Moreover, in order to consider the correlation between samples, the structural consistency constraint is introduced into the subspace learning process so that the intra-class difference decreases and the inter-class difference increases. The experimental results show that GADSL used for face image clustering has achieved better clustering performance than many state-of-the-art methods.

SymNet: A Simple Symmetric Positive Definite Manifold Deep Learning Method for Image Set Classification.

By representing each image set as a nonsingular covariance matrix on the symmetric positive definite (SPD) manifold, visual classification with image sets has attracted much attention. Despite the success made so far, the issue of large within-class variability of representations still remains a key challenge. Recently, several SPD matrix learning methods have been proposed to assuage this problem by directly constructing an embedding mapping from the original SPD manifold to a lower dimensional one. The advantage of this type of approach is that it cannot only implement discriminative feature selection but also preserve the Riemannian geometrical structure of the original data manifold. Inspired by this fact, we propose a simple SPD manifold deep learning network (SymNet) for image set classification in this article. Specifically, we first design SPD matrix mapping layers to map the input SPD matrices into new ones with lower dimensionality. Then, rectifying layers are devised to activate the input matrices for the purpose of forming a valid SPD manifold, chiefly to inject nonlinearity for SPD matrix learning with two nonlinear functions. Afterward, we introduce pooling layers to further compress the input SPD matrices, and the log-map layer is finally exploited to embed the resulting SPD matrices into the tangent space via log-Euclidean Riemannian computing, such that the Euclidean learning applies. For SymNet, the (2-D)2principal component analysis (PCA) technique is utilized to learn the multistage connection weights without requiring complicated computations, thus making it be built and trained easier. On the tail of SymNet, the kernel discriminant analysis (KDA) algorithm is coupled with the output vectorized feature representations to perform discriminative subspace learning. Extensive experiments and comparisons with state-of-the-art methods on six typical visual classification tasks demonstrate the feasibility and validity of the proposed SymNet.

Multi-Branch Tensor Network Structure for Tensor-Train Discriminant Analysis.

Higher-order data with high dimensionality arise in a diverse set of application areas such as computer vision, video analytics and medical imaging. Tensors provide a natural tool for representing these types of data. Two major challenges that confound current tensor based supervised learning algorithms are storage complexity and computational efficiency. In this paper, we address these problems by introducing a multi-branch tensor network structure. The multi-branch structure is a general tensor decomposition that includes Tucker and tensor-train (TT) as special cases and takes advantage of the flexibility of the tensor network to provide a better balance between storage and computational complexity. We then introduce a supervised discriminative tensor-train subspace learning approach referred to as tensor-train discriminant analysis (TTDA), and its implementations using the multi-branch tensor network structure. Multi-branch implementations of TTDA are shown to achieve lower storage and computational complexity while providing improved classification performance with respect to both Tucker and TT based supervised learning methods.

Unsupervised Domain Adaptation Based on Discriminative Subspace Learning for Cross-Project Defect Prediction

Unsupervised Domain Adaptation Based on Discriminative Subspace Learning for Cross-Project Defect Prediction