Discriminant Manifold Learning Research Articles

Unsupervised Domain Adaptation through high-order tensor matching with discriminative manifold learning

Unsupervised Domain Adaptation (UDA) seeks to exploit the source domain knowledge to address similar but unsupervised target domain tasks. To perform UDA, most of the existing works typically learn domain-invariant representations or align the distributions across source and target domains through low-order statistical matching, which fail to explore more discriminative and high-order adaptation information. In this work, we proposed a kind of UDA, namely UDA through high-order tensor matching with discriminative manifold learning (UDA-HOTDML), by exploring the discriminative manifold structure jointly with the high-order tensor to more desirably match cross-domain alignment. In UDA-HOTDML, multiple aspects of domain knowledge are exploited to benefit more generalizable UDA by jointly modeling the discriminative structures of target domain, consistence as well as high-order tensor statistical characteristics. Finally, our evaluation results show that the proposed UDA-HOTDML method outperforms many related UDA methods.

Discriminative aging subspace learning for age estimation

Human age estimation from facial images has become an active research topic in computer vision field because of various real-world applications. Temporal property of facial aging display sequential patterns that lie on the low-dimensional aging manifold. In this paper, we propose hidden factor analysis (HFA) model-based discriminative manifold learning method for age estimation. The hidden factor analysis decomposes facial features into independent age factor and identity factor. Various age invariant face recognition systems in the literature utilize identity factor for face recognition; however, the age factor remains unutilized. The age component of the hidden factor analysis model depends on the subject’s age. Thus it carries significant age-related information. In this paper, we demonstrate that such aging patterns can be effectively extracted from the HFA-based discriminant subspace learning algorithm. Next, we have applied multiple regression methods on low-dimensional aging features learned from the HFA model. Effect of reduced dimensionality on the accuracy has been evaluated by extensive experiments and compared with the state-of-the-art methods. Effectiveness and robustness in terms of MAE and CS of the proposed framework are demonstrated using experimental analysis on a large-scale aging database MORPH II. The accuracy of our method is found superior to the current state-of-the-art methods.

Discriminant analysis based on reliability of local neighborhood

To obtain a compact and effective low-dimensional representation, recently, most existing discriminant manifold learning methods have integrated manifold learning into discriminant analysis (DA) for extracting the intrinsic structure of data. These methods learn two kinds of adjacency graphs, such as intrinsic graph and penalty graph, to characterize the similarity between samples from intraclass and the pseudo similarity of interclass. However, they treat every sample equally, which results in the following defects: (1) These methods cannot accurately characterize the marginal region among different classes only through penalty graphs. (2) They can not identify the noisy and outlier samples which reduce the robustness of these methods. To address these problems, we introduce an adaptive adjacency factor to perform the discriminative based reliability analysis for each sample. By integrating the adjacency factor into discriminant manifold learning methods, we propose a novel method for DA namely discriminant analysis based on reliability of local neighborhood (DA-RoLN). We mainly have three contributions in this paper: (1) By the introduction of adjacency factor, sample points can be divided into three parts: intraclass samples, marginal samples, and outliers. Therefore, DA-RoLN emphasizes the effect of valid samples and filters the influence of outliers. (2) We adaptively calculate the adjacency factor in low-dimensional space, thus, the margin between different classes in low-dimensional space is emphasized. (3) An iterative algorithm is developed to solve the objective function of DA-RoLN, and it is easy to solve with a low computational cost. Extensive experimental results show the effectiveness of DA-RoLN.

Unsupervised Domain Adaptation via Discriminative Manifold Propagation.

Unsupervised domain adaptation is effective in leveraging rich information from a labeled source domain to an unlabeled target domain. Though deep learning and adversarial strategy made a significant breakthrough in the adaptability of features, there are two issues to be further studied. First, hard-assigned pseudo labels on the target domain are arbitrary and error-prone, and direct application of them may destroy the intrinsic data structure. Second, batch-wise training of deep learning limits the characterization of the global structure. In this paper, a Riemannian manifold learning framework is proposed to achieve transferability and discriminability simultaneously. For the first issue, this framework establishes a probabilistic discriminant criterion on the target domain via soft labels. Based on pre-built prototypes, this criterion is extended to a global approximation scheme for the second issue. Manifold metric alignment is adopted to be compatible with the embedding space. The theoretical error bounds of different alignment metrics are derived for constructive guidance. The proposed method can be used to tackle a series of variants of domain adaptation problems, including both vanilla and partial settings. Extensive experiments have been conducted to investigate the method and a comparative study shows the superiority of the discriminative manifold learning framework.

L1‐norm based discriminant manifold learning for multi‐label image classification

Recently, L1-norm based robust discriminant feature extraction technique has been attracted much attention in dimensionality reduction. However, most existing approaches solve the column vectors of the optimal projection matrix one by one with a greedy strategy. Moreover, they are not suitable for solving the multi-label image classification. To solve these problems, the authors give a model named L1-norm based discriminant manifold learning in this study. An iterative non-greedy algorithm is proposed to solve the objective and the obtained optimal projection matrix necessarily best optimise the corresponding trace ratio objective function, which is the essential criterion function for general supervised dimensionality reduction. They also analyse the convergence of the authors’ proposed algorithm in detail. Extensive experiments on some databases illustrate the effectiveness of their proposed method.

Supervised discriminative manifold learning with subsidiary-view information for near infrared spectroscopic classification of crop seeds

This paper introduces a novel manifold learning method for near infrared spectroscopic classification dimensionality reduction and feature extraction. First, similar spectra from different categories reduced by traditional methods are mixed seriously which becomes an obstacle to build an effective model. We present a novel application to discovering the spectral manifold structure through manifold learning. Second, we propose a new supervised discriminative manifold learning method to expanded category overlapping for dimensionality reduction in classification and pattern recognition. The proposed method constructs a feature space which can make the samples of different categories in overlapping region far away from each other and keep the samples of same category in non-overlapping region close to each other. Accordingly, the low-dimensional feature space with expanding the overlap region is more conducive to classification. Moreover, the subsidiary-view version of the proposed method is proposed to further improve the classification accuracy. Finally, an authentic spectra dataset obtained from 200 maize seeds is introduced for building a haploid identification model to verify the method. Experimental results show that the proposed method outperforms several relevant manifold learning method and common dimensionality reduction methods of spectra. Furthermore, it is validated that subsidiary-view manifold learning is better than single view under appropriate parameters in near infrared spectral dimensionality reduction and classification.

Binarized features with discriminant manifold filters for robust single-sample face recognition

Many popular face recognition methods perform excellently when multiple samples per person are available for the training step of feature extraction. However, in many practical applications, there might be only a single sample recorded in systems for each person and most of traditional methods perform not very well because they usually need much more samples for discriminant learning. To address this problem, in this paper we propose a novel method for constructing local histogram based facial image descriptors, inspired by discriminant manifold learning and binary encoding. In manifold learning procedure, we divide each image into several nonoverlapping patches and try to find a matrix to project these patches onto an optimal subspace to maximize manifold margins of different persons. Then each column of the matrix is reshaped to an image filter to process facial images and the responses corresponding to these filters are binarized via thresholding. Thus for each pixel we can obtain a binary code whose length is just the number of filters. Finally, we compute region-wise histograms of pixels’ binary codes and concatenate them to form the representation of a facial image for recognition. We conduct some experiments on Extended Yale-B, AR, FERET and LFW databases to demonstrate the effectiveness of our proposed method in face recognition.

Discriminant Manifold Learning via Sparse Coding for Robust Feature Extraction

Most off-the-shelf subspace learning methods directly calculate the statistical characteristics of the original input images, while ignoring different contributions of different image components. In fact, to extract efficient features for image analysis, the noise or trivial structure in images should have little contribution and the intrinsic structure should be uncovered. Motivated by this observation, we propose a new subspace learning method, namely, discriminant manifold learning via sparse coding (DML_SC) for robust feature extraction. Specifically, we first decompose each input image into several components via dictionary learning, and then regroup the components into a more important part (MIP) and a less important part (LIP). The MIP can be considered as the clean portion of the image residing on a low-dimensional submanifold, while the LIP as noise or trivial structure within the image. Finally, the MIP and LIP are incorporated into manifold learning to learn a desired discriminative subspace. The proposed method is general for both cases with and without class labels, hence generating supervised DML_SC (SDML_SC) and unsupervised DML_SC (UDML_SC). Experimental results on four benchmark data sets demonstrate the efficacy of the proposed DML_SCs on both image recognition and clustering tasks.

Adaptive maximum margin analysis for image recognition

Most existing discriminant manifold learning methods aim to maximize the margin among nearby data, which is determined in the high-dimensional original space. As such, they do not necessarily best maximize the margin between different classes in the low-dimensional space, which is a critically important property for image classification. To handle this problem, we propose an adaptive maximum margin analysis (AMMA) for feature extraction. AMMA aims to seek a projection matrix that best maximize the margin, which is calculated in the low- dimensional space. It uses sparse representation to adaptively construct the intrinsic and penalty graphs. Finally, an iterative algorithm is developed to solve the projection matrix. Extensive experimental results on several image databases illustrate the effectiveness of the proposed approach.

Discriminative Manifold Learning Based Detection of Movement-Related Cortical Potentials.

The detection of voluntary motor intention from EEG has been applied to closed-loop brain-computer interfacing (BCI). The movement-related cortical potential (MRCP) is a low frequency component of the EEG signal, which represents movement intention, preparation, and execution. In this study, we aim at detecting MRCPs from single-trial EEG traces. For this purpose, we propose a detector based on a discriminant manifold learning method, called locality sensitive discriminant analysis (LSDA), and we test it in both online and offline experiments with executed and imagined movements. The online and offline experimental results demonstrated that the proposed LSDA approach for MRCP detection outperformed the Locality Preserving Projection (LPP) approach, which was previously shown to be the most accurate algorithm so far tested for MRCP detection. For example, in the online tests, the performance of LSDA was superior than LPP in terms of a significant reduction in false positives (FP) (passive FP: 1.6 ±0.9/min versus 2.9 ±1.0/min, p = 0.002, active FP: 2.2 ±0.8/min versus 2.7 ±0.6/min , p = 0.03 ), for a similar rate of true positives. In conclusion, the proposed LSDA based MRCP detection method is superior to previous approaches and is promising for developing patient-driven BCI systems for motor function rehabilitation as well as for neuroscience research.

Hierarchical discriminant manifold learning for dimensionality reduction and image classification

In the field of image classification, it has been a trend that in order to deliver a reliable classification performance, the feature extraction model becomes increasingly more complicated, leading to a high dimensionality of image representations. This, in turn, demands greater computation resources for image classification. Thus, it is desirable to apply dimensionality reduction (DR) methods for image classification. It is necessary to apply DR methods to relieve the computational burden as well as to improve the classification accuracy. However, traditional DR methods are not compatible with modern feature extraction methods. A framework that combines manifold learning based DR and feature extraction in a deeper way for image classification is proposed. A multiscale cell representation is extracted from the spatial pyramid to satisfy the locality constraints for a manifold learning method. A spectral weighted mean filtering is proposed to eliminate noise in the feature space. A hierarchical discriminant manifold learning is proposed which incorporates both category label and image scale information to guide the DR process. Finally, the image representation is generated by concatenating dimensionality reduced cell representations from the same image. Extensive experiments are conducted to test the proposed algorithm on both scene and object recognition datasets in comparison with several well-established and state-of-the-art methods with respect to classification precision and computational time. The results verify the effectiveness of incorporating manifold learning in the feature extraction procedure and imply that the multiscale cell representations may be distributed on a manifold.

Constrained discriminant neighborhood embedding for high dimensional data feature extraction

When handling pattern classification problem such as face recognition and digital handwriting identification, image data is always represented to high dimensional vectors, from which discriminant features are extracted using dimensionality reduction methods. So in this paper, we present a supervised manifold learning based dimensionality reduction method named constrained discirminant neighborhood embedding (CDNE). In the proposed CDNE, on one hand, the class information of samples is taken into account to construct both an inter-class graph and an intra-class graph, where neighborhood points in the intra-class graph are selected from those with the same class and any point in the inter-class graph should sample those labeled different classes as its neighborhood points. On the other hand, locally least linear reconstruction technique is also introduced to model an objective function under the local uncorrelation constraint to explore a discriminant subspace. Compared to some related and state-of-the-art dimensionality reduction methods such as discriminant neighborhood embedding (DNE), supervised locality discriminant manifold learning (SLDML), discriminant sparse neighborhood preserving embedding (DSNPE), local graph embedding based on maximum margin criterion (LGE/MMC), uncorrelated discriminant locality preserving projection (UDLPP) and locally uncorrelated discriminant projection (LUDP), the proposed CDNE has been validated to be efficient and feasible by experimental results on some benchmark face data sets including CMU PIE, ORL and FERET.

Discriminative sparsity preserving projections for image recognition

Previous works have demonstrated that image classification performance can be significantly improved by manifold learning. However, performance of manifold learning heavily depends on the manual selection of parameters, resulting in bad adaptability in real-world applications. In this paper, we propose a new dimensionality reduction method called discriminative sparsity preserving projections (DSPP). Different from the existing sparse subspace algorithms, which manually construct a penalty adjacency graph, DSPP employs sparse representation model to adaptively build both intrinsic adjacency graph and penalty graph with weight matrix, and then integrates global within-class structure into the discriminant manifold learning objective function for dimensionality reduction. Extensive experimental results on four image databases demonstrate the effectiveness of the proposed approach.

Locally linear representation Fisher criterion based tumor gene expressive data classification

Tumor gene expressive data are characterized by a large amount of genes with only a small amount of observations, which always appear with high dimensionality. So it is necessary to reduce the dimensionality before identifying their genre. In this paper, a discriminant manifold learning method, named locally linear representation Fisher criterion (LLRFC), is applied to extract features from tumor gene expressive data. In LLRFC, an inter-class graph and an intra-class graph are constructed based on their genre information, where any tumor gene expressive data in the inter-class graph should select k nearest neighbors with different class labels and in the intra-class graph the k nearest neighbors for any tumor gene expressive data must be sampled from those with the same class. And then the locally least linear reconstruction is introduced to optimize the corresponding weights in both graphs. Moreover, a Fisher criterion is modeled to explore a low dimensional subspace where the reconstruction errors in the inter-class graph can be maximized and the reconstruction errors in the intra-class graph can be minimized, simultaneously. Experiments on some benchmark tumor gene expressive data have been conducted with some related algorithms, by which the proposed LLRFC has been validated to be efficient.

Supervised locality discriminant manifold learning for head pose estimation

In this paper, we propose a novel supervised manifold learning approach, supervised locality discriminant manifold learning (SLDML), for head pose estimation. Traditional manifold learning methods focus on preserving only the intra-class geometric properties of the manifold embedded in the high-dimensional ambient space, so they cannot fully utilize the underlying discriminative knowledge of the data. The proposed SLDML aims to explore both geometric structure and discriminant information of the data, and yields a smooth and discriminative low-dimensional embedding by adding the local discriminant terms in the optimization objectives of manifold learning. Moreover, for efficiently handling out-of-sample extension and learning with the local consistency, we decompose the manifold learning as a two-step approach. We incorporate the manifold learning and the regression with a learned discriminant manifold-based projection function obtained by discriminatively Laplacian regularized least squares. The SLDML provides both the low-dimensional embedding and projection function with better intra-class compactness and inter-class separability, therefore preserves the local geometric structures more effectively. Meanwhile, the SLDML is supervised by both biased distance and continuous head pose angle information when constructing the graph, embedding the graph and learning the projection function. Our experiments demonstrate the superiority of the proposed SLDML over several current state-of-art approaches for head pose estimation on the publicly available FacePix dataset.

A Family of Discriminative Manifold Learning Algorithms and Their Application to Speech Recognition

This paper presents a family of discriminative manifold learning approaches to feature space dimensionality reduction in noise robust automatic speech recognition (ASR). The specific goal of these techniques is to preserve local manifold structure in feature space while at the same time maximizing the separability between classes of feature vectors. In the manifold space, the relationships among the feature vectors are defined using nonlinear kernels. Two separate distance measures are used to characterize the kernels, namely the conventional Euclidean distance and a cosine-correlation based distance. The performance of the proposed techniques is evaluated on two task domains involving noise corrupted utterances of connected digits and read newspaper text. Performance is compared to existing approaches used for feature space transformations, including linear discriminant analysis (LDA) and locality preserving linear projections (LPP). The proposed approaches are found to provide a significant reduction in word error rate (WER) with respect to the more well-known techniques for a variety of noise conditions. Another contribution of the paper is to quantify the interaction between acoustic noise conditions and the shape and size of local neighborhoods which are used in manifold learning to define local relationships among feature vectors. Based on this analysis, a procedure for reducing the impact of varying acoustic conditions on manifold learning is proposed .

高光谱图像降维的判别流形学习方法

A discriminant manifold learning approach for hyperspectral image dimension reduction was proposed.In order to overcome the high dimensional and high redundancy of remotely sensed earth observation images,a modified manifold learning algorithm was suggested for dataset linear dimensional reduction to improve the performance of image classification.The proposed method addressed the discriminative information of given training samples into the current manifold learning framework to learn an optimal subspace for subsequent classification,in particular,the linearization of discriminant manifold learning is introduced to deal with the out of sample problem.Experiments on hyperspectral image demonstrated that the proposed method could achieve higher classification rate than the conventional image classification technologies.

Face feature extraction and recognition based on discriminant subclass-center manifold preserving projection

Manifold learning is an effective dimensional reduction technique for face feature extraction, which, generally speaking, tends to preserve the local neighborhood structures of given samples. However, neighbors of a sample often comprise more inter-class data than intra-class data, which is an undesirable effect for classification. In this paper, we address this problem by proposing a subclass-center based manifold preserving projection (SMPP) approach, which aims at preserving the local neighborhood structure of subclass-centers instead of given samples. We theoretically show from a probability perspective that, neighbors of a subclass-center would comprise of more intra-class data than inter-class data, and thus is more desirable for classification. In order to take full advantage of the class separability, we further propose the discriminant SMPP (DSMPP) approach, which incorporates the subclass discriminant analysis (SDA) technique to SMPP. In contrast to related discriminant manifold learning methods, DSMPP is formulated as a dual-objective optimization problem and we present analytical solution to it. Experimental results on the public AR, FERET and CAS-PEAL face databases demonstrate that the proposed approaches are more effective than related manifold learning and discriminant manifold learning methods in classification performance.