Semi-supervised Linear Discriminant Analysis Research Articles

Semi-supervised LDA pedestrian re-identification algorithm based on K-nearest neighbor resampling

Person re-identification identify a specific person in surveillance network by similarity measurement between images of different camera views. However, existing metric learning based methods suffer from over-fitting problem. To solve this problem, a resampled linear discriminant analysis (LDA) method was proposed based on the statistical and topological characteristics of pedestrian images. This method utilized the k-nearest neighbours to form potential positive sample pairs. The potential positive pairs are used to improve the metric model and generalize the metric model to the test data. By minimizing the inter-class divergence of potential positive sample pairs, a semi-supervised re-sampling LDA person re-identification algorithm was established. It was then tested on the VIPeR, CUHK01 and Market 1501datasets. The results show that the proposed method achieves the best performance compared to some available methods. Especially, the proposed method outplays the best comparison method by 0.6% and 5.76% at rank-1 identification rate on the VIPeR and CUHK01 datasets respectively. At the same time, the improved LDA algorithm has improved the rank-1 identification accuracy of traditional LDA method by 9.36% and 32.11% on these two datasets respectively. Besides, the proposed method is limited to Market-1501 dataset when the test data is of large size.

Semi-supervised LDA and Multi-distance Metric Learning for Person Re-identification

The problem of person re-identification has attracted a lot of attention in the field of machine vision. In practice, the non-overlapping sample images change drastically and the sample size is small, which makes the metric model overfitting phenomenon. In this paper, based on the k-NN and the sample normality property, we propose a resampling linear discriminant analysis (LDA) algorithm to suppress the local constraints caused by small samples, then train it to obtain the person re-identification metric learning model. A semi-supervised LDA algorithm with semi-supervised characteristics is developed by optimizing the inter-class scatter for weighting. A joint distance metric-based approach is also proposed to learn both the Mahalanobis distance and Euclidean distance. The improved algorithm is tested on the VIPeR and CUHK01 datasets, and the results indicate that, despite the change in the total number of training samples, the algorithm in this paper shows high recognition accuracy.

Semisupervised Linear Discriminant Analysis Based on Pairwise Constraint Propagation for Hyperspectral Images

As a classical supervised dimensionality reduction (DR) method, linear discriminant analysis (LDA) has been developed for many variants. However, it is not applicable to the case that labeled samples are scarce and unlabeled samples are in large quantity, which always happens in the real world. In this letter, we propose a novel technique format termed semisupervised LDA based on pairwise constraint propagation (SLDA-PCP) for hyperspectral images (HSIs). The basic idea of this method is to use a specially designed PCP technique to propagate label information from the labeled samples to the unlabeled samples. In addition, an extended LDA format to learn the optimal projection vectors according to the newly obtained label information is also created. Comprehensive experiments on two HSIs show that our SLDA-PCP performs better than some state-of-the-art semisupervised DR methods.

Semi-supervised linear discriminant analysis for dimension reduction and classification

When facing high dimensional data, dimension reduction is necessary before classification. Among dimension reduction methods, linear discriminant analysis (LDA) is a popular one that has been widely used. LDA aims to maximize the ratio of the between-class scatter and total data scatter in projected space, and the label of each data is necessary. However, in real applications, the labeled data are scarce and unlabeled data are in large quantity, so LDA is hard to be used under such case. In this paper, we propose a novel method named semi-supervised linear discriminant analysis (SLDA), which can use limited number of labeled data and a quantity of the unlabeled ones for training so that LDA can accommodate to the situation of a few labeled data available. Assuming that F represents the calculated class indicator matrix of the training data and Y denotes the true label of the labeled data, the objective function contains two parts: one is the criterion of LDA (which is a function of projection W, and a class indicator matrix F), the other is the difference between the true data label and calculated label of these labeled data. As far as we know, there is no closed-form solution to the objective function. To solve such problem, we develop an iterative algorithm which calculates the class indicator matrix and the projection alternatively. The convergence of the proposed iterative algorithm is proved and confirmed by experiments. The experimental results on eight datasets show that the performance of SLDA is superior to that of traditional LDA and some state-of-the-art semi-supervised algorithms.

A novel discriminant criterion based on feature fusion strategy for face recognition

Feature extraction is an important problem in face recognition. There are two kinds of structural features, namely the Euclidean structure and the manifold structure. However, the single-structural feature extraction methods cannot fully utilize the advantages of global feature and local feature simultaneously. Thus their performances will be degraded. To overcome the limitations of the single-structural feature based face recognition schemes, this paper proposes a novel discriminant criterion using Feature Fusion Strategy (FFS), which nonlinearly combines both Euclidean and manifold structures in the face pattern space. The proposed discriminant criterion is suitable to develop an iterative algorithm. It is able to automatically determine the optimal parameters and balance the tradeoff between Euclidean structure and manifold structure. The proposed FFS algorithm is successfully applied to face recognition. Three publicly available face databases, ORL, FERET and CMU PIE, are selected for evaluation. Compared with Linear Discriminant Analysis (LDA), Locality Preserving Projection (LPP), Unsupervised Discriminant Projection (UDP) and Semi-Supervised LDA (SSLDA), the experimental results show that the proposed method gives superior performance.

A jointly distributed semi-supervised topic model

Latent topic models are applied to analyze the low-dimensional semantic meaning of documents and images, which are widely used in object categorization. However, the unsupervised topic model cannot guarantee that the learned topics have a good relation with class labels, while manually aligning and labeling all training images are expensive and subjective in real applications. Aiming at using a small amount of partial labels to find topics much more suitable for classification, joint distribution from multi-conditional learning is adopted in this paper to generate semi-supervised topic models. Semi-supervised LDA and pLSA models are proposed when the joint distribution is known or partially known. Experimental results on natural scene categorization and head pose classification tasks show that the proposed method remains promising using only partial labels in the training process, which demonstrates the effectiveness of the proposed method.

Self-training-based face recognition using semi-supervised linear discriminant analysis and affinity propagation

Face recognition is one of the most important applications of machine learning and computer vision. The traditional supervised learning methods require a large amount of labeled face images to achieve good performance. In practice, however, labeled images are usually scarce while unlabeled ones may be abundant. In this paper, we introduce a semi-supervised face recognition method, in which semi-supervised linear discriminant analysis (SDA) and affinity propagation (AP) are integrated into a self-training framework. In particular, SDA is employed to compute the face subspace using both labeled and unlabeled images, and AP is used to identify the exemplars of different face classes in the subspace. The unlabeled data can then be classified according to the exemplars and the newly labeled data with the highest confidence are added to the labeled data, and the whole procedure iterates until convergence. A series of experiments on four face datasets are carried out to evaluate the performance of our algorithm. Experimental results illustrate that our algorithm outperforms the other unsupervised, semi-supervised, and supervised methods.

Semi-supervised linear discriminant analysis through moment-constraint parameter estimation

A semi-supervised version of classical linear discriminant analysis is presented. As opposed to most current approaches to semi-supervised learning, no additional extrinsic assumptions are made to tie information coming from labeled and unlabeled data together. Our approach exploits the fact that the parameters that are to be estimated fulfill particular relations, intrinsic to the classifier, that link label-dependent with label-independent quantities. In this way, the latter type of parameters, which can be estimated based on unlabeled data, impose constraints on the former and lead to a reduction in variability of the label dependent estimates. As a result, the performance of our semi-supervised linear discriminant is typically expected to improve over that of its regular supervised match. Possibly more important, our semi-supervised linear discriminant analysis does not show the severe deteriorations other approaches frequently display with increasing numbers of unlabeled data. This work recapitulates, corrects, extends, and revises our previous work that has been published as part of the First IAPR TC3 Workshop on Partially Supervised Learning. The main novelty it provides over our earlier work is an affine invariant approach to semi-supervised learning befitting linear discriminant analysis. Besides, more elaborate and convincing experimental evidence of the potential of our general approach is provided. We essentially believe that the general principle of intrinsic constraints is of interest as such and may inspire other novel semi-supervised methods.