Unsupervised Discriminant Projection Research Articles

Change Detection in Landsat Images Using Unsupervised Learning and RBF-Based Clustering

Change detection is one of the extensively investigated problems in the field of remote sensing. Generally, it involves the analysis of multitemporal remote sensing images of the same sensor or different sensors. The nonnormalizable radiometric differences, which are added due to environmental interferences, could lead to increase the intra-class variability in multitemporal images, and hence, the change detection problem would be nonlinear. To cope with these issues, a novel unsupervised manifold learning based change detection technique is proposed, which discovers the nonlinear structure of data hidden in original space. The proposed framework consists of two processing stages. In the first stage, a novel orthogonal unsupervised discriminant projection (OUDP) based technique is proposed for feature extraction. The orthogonal basis vectors, obtained from OUDP, are able to provide discriminant features. In addition, in the second stage, a novel radial-basis function based clustering is presented, which yields the better clustering than the single clustering approach. Moreover, an extended kernel-weighted OUDP technique is also presented, which improves the performance of OUDP technique. Particularity of the proposed framework resides on following three things. First one is the creation of eigenvector space by using the nonoverlapping blocks of the image. Second one is the extraction of features by exploiting the local neighborhood information around each pixel, this generates clear or spot free change maps. The last one is the RBF stage of clustering technique, which greatly improves the classification of the unchanged and changed pixels. Experimental results on multispectral images of different sensors validate the effectiveness of the proposed method.

Sparsity Regularization Discriminant Projection for Feature Extraction

Recently, sparse representation models have attracted considerable interests in the field of feature extraction. In this paper, we propose a novel supervised feature extraction method called sparsity regularization discriminant projection (SRDP), which aims to preserve the sparse representation structure of the data and simultaneously maximize the ratio of nonlocal scatter to local scatter. More specifically, SRDP first constructs a concatenated dictionary through the class-wise principal component analysis decompositions. Second, the sparse representation structure of each sample is quickly learned with the constructed dictionary by matrix–vector multiplications. Then SRDP regards the learned sparse representation structure as an additional regularization term of unsupervised discriminant projection so as to construct a new discriminant function. Finally, SRDP is transformed into a generalized eigenvalue problem. Experimental results on five representative image databases demonstrate the effectiveness of our proposed method.

BULDP: Biomimetic Uncorrelated Locality Discriminant Projection for Feature Extraction in Face Recognition.

This paper develops a new dimensionality reduction method, named Biomimetic Uncorrelated Locality Discriminant Projection (BULDP), for face recognition. It is based on unsupervised discriminant projection and two human bionic characteristics: principle of homology continuity and principle of heterogeneous similarity. With these two human bionic characteristics, we propose a novel adjacency coefficient representation, which does not only capture the category information between different samples, but also reflects the continuity between similar samples and the similarity between different samples. By applying this new adjacency coefficient into the unsupervised discriminant projection, it can be shown that we can transform the original data space into an uncorrelated discriminant subspace. A detailed solution of the proposed BULDP is given based on singular value decomposition. Moreover, we also develop a nonlinear version of our BULDP using kernel functions for nonlinear dimensionality reduction. The performance of the proposed algorithms is evaluated and compared with the state-of-the-art methods on four public benchmarks for face recognition. Experimental results show that the proposed BULDP method and its nonlinear version achieve much competitive recognition performance.

Modified unsupervised discriminant projection with an electronic nose for the rapid determination of Chinese mitten crab freshness

In this paper, a method to rapidly determine living Chinese mitten crab freshness using an electronic nose (E-nose) and a non-linear data processing technique was studied.

A multilinear unsupervised discriminant projections method for feature extraction

Despite considering the distribution information of data, unsupervised discriminant projection (UDP) ignores the space structure information of data for high order tensor objects. To address these problems, many tensor methods are developed for charactering the space structure information. Albeit effective, these methods ignore the local manifold structure of the samples, and thus achieve sub-optimal performance. In this paper, we formulate UDP in a high order tensor space and develop a Multilinear UDP (MUDP) for feature extraction on tensor objects. MUDP inherits the merits of UDP and Tensor based methods. The experiments tell that MUDP is an efficient and effective method and works well.

Category-specific object segmentation via unsupervised discriminant shape

Category-specific object segmentation has been a long-standing research topic in pattern recognition. This paper presents an unsupervised discriminant shape (UDS) to address category-specific object segmentation by incorporating the proposed shape prior into an intuitive energy minimization framework. Recently, based on the region proposal methods, deep Convolutional Neural Networks (CNNs) provide access to candidate segments in categories of interest from images. However, the segments obtained from bottom-up proposals tend to undershoot or overshoot objects and are easily classified into one specific class. To address this problem, we propose an unsupervised discriminant projection based clustering algorithm (UDC) to obtain more precise shape prior to guide the segmentation, and the class-specific proposals are clustered based on their projections onto the discriminant projection direction. Based on the set of proposals, we then obtain the prior information of foreground UDS with an easy voting scheme. The derived UDS prior is finally utilized in the subsequent energy minimizing formulation based figure-ground segmentation. We conduct extensive and comprehensive evaluations on the MSRC, Object Discovery, Fashionista and PASCAL-S datasets, demonstrating the effectiveness and robustness of the UDS based segmentation.

Multi-class Fukunaga Koontz discriminant analysis for enhanced face recognition

Linear subspace learning methods such as Fisher׳s Linear Discriminant Analysis (LDA), Unsupervised Discriminant Projection (UDP), and Locality Preserving Projections (LPP) have been widely used in face recognition applications as a tool to capture low dimensional discriminant information. However, when these methods are applied in the context of face recognition, they often encounter the small-sample-size problem. In order to overcome this problem, a separate Principal Component Analysis (PCA) step is usually adopted to reduce the dimensionality of the data. However, such a step may discard dimensions that contain important discriminative information that can aid classification performance. In this work, we propose a new idea which we named Multi-class Fukunaga Koontz Discriminant Analysis (FKDA) by incorporating the Fukunaga Koontz Transform within the optimization for maximizing class separation criteria in LDA, UDP, and LPP. In contrast to traditional LDA, UDP, and LPP, our approach can work with very high dimensional data as input, without requiring a separate dimensionality reduction step to make the scatter matrices full rank. In addition, the FKDA formulation seeks optimal projection direction vectors that are orthogonal which the existing methods cannot guarantee, and it has the capability of finding the exact solutions to the “trace ratio” objective in discriminant analysis problems while traditional methods can only deal with a relaxed and inexact “ratio trace” objective. We have shown using six face database, in the context of large scale unconstrained face recognition, face recognition with occlusions, and illumination invariant face recognition, under “closed set”, “semi-open set”, and “open set” recognition scenarios, that our proposed FKDA significantly outperforms traditional linear discriminant subspace learning methods as well as five other competing algorithms.

Fuzzy local discriminant embedding for image feature extraction

In pattern recognition, feature extraction techniques have been widely employed to reduce the high dimensionality of data. In this paper, we propose a novel algorithm called fuzzy local discriminant embedding (FLDE) based on the unsupervised discriminant projection criterion and fuzzy set theory for image feature extraction and recognition. In the proposed method, a membership degree matrix is firstly calculated using the fuzzy k-nearest neighbor (FKNN) algorithm, and then the membership degree and the label information are incorporated into the definition of the weighted matrices to get the fuzzy local scatter and fuzzy nonlocal scatter. After characterizing the fuzzy nonlocal scatter and the fuzzy local scatter, a concise feature extraction criterion is derived via maximizing the ratio between them. Experimental results on the ORL, FERET, and CMU PIE face databases show the effectiveness of the proposed method.

A novel discriminant criterion based on feature fusion strategy for face recognition

Feature extraction is an important problem in face recognition. There are two kinds of structural features, namely the Euclidean structure and the manifold structure. However, the single-structural feature extraction methods cannot fully utilize the advantages of global feature and local feature simultaneously. Thus their performances will be degraded. To overcome the limitations of the single-structural feature based face recognition schemes, this paper proposes a novel discriminant criterion using Feature Fusion Strategy (FFS), which nonlinearly combines both Euclidean and manifold structures in the face pattern space. The proposed discriminant criterion is suitable to develop an iterative algorithm. It is able to automatically determine the optimal parameters and balance the tradeoff between Euclidean structure and manifold structure. The proposed FFS algorithm is successfully applied to face recognition. Three publicly available face databases, ORL, FERET and CMU PIE, are selected for evaluation. Compared with Linear Discriminant Analysis (LDA), Locality Preserving Projection (LPP), Unsupervised Discriminant Projection (UDP) and Semi-Supervised LDA (SSLDA), the experimental results show that the proposed method gives superior performance.

Optimal Uncorrelated Unsupervised Discriminant Projection

Unsupervised Discriminant Projection (UDP) is a typical manifold-based dimensionality reduction method, and has been successfully applied in face recognition. However, UDP suffers from the small sample size problem and usually deteriorates because the basis vectors of UDP are statistically correlated. In order to resolve these problems, we propose an Optimal Uncorrelated Unsupervised Discriminant Projection (OUUDP).The aim of OUUDP is to seek a feature submanifold such that the local scatter is minimized and non-local scatter scatter is maximized simultaneously in the embedding space by using a difference-based optimization objective function. Moreover, we impose an appropriate constraint to make the extracted features statistically uncorrelated. As a result, OUUDP can solve the small sample size problem and exploit statistically uncorrelated features. Experimental results on ORL databases demonstrate the effectiveness of the proposed algorithm.

Nonparametric discriminant multi-manifold learning for dimensionality reduction

Based on that data sampled from the same class locate on one manifold and those labeled different classes reside on the corresponding manifolds, traditional data classification problem can be reasoned to multiply manifolds identification. Thus in this paper, a dimensionality reduction method titled nonparametric discirminant multi-manifold learning (NDML) is put forward and involved in different manifolds recognition. In the proposed method, a novel nonparametric manifold-to-manifold distance is defined to characterize the separability between manifolds. And then an objective function is modeled to project the original data into a low dimensional space, where the manifold-to-manifold distances can be maximized and manifolds locality will be preserved. Experiments have been carried out on benchmark face data sets with comparisons to some related dimensionality reduction methods such as Unsupervised Discriminant Projection (UDP), Constrained Maximum Variance Mapping (CMVM) and Linear Discriminant Analysis (LDA). The experimental results validate that the proposed NDML can obtain better performance than other methods.

Subspace-based discrete transform encoded local binary patterns representations for robust periocular matching on NIST's face recognition grand challenge.

In this paper, we employ several subspace representations (principal component analysis, unsupervised discriminant projection, kernel class-dependence feature analysis, and kernel discriminant analysis) on our proposd discrete transform encoded local binary patterns (DT-LBP) to match periocular region on a large data set such as NIST's face recognition grand challenge (FRGC) ver2 database. We strictly follow FRGC Experiment 4 protocol, which involves 1-to-1 matching of 8014 uncontrolled probe periocular images to 16 028 controlled target periocular images (~128 million pairwise face match comparisons). The performance of the periocular region is compared with that of full face with different illumination preprocessing schemes. The verification results on periocular region show that subspace representation on DT-LBP outperforms LBP significantly and gains a giant leap from traditional subspace representation on raw pixel intensity. Additionally, our proposed approach using only the periocular region is almost as good as full face with only 2.5% reduction in verification rate at 0.1% false accept rate, yet we gain tolerance to expression, occlusion, and capability of matching partial faces in crowds. In addition, we have compared the best standalone DT-LBP descriptor with eight other state-of-the-art descriptors for facial recognition and achieved the best performance. The two general frameworks are our major contribution: 1) a general framework that employs various generative and discriminative subspace modeling techniques for DT-LBP representation and 2) a general framework that encodes discrete transforms with local binary patterns for the creation of robust descriptors.

Two-Dimensional Orthogonal Unsupervised Discriminant Projection

Unsupervised Discriminant Projection (UDP) is one of the most promising feature extraction methods. However, UDP suffers from the small sample size problem and the optimal basis vectors obtained by the UDP are nonorthogonal. In this paper, we present a new method called Two-dimensional Orthogonal Unsupervised Discriminant Projection (2DOUDP), which is not necessary to convert the image matrix into high-dimensional image vector and does not suffer the small sample size problem. To further improve the recognition performance, the orthogonal projection matrix obtained based on Gram–Schmidt orthogonalization is given. Experimental results on ORL database indicate that the proposed 2DOUDP method achieves better recognition rate than the UDP method and some other orthogonal feature extraction algorithms.

Modified orthogonal discriminant projection for classification

From the perspective of manifold learning, the weight between two nodes of graph plays an indispensable role, which provides the similarity between pairwise nodes, and can effectively reveal the intrinsic relationship between data classes. In the original Locality Preserving Projections (LPP), Unsupervised Discriminant Projection (UDP), Orthogonal LPP (OLPP), and other spectral mapping methods, the weight between two points is usually defined as a heat kernel or simply 0–1 weight, which cannot effectively reflect the sample class information. In Orthogonal Discriminant Projection (ODP), the weight between two points was defined based on their local information and class information, but it is not a monotonically decreasing with the increase of the distance between two nodes, so it is not very sound. In this paper, we first analyze the defect of the weight in ODP, then propose a novel weight measure between two nodes of a graph by combining their label information and local information, finally present a modified ODP algorithm following the ODP technique. The modified ODP algorithm can explore the intrinsic structure of original data and enhance the classification ability. The experimental results show that the modified ODP algorithm is effective and feasible.

Supervised Discriminant Projection with Its Application to Face Recognition

In the past few decades, many face recognition methods have been developed. Among these methods, subspace analysis is an effective approach for face recognition. Unsupervised discriminant projection (UDP) finds an embedding subspace that preserves local structure information, and uncovers and separates embedding corresponding to different manifolds. Though UDP has been applied in many fields, it has limits to solve the classification tasks, such as the ignorance of the class information. Thus, a novel subspace method, called supervised discriminant projection (SDP), is proposed for face recognition in this paper. In our method, the class information was utilized in the procedure of feature extraction. In SDP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and class information. We test the performance of the proposed method SDP on three popular face image databases (i.e. AR database, Yale database, and a subset of FERET database). Experimental results show that the proposed method is effective.

IMPROVED ALGORITHMS FOR UNSUPERVISED DISCRIMINANT PROJECTION

Dimension reduction has been applied in various areas of pattern recognition and data mining. While a traditional dimension reduction method, Principal Component Analysis (PCA) finds projective directions to maximize the global scatter in data, Locality Preserving Projection (LPP) pursues linear dimension reduction to minimize the local scatter. However, the discriminative power by either global or local scatter optimization is not guaranteed to be effective for classification. A recently proposed method, Unsupervised Discriminant Projection (UDP) aims to minimize the local scatter among near points and maximize the global scatter of distant points at the same time. Although its performance has been proven to be comparable to other dimension reduction methods, PCA preprocessing step due to the singularity of global and local scatter matrices may degrade the performance of UDP. In this paper, we propose several algorithms to improve the performances of UDP greatly. An improved algorithm for UDP is presented which applies the Generalized Singular Value Decomposition (GSVD) to overcome singularities of scatter matrices in UDP. Two-dimensional UDP and nonlinear extension of UDP are also proposed. Extensive experimental results demonstrate superiority of the proposed algorithms.

Supervised feature extraction based on orthogonal discriminant projection

In this paper, a supervised feature extraction method, named orthogonal discriminant projection (ODP), is presented. As an extension of spectral mapping method, the proposed algorithm maximizes the weighted difference between the non-local scatter and the local scatter. Moreover, the weights between two nodes of a graph are adjusted according to their class information and local information. Experiments on FERET face data, Yale face data and MNIST handwriting digits data validate that ODP can offer better recognition rate than some other feature extraction methods, such as local preserving projection (LPP), unsupervised discriminant projection (UDP) and orthogonal LPP (OLPP).

Comments on "Globally Maximizing, Locally Minimizing: Unsupervised Discriminant Projection with Application to Face and Palm Biometrics"

In [1], UDP is proposed to address the limitation of LPP for the clustering and classification tasks. In this communication, we show that the basic ideas of UDP and LPP are identical. In particular, UDP is just a simplified version of LPP on the assumption that the local density is uniform.

Feature extraction using constrained maximum variance mapping

In this paper, an efficient feature extraction method named as constrained maximum variance mapping (CMVM) is developed. The proposed algorithm can be viewed as a linear approximation of multi-manifolds learning based approach, which takes the local geometry and manifold labels into account. The CMVM and the original manifold learning based approaches have a point in common that the locality is preserved. Moreover, the CMVM is globally maximizing the distances between different manifolds. After the local scatters have been characterized, the proposed method focuses on developing a linear transformation that can maximize the dissimilarities between all the manifolds under the constraint of locality preserving. Compared to most of the up-to-date manifold learning based methods, this trick makes contribution to pattern classification from two aspects. On the one hand, the local structure in each manifold is still kept; on the other hand, the discriminant information between manifolds can be explored. Finally, FERET face database, CMU PIE face database and USPS handwriting data are all taken to examine the effectiveness and efficiency of the proposed method. Experimental results validate that the proposed approach is superior to other feature extraction methods, such as linear discriminant analysis (LDA), locality preserving projection (LPP), unsupervised discriminant projection (UDP) and maximum variance projection (MVP).

A THEORETICAL FRAMEWORK FOR MATRIX-BASED FEATURE EXTRACTION ALGORITHMS WITH ITS APPLICATION TO IMAGE RECOGNITION

Recently proposed matrix-based methods, two-dimensional Principal Component Analysis (2DPCA), two-dimensional Linear Discriminant Analysis (2DLDA) and two-dimensional Locality Preserving Projections (2DLPP) have been shown to be effective ways to avoid the problems of high dimensionality and small sample sizes that are associated with vector-based methods. In this paper, we propose a general theoretical framework for matrix-based feature extraction algorithms from the point of view of graph embedding. Our framework can be applied to extend two recently proposed vector-based algorithms, i.e. Unsupervised Discriminant Projection (UDP) and Marginal Fisher Analysis (MFA) algorithms, to their matrix-based versions. Further, our framework can also be used as a platform to generate new matrix-based feature extraction algorithms by designing meaningful graphs, e.g. two-dimensional Discriminant Embedding Analysis (2DDEA) in this paper. It is shown that 2DLDA is actually a special case of the 2DDEA method. Experiments on three publicly available image databases demonstrate the effectiveness of the proposed algorithm. Our results fit into the scene for a better picture about the matrix-based feature extraction algorithms.