Linear Dimensionality Reduction Methods Research Articles

Discriminant subspace learning constrained by locally statistical uncorrelation for face recognition

High-dimensionality of data and the small sample size problem are two significant limitations for applying subspace methods which are favored by face recognition. In this paper, a new linear dimension reduction method called locally uncorrelated discriminant projections (LUDP) is proposed, which addresses the two problems from a new aspect. More specifically, we propose a locally uncorrelated criterion, which aims to decorrelate learned discriminant factors over data locally rather than globally. It has been shown that the statistical uncorrelation criterion is an important property for reducing dimension and learning robust discriminant projection as well. However, data are always locally distributed, so it is more important to explore locally statistical uncorrelated discriminant information over data. We impose this new constraint into a graph-based maximum margin analysis, so that LUDP also characterizes the local scatter as well as nonlocal scatter, seeking to find a projection that maximizes the difference, rather than the ratio between the nonlocal scatter and the local scatter. Experiments on ORL, Yale, Extended Yale face database B and FERET face database demonstrate the effectiveness of our proposed method.

A Supervised Linear Dimensionality Reduction Method for Face Recognition

A Supervised Linear Dimensionality Reduction Method for Face Recognition

Sparse local discriminant projections for discriminant knowledge extraction and classification

One of the major disadvantages of the linear dimensionality reduction algorithms, such as principle component analysis (PCA) and linear discriminant analysis (LDA), is that the projections are lack of physical interpretation. Moreover, which features or variables play an important role in feature extraction and classification in classical linear dimensionality reduction methods is still not investigated well. This paper proposes a novel supervised learning method called sparse local discriminant projections (SLDPs) for linear dimensionality reduction. Differed from the recent manifold-learning-based methods such as local preserving projections (LPPs), SLDP introduces a sparse constraint into the objective function and integrates the local geometry, discriminant information and within-class geometry to obtain the sparse projections. The sparse projections can be efficiently computed by the Elastic Net. The most important and interesting thing is that the sparse projections learned by SLDP have a direct physical interpretation and provide us the discriminant knowledge and insightful understanding for the extracted features. The experimental results show that SLDP can give reasonable semantic results and achieves competitive performance compared with some techniques such as PCA, LPP, neighbourhood preserving embedding (NPE) and the recently proposed unified sparse subspace learning (USSL).

On Dimensionality Reduction for Indexing and Retrieval of Large-Scale Solar Image Data

This work investigates the applicability of several dimensionality reduction techniques for large-scale solar data analysis. Using a solar benchmark dataset that contains images of multiple types of phenomena, we investigate linear and nonlinear dimensionality reduction methods in order to reduce our storage and processing costs and maintain a good representation of our data in a new vector space. We present a comparative analysis of several dimensionality reduction methods and different numbers of target dimensions by utilizing different classifiers in order to determine the degree of data dimensionality reduction that can be achieved with these methods, and to discover the method that is the most effective for solar images. After determining the optimal number of dimensions, we then present preliminary results on indexing and retrieval of the dimensionally reduced data.

Visualizing dimensionality reduction of systems biology data

One of the challenges in analyzing high-dimensional expression data is the detection of important biological signals. A common approach is to apply a dimension reduction method, such as principal component analysis. Typically, after application of such a method the data is projected and visualized in the new coordinate system, using scatter plots or profile plots. These methods provide good results if the data have certain properties which become visible in the new coordinate system and which were hard to detect in the original coordinate system. Often however, the application of only one method does not suffice to capture all important signals. Therefore several methods addressing different aspects of the data need to be applied. We have developed a framework for linear and non-linear dimension reduction methods within our visual analytics pipeline SpRay. This includes measures that assist the interpretation of the factorization result. Different visualizations of these measures can be combined with functional annotations that support the interpretation of the results. We show an application to high-resolution time series microarray data in the antibiotic-producing organism Streptomyces coelicolor as well as to microarray data measuring expression of cells with normal karyotype and cells with trisomies of human chromosomes 13 and 21.

Face recognition using Elasticfaces

Though principle component analysis (PCA) and locality preserving projections (LPPs) are two of the most popular linear methods for face recognition, PCA can only see the Euclidean structure of the training set and LPP preserves the nonlinear submanifold structure hidden in the training set. In this paper, we propose the elastic preserving projections (EPPs) which by incorporating the merits of the local geometry and the global information of the training set. EPP outputs a sample subspace which simultaneously preserves the local geometrical structure and exploits the global information of the training set. Different from some other linear dimensionality reduction methods, EPP can be deemed as learning both the coordinates and the affinities between sample points. Furthermore, the effectiveness of our proposed algorithm is analyzed theoretically and confirmed by some experiments on several well-known face databases. The obtained results indicate that EPP significantly outperforms its other rival algorithms.

Identifying Key Life Cycle Assessment Metrics in the Multiobjective Design of Bioethanol Supply Chains Using a Rigorous Mixed-Integer Linear Programming Approach

The design of more sustainable bioethanol supply chains (SCs) has recently emerged as an active area of research. Most of the approaches presented so far have somehow a limited scope, as they focus on minimizing the emitted greenhouse gases as unique criterion, neglecting the damage caused in other impact categories. In this work, we address the multiobjective design of bioethanol SCs considering several life cycle assessment impacts. To overcome the numerical difficulties of dealing with several objective functions, we investigate the application of a rigorous mixed-integer linear programming-based dimensionality reduction method that minimizes the error of omitting objectives. The usefulness of this approach is tested through its application to the design of a bioethanol/sugar SC in Argentina, in which five environmental objectives are simultaneously optimized along with the net present value. The proposed method makes it possible to reduce the number of environmental indicators, thereby facilitating the calculation and analysis of the Pareto solutions.

Linear Semi-Supervised Dimensionality Reduction with Pairwise Constraint for Multiple Subclasses

We propose a new method, called Subclass-oriented Dimensionality Reduction with Pairwise Constraints (SODRPaC), for dimensionality reduction. In a high dimensional space, it is common that a group of data points with one class may scatter in several different groups. Current linear semi-supervised dimensionality reduction methods would fail to achieve fair performances, as they assume two data points linked by a must-link constraint are close each other, while they are likely to be located in different groups. Inspired by the above observation, we classify the must-link constraint into two categories, which are the inter-subclass must-link constraint and the intra-subclass must-link constraint, respectively. We carefully generate cannot-link constraints by using must-link constraints, and then propose a new discriminant criterion by employing the cannot-link constraints and the compactness of shared nearest neighbors. The manifold regularization is also incorporated in our dimensionality reduction framework. Extensive experiments on both synthetic and practical data sets illustrate the effectiveness of our method.

Motion Retrieval Using Low‐Rank Subspace Decomposition of Motion Volume

AbstractThis paper proposes a novel framework that allows for a flexible and an efficient retrieval of motion capture data in huge databases. The method first converts an action sequence into a novel representation, i.e. the Self‐Similarity Matrix (SSM), which is based on the notion of self‐similarity. This conversion of the motion sequences into compact and low‐rank subspace representations greatly reduces the spatiotemporal dimensionality of the sequences. The SSMs are then used to construct order‐3 tensors, and we propose a low‐rank decomposition scheme that allows for converting the motion sequence volumes into compact lower dimensional representations, without losing the nonlinear dynamics of the motion manifold. Thus, unlike existing linear dimensionality reduction methods that distort the motion manifold and lose very critical and discriminative components, the proposed method performs well even when inter‐class differences are small or intra‐class differences are large. In addition, the method allows for an efficient retrieval and does not require the time‐alignment of the motion sequences. We evaluate the performance of our retrieval framework on the CMU mocap dataset under two experimental settings, both demonstrating promising retrieval rates.

Multi-scale gist feature manifold for building recognition

Multi-scale gist (MS-gist) feature manifold for building recognition is presented in the paper. It is described as a two-stage model. In the first stage, we extract the multi-scale gist features that represent the structural information of the building images. Since the MS-gist features are extrinsically high dimensional and intrinsically low dimensional, in the second stage, an enhanced fuzzy local maximal marginal embedding (EFLMME) algorithm is proposed to project MS-gist feature manifold to low-dimensional subspace. EFLMME aims to preserve local intra-class geometry and maximize local interclass margin separability of MS-gist feature manifold of different classes at the same time. To evaluate the performance of our proposed model, experiments were carried out on the Sheffield buildings database, compared with the existing works: (a) the visual gist based building recognition model (VGBR) and (b) the hierarchical building recognition model (HBR). Moreover, EFLMME is evaluated on Sheffield buildings database compared with some linear dimensionality reduction methods. The results show that the proposed model is superior to other models in practice of building recognition and can handle the building recognition problem caused by rotations, variant lighting conditions and occlusions very well.

Orthogonal isometric projection for face recognition

Isometric projection (IsoProjection) is a linear dimensionality reduction method, which explicitly takes into account the manifold structure embedded in the data. However, IsoProjection is non-orthogonal, which makes it extremely sensitive to the dimensions of reduced space and difficult to estimate the intrinsic dimensionality. The non-orthogonality also distorts the metric structure embedded in the data. This paper proposes a new method called orthogonal isometric projection (O-IsoProjection), which shares the same linear character as IsoProjection and overcomes the metric distortion problem of IsoProjection. Similar to IsoProjection, O-IsoProjection firstly constructs an adjacency graph which can reflect the manifold structure embedded in the data and the class relationship between the sample points of face space, and then obtains the projections by preserving such a graph structure. Different from IsoProjection, O-IsoProjection requires the basis vectors to be orthogonal, and the orthogonal basis vectors can be calculated by iterative way. Experimental results on ORL and Yale databases show that O-IsoProjection has better recognition rate for face recognition than Eigenface, Fisherface and IsoProjection.

Locally linear embedding-based seismic attribute extraction and applications

How to extract optimal composite attributes from a variety of conventional seismic attributes to detect reservoir features is a reservoir predication key, which is usually solved by reducing dimensionality. Principle component analysis (PCA) is the most widely-used linear dimensionality reduction method at present. However, the relationships between seismic attributes and reservoir features are non-linear, so seismic attribute dimensionality reduction based on linear transforms can’t solve non-linear problems well, reducing reservoir prediction precision. As a new non-linear learning method, manifold learning supplies a new method for seismic attribute analysis. It can discover the intrinsic features and rules hidden in the data by computing low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs. In this paper, we try to extract seismic attributes using locally linear embedding (LLE), realizing inter-horizon attributes dimensionality reduction of 3D seismic data first and discuss the optimization of its key parameters. Combining model analysis and case studies, we compare the dimensionality reduction and clustering effects of LLE and PCA, both of which indicate that LLE can retain the intrinsic structure of the inputs. The composite attributes and clustering results based on LLE better characterize the distribution of sedimentary facies, reservoir, and even reservoir fluids.

Enhanced locality preserving projections using robust path based similarity

Curse of dimensionality is a bothering problem in high dimensional data analysis. To enhance the performances of classification or clustering on these data, their dimensionalities should be reduced beforehand. Locality Preserving Projections (LPP) is a widely used linear dimensionality reduction method. It seeks a subspace in which the neighborhood graph structure of samples is preserved. However, like most dimensionality reduction methods based on graph embedding, LPP is sensitive to noise and outliers, and its effectiveness depends on choosing suitable parameters for constructing the neighborhood graph. Unfortunately, it is difficult to choose effective parameters for LPP. To address these problems, we propose an Enhanced LPP (ELPP) using a similarity metric based on robust path and a Semi-supervised ELPP (SELPP) with pairwise constraints. In comparison with original LPP, our methods are not only robust to noise and outliers, but also less sensitive to parameters selection. Besides, SELPP makes use of pairwise constraints more efficiently than other comparing methods. Experimental results on real world face databases confirm their effectiveness.

Distance metric learning by minimal distance maximization

Classic linear dimensionality reduction (LDR) methods, such as principal component analysis (PCA) and linear discriminant analysis (LDA), are known not to be robust against outliers. Following a systematic analysis of the multi-class LDR problem in a unified framework, we propose a new algorithm, called minimal distance maximization (MDM), to address the non-robustness issue. The principle behind MDM is to maximize the minimal between-class distance in the output space. MDM is formulated as a semi-definite program (SDP), and its dual problem reveals a close connection to ''weighted'' LDR methods. A soft version of MDM, in which LDA is subsumed as a special case, is also developed to deal with overlapping centroids. Finally, we drop the homoscedastic Gaussian assumption made in MDM by extending it in a non-parametric way, along with a gradient-based convex approximation algorithm to significantly reduce the complexity of the original SDP. The effectiveness of our proposed methods are validated on two UCI datasets and two face datasets.

Canonical correlation analysis using within-class coupling

Fisher’s linear discriminant analysis (LDA) is one of the most popular supervised linear dimensionality reduction methods. Unfortunately, LDA is not suitable for problems where the class labels are not available and only the spatial or temporal association of data samples is implicitly indicative of class membership. In this study, a new strategy for reducing LDA to Hotelling’s canonical correlation analysis (CCA) is proposed. CCA seeks prominently correlated projections between two views of data and it has been long known to be equivalent to LDA when the data features are used in one view and the class labels are used in the other view. The basic idea of the new equivalence between LDA and CCA, which we call within-class coupling CCA (WCCCA), is to apply CCA to pairs of data samples that are most likely to belong to the same class. We prove the equivalence between LDA and such an application of CCA. With such an implicit representation of the class labels, WCCCA is applicable both to regular LDA problems and to problems in which only spatial and/or temporal continuity provides clues to the class labels.

Clustering via dimensional reduction method for the projection pursuit based on the ICSA

The performance of the classical clustering algorithm is not always satisfied with the high-dimensional datasets, which make clustering method limited in many application. To solve this problem, clustering method with Projection Pursuit dimension reduction based on Immune Clonal Selection Algorithm (ICSA-PP) is proposed in this paper. Projection pursuit strategy can maintain consistent Euclidean distances between points in the low-dimensional embeddings where the ICSA is used to search optimizing projection direction. The proposed algorithm can converge quickly with less iteration to reduce dimension of some high-dimensional datasets, and in which space, K-mean clustering algorithm is used to partition the reduced data. The experiment results on UCI data show that the presented method can search quicker to optimize projection direction than Genetic Algorithm (GA) and it has better clustering results compared with traditional linear dimension reduction method for Principle Component Analysis (PCA).

Randomized anisotropic transform for nonlinear dimensionality reduction

An innovative method is introduced in this paper to significantly increase computational speed and to reduce memory usage, when applied to nonlinear methods and algorithms for dimensionality reduction (DR). Due to the incapability of linear DR methods in the preservation of data geometry, the need of effective nonlinear approaches has recently attracted the attention of many researchers, particularly from the Mathematics and Computer Science communities. The common theme of the current nonlinear DR approaches is formulation of certain matrices, called dimensionality reduction kernels (DRK) in terms of the data points, followed by performing spectral decomposition. Hence, for datasets of large size with data points that lie in some high dimensional space, the matrix dimension of the DRK is very large. Typical examples for the need of very high dimensional DRK arise from such applications as processing multispectral imagery data, searching desired text documentary data in the internet, and recognizing human faces from given libraries. For such and many other applications, the matrix dimension of the DRK is so large that computation for carrying out spectral decomposition of the DRK often encounters various difficulties, not only due to the need of significantly large read-only memory (ROM), but also due to computational instability. The main objective of this paper is to introduce the notion of the anisotropic transform (AT) and to develop its corresponding effective and efficient computational algorithms by integrating random embedding with the AT to formulate the randomized anisotropic transform (RAT), in order to reduce the size of the DRK significantly, while preserving local geometries of the given datasets. Illustrations with various examples will also be given in this paper to demonstrate that RAT algorithms dramatically reduce the ROM and CPU requirement, and thus allow fast processing, even on PC and potentially for handhold devices as well.

Predicting Subcellular Localization of Gram-Negative Bacterial Proteins by Linear Dimensionality Reduction Method

With the rapid increase of protein sequences in the post-genomic age, the need for an automated and accurate tool to predict protein subcellular localization becomes increasingly important. Many efforts have been tried. Most of them aim to find the optimal classification scheme and less of them take the simplifying the complexity of biological system into consideration. This work shows how to decrease the complexity of biological system with linear DR (Dimensionality Reduction) method by transforming the original high-dimensional feature vectors into the low-dimensional feature vectors. A powerful sequence encoding scheme by fusing PSSM (Position-Specific Score Matrix) and Chou's PseAA (Pseudo Amino Acid) composition is proposed to represent the protein samples. Then, the K-NN (K-Nearest Neighbor) classifier is employed to identify the subcellular localization based on their reduced low-dimensional feature vectors. Experimental results thus obtained are quite encouraging, indicating that the aforementioned linear DR method is quite promising in dealing with complicated biological problems, such as predicting the subcellular localization of Gram-negative bacterial proteins.

Application of Locality Preserving Projections in Face Recognition

Face recognition technology has evolved as an enchanting solution to address the contemporary needs in order to perform identification and verification of identity claims. By advancing the feature extraction methods and dimensionality reduction techniques in the application of pattern recognition, a number of face recognition systems has been developed with distinct degrees of success. Locality preserving projection (LPP) is a recently proposed method for unsupervised linear dimensionality reduction. LPP preserve the local structure of face image space which is usually more significant than the global structure preserved by principal component analysis (PCA) and linear discriminant analysis (LDA). This paper focuses on a systematic analysis of locality-preserving projections and the application of LPP in combination with an existing technique This combined approach of LPP through MPCA can preserve the global and the local structure of the face image which is proved very effective. Proposed approach is tested using the AT & T face database. Experimental results show the significant improvements in the face recognition performance in comparison with some previous methods.

Exploring nonlinear feature space dimension reduction and data representation in breast CADx with Laplacian eigenmaps and ‐SNE

In this preliminary study, recently developed unsupervised nonlinear dimension reduction (DR) and data representation techniques were applied to computer-extracted breast lesion feature spaces across three separate imaging modalities: Ultrasound (U.S.) with 1126 cases, dynamic contrast enhanced magnetic resonance imaging with 356 cases, and full-field digital mammography with 245 cases. Two methods for nonlinear DR were explored: Laplacian eigenmaps [M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput. 15, 1373-1396 (2003)] and t-distributed stochastic neighbor embedding (t-SNE) [L. van der Maaten and G. Hinton, "Visualizing data using t-SNE," J. Mach. Learn. Res. 9, 2579-2605 (2008)]. These methods attempt to map originally high dimensional feature spaces to more human interpretable lower dimensional spaces while preserving both local and global information. The properties of these methods as applied to breast computer-aided diagnosis (CADx) were evaluated in the context of malignancy classification performance as well as in the visual inspection of the sparseness within the two-dimensional and three-dimensional mappings. Classification performance was estimated by using the reduced dimension mapped feature output as input into both linear and nonlinear classifiers: Markov chain Monte Carlo based Bayesian artificial neural network (MCMC-BANN) and linear discriminant analysis. The new techniques were compared to previously developed breast CADx methodologies, including automatic relevance determination and linear stepwise (LSW) feature selection, as well as a linear DR method based on principal component analysis. Using ROC analysis and 0.632+bootstrap validation, 95% empirical confidence intervals were computed for the each classifier's AUC performance. In the large U.S. data set, sample high performance results include, AUC0.632+ = 0.88 with 95% empirical bootstrap interval [0.787;0.895] for 13 ARD selected features and AUC0.632+ = 0.87 with interval [0.817;0.906] for four LSW selected features compared to 4D t-SNE mapping (from the original 81D feature space) giving AUC0.632+ = 0.90 with interval [0.847;0.919], all using the MCMC-BANN. Preliminary results appear to indicate capability for the new methods to match or exceed classification performance of current advanced breast lesion CADx algorithms. While not appropriate as a complete replacement of feature selection in CADx problems, DR techniques offer a complementary approach, which can aid elucidation of additional properties associated with the data. Specifically, the new techniques were shown to possess the added benefit of delivering sparse lower dimensional representations for visual interpretation, revealing intricate data structure of the feature space.