Nonlinear Dimensionality Reduction Algorithm Research Articles

Regularization parameter choice in locally linear embedding

Locally linear embedding (LLE) is a recent unsupervised learning algorithm for non-linear dimensionality reduction of high dimensional data. One advantage of this algorithm is that just two parameters are needed to be set by user: the number of nearest neighbors and a regularization parameter. The choice of the regularization parameter plays an important role in the embedding results. In this paper, an automated method for choosing this parameter is proposed. Besides, in order to objectively qualify the performance of the embedding results, a new measure of embedding quality is suggested. Our approach is experimentally verified on 9 artificial data sets and 2 real world data sets. Numerical results are compared against two methods previously found in the state of art.

Multi-Channel Transfer Function with Dimensionality Reduction.

The design of transfer functions for volume rendering is a difficult task. This is particularly true for multi-channel data sets, where multiple data values exist for each voxel. In this paper, we propose a new method for transfer function design. Our new method provides a framework to combine multiple approaches and pushes the boundary of gradient-based transfer functions to multiple channels, while still keeping the dimensionality of transfer functions to a manageable level, i.e., a maximum of three dimensions, which can be displayed visually in a straightforward way. Our approach utilizes channel intensity, gradient, curvature and texture properties of each voxel. The high-dimensional data of the domain is reduced by applying recently developed nonlinear dimensionality reduction algorithms. In this paper, we used Isomap as well as a traditional algorithm, Principle Component Analysis (PCA). Our results show that these dimensionality reduction algorithms significantly improve the transfer function design process without compromising visualization accuracy. In this publication we report on the impact of the dimensionality reduction algorithms on transfer function design for confocal microscopy data.

Predicting human immunodeficiency virus protease cleavage sites in nonlinear projection space

HIV-1 protease has a broad and complex substrate specificity. The discovery of an accurate, robust, and rapid method for predicting the cleavage sites in proteins by HIV protease would greatly expedite the search for inhibitors of HIV protease. During the last two decades, various methods have been developed to explore the specificity of HIV protease cleavage activity. However, because little advancement has been made in the understanding of HIV-1 protease cleavage site specificity, not much progress has been reported in either extracting effective methods or maintaining high prediction accuracy. In this article, a theoretical framework is developed, based on the kernel method for dimensionality reduction and prediction for HIV-1 protease cleavage site specificity. A nonlinear dimensionality reduction kernel method, based on manifold learning, is proposed to reduce the high dimensions of protease specificity. A support vector machine is applied to predict the protease cleavage. Superior performance in comparison to that previously published in literature is obtained using numerical simulations showing that the basic specificities of the HIV-1 protease are maintained in reduction feature space, and by combining the nonlinear dimensionality reduction algorithm with a support vector machine classifier.

Nonlinear dimensionality reduction with hybrid distance for trajectory representation of dynamic texture

Dynamic textures play an important role in video content analysis. Current works of dynamic textures mainly focus on overall texture and motion analysis for segmentation or classification based on statistical features and structure models. This paper proposes a novel framework to study the dynamic textures by exploring the motion trajectory using unsupervised learning. A nonlinear dimensionality reduction algorithm, called hybrid distance isometric embedding (HDIE), is proposed, to generate a low-dimensional motion trajectory from high-dimensional feature space of the raw video data. First, we partition the high-dimensional data points into a set of data clusters and construct the intra-cluster graphs based on the individual character of each data cluster to build the basic layer of HDIE. Second, we construct the inter-cluster graph by analyzing the interrelation among these isolated data clusters to build the top layer of HDIE. Finally, we generate a whole graph and map all data points into a unique low-dimensional feature space, trying to maintain the distances of all pairs of high-dimensional data points. Experiments on the standard dynamic texture database show that the proposed framework with the novel algorithm can represent the motion characters of the dynamic textures very well.

Nonlinear Dimensionality Reduction with Local Spline Embedding

This paper presents a new algorithm for nonlinear dimensionality reduction (NLDR). Our algorithm is developed under the conceptual framework of compatible mapping. Each such mapping is a compound of a tangent space projection and a group of splines. Tangent space projection is estimated at each data point on the manifold, through which the data point itself and its neighbors are represented in tangent space with local coordinates. Splines are then constructed to guarantee that each of the local coordinates can be mapped to its own single global coordinate with respect to the underlying manifold. Thus, the compatibility between local alignments is ensured. In such a work setting, we develop an optimization framework based on reconstruction error analysis, which can yield a global optimum. The proposed algorithm is also extended to embed out of samples via spline interpolation. Experiments on toy data sets and real-world data sets illustrate the validity of our method.

Enhanced supervised locally linear embedding

In this paper, a new nonlinear dimensionality reduction algorithm, called enhanced supervised locally linear embedding (ESLLE), is proposed. The ESLLE method attempts to make the interclass dissimilarity definitely larger than the intraclass dissimilarity in an effort to strengthen the discriminating power and generalization ability of embedded data representation. Simulation studies on artificial manifold data demonstrate that ESLLE can give better embedding results in dimensionality reduction and is more robust to noise in comparison with the original supervised LLE (SLLE). Experimental results on extended Yale face database B and CMU PIE face databases demonstrate that ESLLE obtains better performance on face recognition compared with other famous methods such as principal component analysis (PCA), locally linear embedding (LLE) as well as SLLE.

Manifold Analysis by Topologically Constrained Isometric Embedding

— We present a new algorithm for nonlinear dimensionality reduction that consistently uses global information, and that enables understanding the intrinsic geometry of non-convex manifolds. Compared to methods that consider only local information, our method appears to be more robust to noise. Unlike most methods that incorporate global information, the proposed approach automatically handles non-convexity of the data manifold. We demonstrate the performance of our algorithm and compare it to state-of-the-art methods on synthetic as well as real data.

Nonlinear dimensionality reduction with relative distance comparison

This paper proposes a new algorithm for nonlinear dimensionality reduction. Our basic idea is to explore and exploit the local geometry of the manifold with relative distance comparisons. All such comparisons derived from local neighborhoods are enumerated to constrain the manifold to be learned. The task is formulated as a problem of quadratically constrained quadratic programming (QCQP). However, such a QCQP problem is not convex. We relax it to be a problem of semi-definite programming (SDP), from which a globally optimal embedding is obtained. Experimental results illustrate the validity of our algorithm.

Dimensionality reduction for heterogeneous dataset in rushes editing

Rushes editing enables the computer to edit the film like a professional film cutter based on the raw footage. The most important issue in rushes editing is the generation of the effective, efficient, and robust descriptors for footage content analysis. Dimensionality reduction technology provides the means to generate such descriptors by seeking a low-dimensional equivalence of the high-dimensional video data using intelligent algorithms. However, existing dimensionality reduction techniques are not directly applicable to the editing of rushes because of the heterogeneity of rushes data. To deal with this heterogeneity, this paper proposes a novel non-linear dimensionality reduction algorithm called multi-layer isometric feature mapping (ML-Isomap). First, a clustering algorithm is utilized to partition the high-dimensional data points into a set of data blocks in the high-dimensional feature space. Second, intra-cluster graphs are constructed based on the individual character of each data block to build the basic layer for the ML-Isomap. Third, the inter-cluster graph is constructed by analyzing the interrelation among these isolated data blocks to build the hyper-layers for the ML-Isomap. Finally, all the data points are mapped into the unique low-dimensional feature space by maintaining to the greatest extent the corresponding relations of the multiple layers in the high-dimensional feature space. Comparative experiments on synthetic data as well as real rushes editing tasks demonstrate that the proposed algorithm can reduce the dimensions of various datasets efficiently while preserving both the global structure and the local details of the heterogeneous dataset.

Modeling multiscale diffusion processes in random heterogeneous media

A simple stochastic up-scaling strategy is presented for incorporating the effects of micro-scale variations of thermal conductivity in the thermal analysis of macro-scale systems. Incomplete characterization of the fine-scale variation necessitates posing the thermal conductivity as a stochastic multiscale random field. A data-driven approach coupled with a non-linear dimension reduction algorithm is used to construct a low-dimensional stochastic descriptor of the fine-scale variation of the stochastic conductivity. A representative volume element (RVE) argument is utilized to construct a simple model for up-scaling the stochastic thermal conductivity. In this model, the effective stochastic thermal conductivity is estimated by virtually interrogating the stochastic RV. A realistic example of the proposed stochastic multiscale diffusion strategy is illustrated utilizing limited experimental data.

Application of a non-linear dimension reduction algorithm on document clustering

This paper presented a non-linear dimension reduction algorithm-Self-organizing Isometric Embedding(SIE)to compress high-dimensional document data.The algorithm was then validated in document clustering by being compared with the typical linear dimension reduction algorithm-Latent Semantic Indexing(LSI).Experimental results show that while significantly lowering the complexity,the performance of SIE is better than that of LSI and the benchmark.

MODIFIED LAPLACIAN EIGENMAP METHOD FOR FAULT DIAGNOSIS

A novel method based on the improved Laplacian eigenmap algorithm for fault pattern classification is proposed.Via modifying the Laplacian eigenmap algorithm to replace Euclidean distance with kernel-based geometric distance in the neighbor graph construction,the method can preserve the consistency of local neighbor information and effectively extract the low-dimensional manifold features embedded in the high-dimensional nonlinear data sets.A nonlinear dimensionality reduction algorithm based on the improved Laplacian eigenmap is to directly learn high-dimensional fault signals and extract the intrinsic manifold features from them.The method greatly preserves the global geometry structure information embedded in the signals,and obviously improves the classification performance of fault pattern recognition.The experimental results on both simulation and engineering indicate the feasibility and effectiveness of the new method.

Supervised kernel neighborhood preserving projections for radar target recognition

Dimensionality reduction is a crucial step for pattern recognition tasks and finding a suitable low-dimensional subspace has an important effect on recognition performance. Recently, the computer vision and pattern recognition community has witnessed the rapid growth of a new kind of dimensionality reduction method, manifold learning. Among them, neighborhood preserving projections (NPP) is one of the most promising techniques. Based on NPP, we propose a novel nonlinear dimensionality reduction algorithm, called supervised kernel neighborhood preserving projections (SKNPP), which aims at preserving the local manifold structures defined by within-class samples in some high-dimensional feature space. SKNPP can not only gain a perfect nonlinear approximation of data manifold through kernel technique, but also enhance the local within-class relations by taking into account class label information. The proposed SKNPP is compared with NPP, LDA and KLDA on radar target recognition with range profiles. Experimental results indicate the promising recognition performance of the proposed method.

Analysis of an alignment algorithm for nonlinear dimensionality reduction

The goal of dimensionality reduction or manifold learning for a given set of high-dimensional data points, is to find a low-dimensional parametrization for them. Usually it is easy to carry out this parametrization process within a small region to produce a collection of local coordinate systems. Alignment is the process to stitch those local systems together to produce a global coordinate system and is done through the computation of a partial eigendecomposition of a so-called alignment matrix. In this paper, we present an analysis of the alignment process, giving conditions under which the null space of the alignment matrix recovers the global coordinate system up to an affine transformation. We also propose a post-processing step that can determine the global coordinate system up to a rigid motion. This in turn shows that Local Tangent Space Alignment method (LTSA) can recover a locally isometric embedding up to a rigid motion.

MANIFOLD LEARNING FOR ROBOT NAVIGATION

In this paper we introduce methods to build a SOM that can be used as an isometric map for mobile robots. That is, given a dataset of sensor readings collected at points uniformly distributed with respect to the ground, we wish to build a SOM whose neurons (prototype vectors in sensor space) correspond to points uniformly distributed on the ground. Manifold learning techniques have already been used for dimensionality reduction of sensor space in navigation systems. Our focus is on the isometric property of the SOM. For reliable path-planning and information sharing between several robots, it is desirable that the robots build an internal representation of the sensor manifold, a map, that is isometric with the environment. We show experimentally that standard Non-Linear Dimensionality Reduction (NLDR) algorithms do not provide isometric maps for range data and bearing data. However, the auxiliary low dimensional manifolds created can be used to improve the distribution of the neurons of a SOM (that is, make the neurons more evenly distributed with respect to the ground). We also describe a method to create an isometric map from a sensor readings collected along a polygonal line random walk.

Principal manifolds and nonlinear dimensionality reduction via tangent space alignment

We present a new algorithm for manifold learning and nonlinear dimensionality reduction. Based on a set of unorganized data points sampled with noise from a parameterized manifold, the local geometry of the manifold is learned by constructing an approximation for the tangent space at each point, and those tangent spaces are then aligned to give the global coordinates of the data points with respect to the underlying manifold. We also present an error analysis of our algorithm showing that reconstruction errors can be quite small in some cases. We illustrate our algorithm using curves and surfaces both in 2D/3D Euclidean spaces and higher dimensional Euclidean spaces. We also address several theoretical and algorithmic issues for further research and improvements.

Locally Linear Embedding for dimensionality reduction in QSAR

Current practice in Quantitative Structure Activity Relationship (QSAR) methods usually involves generating a great number of chemical descriptors and then cutting them back with variable selection techniques. Variable selection is an effective method to reduce the dimensionality but may discard some valuable information. This paper introduces Locally Linear Embedding (LLE), a local non-linear dimensionality reduction technique, that can statistically discover a low-dimensional representation of the chemical data. LLE is shown to create more stable representations than other non-linear dimensionality reduction algorithms, and to be capable of capturing non-linearity in chemical data.

Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment

We present a new algorithm for manifold learning and nonlinear dimensionality reduction. Based on a set of unorganized data points sampled with noise from a parameterized manifold, the local geometry of the manifold is learned by constructing an approximation for the tangent space at each data point, and those tangent spaces are then aligned to give the global coordinates of the data points with respect to the underlying manifold. We also present an error analysis of our algorithm showing that reconstruction errors can be quite small in some cases. We illustrate our algorithm using curves and surfaces both in two-dimensional/three-dimensional (2D/3D) Euclidean spaces and in higher-dimensional Euclidean spaces. We also address several theoretical and algorithmic issues for further research and improvements.