Unsupervised Nonlinear Dimensionality Reduction Research Articles

Assessing the Interplay of Shape and Physical Parameters by Unsupervised Nonlinear Dimensionality Reduction Methods

The article presents an exploratory study on the application to ship hydrodynamics of unsupervised nonlinear design-space dimensionality reduction methods, assessing the interaction of shape and physical parameters. Nonlinear extensions of the principal component analysis (PCA) are applied, namely local PCA (LPCA) and kernel PCA (KPCA). An artificial neural network approach, specifically a deep autoencoder (DAE) method, is also applied and compared with PCA-based approaches. The data set under investigation is formed by the results of 9000 potential flow simulations coming from an extensive exploration of a 27-dimensional design space, associated with a shape optimization problem of the DTMB 5415 model in calm water at 18 kn (Froude number, Fr = 25). Data include three heterogeneous distributed and suitably discretized parameters (shape modification vector, pressure distribution on the hull, and wave elevation pattern) and one lumped parameter (wave resistance coefficient), for a total of 9000 x 5101 elements. The reduced-dimensionality representation of shape and physical parameters is set to provide a normalized mean squared error smaller than 5%. The standard PCA meets the requirement using 19 principal components/parameters. LPCA and KPCA provide the most promising compression capability with 14 parameters required by the reduced-dimensionality parametrizations, indicating significant nonlinear interactions in the data structure of shape and physical parameters. The DAE achieves the same error with 17 components. Although the focus of the current work is on design-space dimensionality reduction, the formulation goes beyond shape optimization and can be applied to large sets of heterogeneous physical data from simulations, experiments, and real operation measurements.

Visualization of Single Cell RNA-Seq Data Using t-SNE in R.

Single cell RNA sequencing (scRNA-seq) is a powerful tool to analyze cellular heterogeneity, identify new cell types, and infer developmental trajectories, which has greatly facilitated studies on development, immunity, cancer, neuroscience, and so on. Visualizing of scRNA-Seq data is fundamental and essential because it is critical to biological interpretation. Although principal component analysis (PCA) is used for visualizing scRNA-seq at early studies, t-Distributed Stochastic Neighbor embedding (t-SNE), an unsupervised nonlinear dimensionality reduction technique, is widely used nowadays due to its advantage in visualization of scRNA-seq data. Here, we detailed the process of visualization of single-cell RNA-seq data using t-SNE via Seurat, an R toolkit for single cell genomics.

Multilayer bootstrap networks

Multilayer bootstrap network builds a gradually narrowed multilayer nonlinear network from bottom up for unsupervised nonlinear dimensionality reduction. Each layer of the network is a nonparametric density estimator. It consists of a group of k-centroids clusterings. Each clustering randomly selects data points with randomly selected features as its centroids, and learns a one-hot encoder by one-nearest-neighbor optimization. Geometrically, the nonparametric density estimator at each layer projects the input data space to a uniformly-distributed discrete feature space, where the similarity of two data points in the discrete feature space is measured by the number of the nearest centroids they share in common. The multilayer network gradually reduces the nonlinear variations of data from bottom up by building a vast number of hierarchical trees implicitly on the original data space. Theoretically, the estimation error caused by the nonparametric density estimator is proportional to the correlation between the clusterings, both of which are reduced by the randomization steps.

Extracting the nonlinear features of motor imagery EEG using parametric t-SNE

When performing studies on brain computer interface based rehabilitation problems, researchers frequently encounter difficulty due to the curse of dimensionality and the nonlinear nature of Motor Imagery Electroencephalography (MI-EEG). Though many approaches have been proposed recently to address the feature extraction problem and have shown surprising performance, unfortunately, most of them are non-parametric or linear dimension reduction techniques, which are limited in utility for out-of-sample extension for MI-EEG classification. To address the problem and obtain accurate MI-EEG features, a new unsupervised nonlinear dimensionality reduction technique termed parametric t-Distributed Stochastic Neighbor Embedding (P. t-SNE) is employed to extract the nonlinear features from MI-EEG. Considering that MI-EEG is a kind of non-stationary signal with remarkable time-frequency rhythmic distribution characteristics, Discrete Wavelet Transform (DWT) is used to extract the time-frequency features of MI-EEG. Furthermore, P. t-SNE is applied to selected wavelet components to get the nonlinear features. They are then combined serially to construct the feature vector. Experiments are conducted on a publicly available dataset, and the experimental results show that the nonlinear features have great visualization performance with obvious clustering distribution, and the feature extraction method indicates excellent classification performance as evaluated by a support vector machine classifier. This paper suggests a manifold based technique for further analysis and classification research of MI-EEG.

Discriminating complex networks through supervised NDR and Bayesian classifier

Discriminating complex networks is a particularly important task for the purpose of the systematic study of networks. In order to discriminate unknown networks exactly, a large set of network measurements are needed to be taken into account for comprehensively considering network properties. However, as we demonstrate in this paper, these measurements are nonlinear correlated with each other in general, resulting in a wide variety of redundant measurements which unintentionally explain the same aspects of network properties. To solve this problem, we adopt supervised nonlinear dimensionality reduction (NDR) to eliminate the nonlinear redundancy and visualize networks in a low-dimensional projection space. Though unsupervised NDR can achieve the same aim, we illustrate that supervised NDR is more appropriate than unsupervised NDR for discrimination task. After that, we perform Bayesian classifier (BC) in the projection space to discriminate the unknown network by considering the projection score vectors as the input of the classifier. We also demonstrate the feasibility and effectivity of this proposed method in six extensive research real networks, ranging from technological to social or biological. Moreover, the effectiveness and advantage of the proposed method is proved by the contrast experiments with the existing method.

Kernel ridge regression for out-of-sample mapping in supervised manifold learning

Manifold learning methods for unsupervised nonlinear dimensionality reduction have proven effective in the visualization of high dimensional data sets. When dealing with classification tasks, supervised extensions of manifold learning techniques, in which class labels are used to improve the embedding of the training points, require an appropriate method for out-of-sample mapping.In this paper we propose multi-output kernel ridge regression (KRR) for out-of-sample mapping in supervised manifold learning, in place of general regression neural networks (GRNN) that have been adopted by previous studies on the subject. Specifically, we consider a supervised agglomerative variant of Isomap and compare the performance of classification methods when the out-of-sample embedding is based on KRR and GRNN, respectively. Extensive computational experiments, using support vector machines and k-nearest neighbors as base classifiers, provide statistical evidence that out-of-sample mapping based on KRR consistently dominates its GRNN counterpart, and that supervised agglomerative Isomap with KRR achieves a higher accuracy than direct classification methods on most data sets.

Non-linear dimensionality reduction of signaling networks

BackgroundSystems wide modeling and analysis of signaling networks is essential for understanding complex cellular behaviors, such as the biphasic responses to different combinations of cytokines and growth factors. For example, tumor necrosis factor (TNF) can act as a proapoptotic or prosurvival factor depending on its concentration, the current state of signaling network and the presence of other cytokines. To understand combinatorial regulation in such systems, new computational approaches are required that can take into account non-linear interactions in signaling networks and provide tools for clustering, visualization and predictive modeling.ResultsHere we extended and applied an unsupervised non-linear dimensionality reduction approach, Isomap, to find clusters of similar treatment conditions in two cell signaling networks: (I) apoptosis signaling network in human epithelial cancer cells treated with different combinations of TNF, epidermal growth factor (EGF) and insulin and (II) combination of signal transduction pathways stimulated by 21 different ligands based on AfCS double ligand screen data. For the analysis of the apoptosis signaling network we used the Cytokine compendium dataset where activity and concentration of 19 intracellular signaling molecules were measured to characterise apoptotic response to TNF, EGF and insulin. By projecting the original 19-dimensional space of intracellular signals into a low-dimensional space, Isomap was able to reconstruct clusters corresponding to different cytokine treatments that were identified with graph-based clustering. In comparison, Principal Component Analysis (PCA) and Partial Least Squares – Discriminant analysis (PLS-DA) were unable to find biologically meaningful clusters. We also showed that by using Isomap components for supervised classification with k-nearest neighbor (k-NN) and quadratic discriminant analysis (QDA), apoptosis intensity can be predicted for different combinations of TNF, EGF and insulin. Prediction accuracy was highest when early activation time points in the apoptosis signaling network were used to predict apoptosis rates at later time points. Extended Isomap also outperformed PCA on the AfCS double ligand screen data. Isomap identified more functionally coherent clusters than PCA and captured more information in the first two-components. The Isomap projection performs slightly worse when more signaling networks are analyzed; suggesting that the mapping function between cues and responses becomes increasingly non-linear when large signaling pathways are considered.ConclusionWe developed and applied extended Isomap approach for the analysis of cell signaling networks. Potential biological applications of this method include characterization, visualization and clustering of different treatment conditions (i.e. low and high doses of TNF) in terms of changes in intracellular signaling they induce.