Manifold Embedding Research Articles

Capturing discrete latent structures: choose LDs over PCs.

High-dimensional biological data collection across heterogeneous groups of samples has become increasingly common, creating high demand for dimensionality reduction techniques that capture underlying structure of the data. Discovering low-dimensional embeddings that describe the separation of any underlying discrete latent structure in data is an important motivation for applying these techniques since these latent classes can represent important sources of unwanted variability, such as batch effects, or interesting sources of signal such as unknown cell types. The features that define this discrete latent structure are often hard to identify in high-dimensional data. Principal component analysis (PCA) is one of the most widely used methods as an unsupervised step for dimensionality reduction. This reduction technique finds linear transformations of the data which explain total variance. When the goal is detecting discrete structure, PCA is applied with the assumption that classes will be separated in directions of maximum variance. However, PCA will fail to accurately find discrete latent structure if this assumption does not hold. Visualization techniques, such as t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP), attempt to mitigate these problems with PCA by creating a low-dimensional space where similar objects are modeled by nearby points in the low-dimensional embedding and dissimilar objects are modeled by distant points with high probability. However, since t-SNE and UMAP are computationally expensive, often a PCA reduction is done before applying them which makes it sensitive to PCAs downfalls. Also, tSNE is limited to only two or three dimensions as a visualization tool, which may not be adequate for retaining discriminatory information. The linear transformations of PCA are preferable to non-linear transformations provided by methods like t-SNE and UMAP for interpretable feature weights. Here, we propose iterative discriminant analysis (iDA), a dimensionality reduction technique designed to mitigate these limitations. iDA produces an embedding that carries discriminatory information which optimally separates latent clusters using linear transformations that permit post hoc analysis to determine features that define these latent structures.

Extracting the Main Trend in a Data Set: The Sequencer Algorithm

Abstract Scientists aim to extract simplicity from observations of the complex world. An important component of this process is the exploration of data in search of trends. In practice, however, this tends to be more of an art than a science. Among all trends existing in the natural world, one-dimensional trends, often called sequences, are of particular interest, as they provide insights into simple phenomena. However, some are challenging to detect, as they may be expressed in complex manners. We present the Sequencer, an algorithm designed to generically identify the main trend in a data set. It does so by constructing graphs describing the similarities between pairs of observations, computed with a set of metrics and scales. Using the fact that continuous trends lead to more elongated graphs, the algorithm can identify which aspects of the data are relevant in establishing a global sequence. Such an approach can be used beyond the proposed algorithm and can optimize the parameters of any dimensionality reduction technique. We demonstrate the power of the Sequencer using real-world data from astronomy, geology, and images from the natural world. We show that, in a number of cases, it outperforms the popular t-Distributed Stochastic Neighbor Embedding and Uniform Manifold Approximation and Projection dimensionality reduction techniques. This approach to exploratory data analysis, which does not rely on training or tuning any parameter, has the potential to enable discoveries in a wide range of scientific domains. The source code is available on GitHub, and we provide an online interface at http://sequencer.org.

Computing the Riemannian curvature of image patch and single-cell RNA sequencing data manifolds using extrinsic differential geometry

Most high-dimensional datasets are thought to be inherently low-dimensional-that is, data points are constrained to lie on a low-dimensional manifold embedded in a high-dimensional ambient space. Here, we study the viability of two approaches from differential geometry to estimate the Riemannian curvature of these low-dimensional manifolds. The intrinsic approach relates curvature to the Laplace-Beltrami operator using the heat-trace expansion and is agnostic to how a manifold is embedded in a high-dimensional space. The extrinsic approach relates the ambient coordinates of a manifold's embedding to its curvature using the Second Fundamental Form and the Gauss-Codazzi equation. We found that the intrinsic approach fails to accurately estimate the curvature of even a two-dimensional constant-curvature manifold, whereas the extrinsic approach was able to handle more complex toy models, even when confounded by practical constraints like small sample sizes and measurement noise. To test the applicability of the extrinsic approach to real-world data, we computed the curvature of a well-studied manifold of image patches and recapitulated its topological classification as a Klein bottle. Lastly, we applied the extrinsic approach to study single-cell transcriptomic sequencing (scRNAseq) datasets of blood, gastrulation, and brain cells to quantify the Riemannian curvature of scRNAseq manifolds.

Dimensionality reduction by UMAP reinforces sample heterogeneity analysis in bulk transcriptomic data.

Transcriptomic analysis plays a key role in biomedical research. Linear dimensionality reduction methods, especially principal-component analysis (PCA), are widely used in detecting sample-to-sample heterogeneity, while recently developed non-linear methods, such as t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP), can efficiently cluster heterogeneous samples in single-cell RNA sequencing analysis. Yet, the application of t-SNE and UMAP in bulk transcriptomic analysis and comparison with conventional methods have not been achieved. We compare four major dimensionality reduction methods (PCA, multidimensional scaling [MDS], t-SNE, and UMAP) in analyzing 71 large bulk transcriptomic datasets. UMAP is superior to PCA and MDS but shows some advantages over t-SNE in differentiating batch effects, identifying pre-defined biological groups, and revealing in-depth clusters in two-dimensional space. Importantly, UMAP generates sample clusters uncovering biological features and clinical meaning. We recommend deploying UMAP in visualizing and analyzing sizable bulk transcriptomic datasets to reinforce sample heterogeneity analysis.

Semi‐supervised Robust Feature Selection with ℓq‐Norm Graph for Multiclass Classification

Flexible manifold embedding (FME) is a semi-supervised dimension reduction framework. It has been extended into feature selection by using different loss functions and sparse regularization methods. However, these kind of methods used the quadratic form of graph embedding, thus the results are sensitive to noise and outliers. In this paper, we propose a general semi-supervised feature selection model that optimizes an ℓ q -norm of FME to decrease the noise sensitivity. Compare to the fixed parameter model, the ℓ q -norm graph brings flexibility to balance the manifold smoothness and the sensitivity to noise by tuning its parameter. We present an efficient iterative algorithm to solve the proposed ℓ q -norm graph embedding based semi-supervised feature selection problem, and offer a rigorous convergence analysis. Experiments performed on typical image and speech emotion datasets demonstrate that our method is effective for the multiclass classification task, and outperforms the related state-of-the-art methods.

Quantile–Quantile Embedding for distribution transformation and manifold embedding with ability to choose the embedding distribution

We propose a new embedding method, named Quantile–Quantile Embedding (QQE), for distribution transformation and manifold embedding with the ability to choose the embedding distribution. QQE, which uses the concept of quantile–quantile plot from visual statistical tests, can transform the distribution of data to any theoretical desired distribution or empirical reference sample. Moreover, QQE gives the user a choice of embedding distribution in embedding the manifold of data into the low dimensional embedding space. It can also be used for modifying the embedding distribution of other dimensionality reduction methods, such as PCA, t-SNE, and deep metric learning, for better representation or visualization of data. We propose QQE in both unsupervised and supervised forms. QQE can also transform a distribution to either an exact reference distribution or its shape. We show that QQE allows for better discrimination of classes in some cases. Our experiments on different synthetic and image datasets show the effectiveness of the proposed embedding method.

ReliefE: feature ranking in high-dimensional spaces via manifold embeddings

Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embedding-based methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding’s dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.

Constraint Preserving Score for Automatic Hyperparameter Tuning of Dimensionality Reduction Methods for Visualization

In data analysis, visualization through dimensionality reduction (DR) is one of the most effective ways to understand a dataset. However, the quality of a visualization is hard to evaluate quantitatively and the hyperparameters of visualization algorithms are sometimes difficult to tune for end-users. This article proposes a score for visualization assessment that can be used to ease the choice of hyperparameter values for widely used DR methods like <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$t$</tex-math></inline-formula> -distributed stochastic neighbor embedding, LargeVis, and uniform manifold approximation and projection. We present the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">constraint preserving score</i> , a computationally efficient score to measure visualization quality. The idea is to measure how well a visualization preserves the information encoded in pairwise constraints like group information or similarity/dissimilarity relationships between instances. Based on this quantitative measure, we use Bayesian optimization to effectively explore the solution space of all visualizations and find the most suitable one. The proposed score is flexible as it can measure quality in different ways depending on the provided constraints. Experiments show its interest for end-users, its complementarity with existing visualization quality measures, and its flexibility to easily express different quality aspects. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>Impact Statement</i>—When working with high-dimensional data, visualization techniques are useful tools to help us to understand patterns in data. Widely used visualization methods such as <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula>-distributed stochastic neighbor embedding, LargeVis, and uniform manifold approximation and projection require tuning several hyperparameters, which is a tedious task for end-users. The visualizations are usually assessed qualitatively and subjectively by users since we lack quantitative measures that fit their needs. Our work tackles this problem by proposing a novel score based on user's constraints to measure visualization quality. This score can, thus, be used to automatically tune the hyperparameters of visualization methods. For real-world datasets, there are typically multiple aspects hidden in the data under the form of local or global structures, or relationships between data groups. One visualization gives us one vantage point to look at the data and, thus, reveals one specific aspect of the data. Assessing the visualization quality is still an open question and each state-of-the-art visualization quality metric is designed to capture only one specific aspect like local neighborhood structure. However, our proposed constraints preserving score can capture other different aspects of the visualization like the global structure or semantic relationships between groups according to the information encoded in the input constraints. Our score measures how well the information encoded in input constraints is preserved in a visualization, and suggests the best visualization corresponding to the users’ needs. This score can have a large impact since it is very easy to use and works with any visualization method. Domain experts can express their knowledge in a simple form of similar or dissimilar groups of points. If needed, end-users can use a small amount of labeled data to express their constraints.

Moving frames and compatibility conditions for three-dimensional director fields

The geometry and topology of the region in which a director field is embedded impose limitations on the kind of supported orientational order. These limitations manifest as compatibility conditions that relate the quantities describing the director field to the geometry of the embedding space. For example, in two dimensions the splay and bend fields suffice to determine a director uniquely (up to rigid motions) and must comply with one relation linear in the Gaussian curvature of the embedding manifold. In 3D there are additional local fields describing the director, i.e. fields available to a local observer residing within the material, and a number of distinct ways to yield geometric frustration. So far it was unknown how many such local fields are required to uniquely describe a 3D director field, nor what are the compatibility relations they must satisfy. In this work, we address these questions directly. We employ the method of moving frames to show that a director field is fully determined by five local fields. These fields are shown to be related to each other and to the curvature of the embedding space through six differential relations. As an application of our method, we characterize all uniform distortion director fields, i.e., directors for which all the local characterizing fields are constant in space, in manifolds of constant curvature. The classification of such phases has been recently provided for directors in Euclidean space, where the textures correspond to foliations of space by parallel congruent helices. For non-vanishing curvature, we show that the pure twist phase is the only solution in positively curved space, while in the hyperbolic space uniform distortion fields correspond to foliations of space by (non-necessarily parallel) congruent helices. Further analysis of the obtained compatibility fields is expected to allow to also construct new non-uniform director fields.

Generalized vanishing theorems for Yukawa couplings in heterotic compactifications

Heterotic compactifications on Calabi-Yau threefolds frequently exhibit textures of vanishing Yukawa couplings in their low energy description. The vanishing of these couplings is often not enforced by any obvious symmetry and appears to be topological in nature. Recent results used differential geometric methods to explain the origin of some of this structure [1, 2]. A vanishing theorem was given which showed that the effect could be attributed, in part, to the embedding of the Calabi-Yau manifolds of interest inside higher dimensional ambient spaces, if the gauge bundles involved descended from vector bundles on those larger manifolds. In this paper, we utilize an algebro-geometric approach to provide an alternative derivation of some of these results, and are thus able to generalize them to a much wider arena than has been considered before. For example, we consider cases where the vector bundles of interest do not descend from bundles on the ambient space. In such a manner we are able to highlight the ubiquity with which textures of vanishing Yukawa couplings can be expected to arise in heterotic compactifications, with multiple different constraints arising from a plethora of different geometric features associated to the gauge bundle.

Deep Manifold Embedding for Hyperspectral Image Classification.

Deep learning methods have played a more important role in hyperspectral image classification. However, general deep learning methods mainly take advantage of the samplewise information to formulate the training loss while ignoring the intrinsic data structure of each class. Due to the high spectral dimension and great redundancy between different spectral channels in the hyperspectral image, these former training losses usually cannot work so well for the deep representation of the image. To tackle this problem, this work develops a novel deep manifold embedding method (DMEM) for deep learning in hyperspectral image classification. First, each class in the image is modeled as a specific nonlinear manifold, and the geodesic distance is used to measure the correlation between the samples. Then, based on the hierarchical clustering, the manifold structure of the data can be captured and each nonlinear data manifold can be divided into several subclasses. Finally, considering the distribution of each subclass and the correlation between different subclasses under data manifold, DMEM is constructed as the novel training loss to incorporate the special classwise information in the training process and obtain discriminative representation for the hyperspectral image. Experiments over four real-world hyperspectral image datasets have demonstrated the effectiveness of the proposed method when compared with general sample-based losses and showed superiority when compared with state-of-the-art methods.

Geodesic Multi-Class SVM with Stiefel Manifold Embedding.

Manifold of geodesic plays an essential role in characterizing the intrinsic data geometry. However, the existing SVM methods have largely neglected the manifold structure. As such, functional degeneration may occur due to the potential polluted training. Even worse, the entire SVM model might collapse in the presence of excessive training contamination. To address these issues, this paper devises a manifold SVM method based on a novel ξ -measure geodesic, whose primary design objective is to extract and preserve the data manifold structure in the presence of training noises. To further cope with overly contaminated training data, we introduce Kullback-Leibler (KL) regularization with steerable sparsity constraint. In this way, each loss weight is adaptively obtained by obeying the prior distribution and sparse activation during model training for robust fitting. Moreover, the optimal scale for Stiefel manifold can be automatically learned to improve the model flexibility. Accordingly, extensive experiments verify and validate the superiority of the proposed method.

Accelerated manifold embedding for multi-view semi-supervised classification

Multi-view semi-supervised learning has gained much attention since a great number of unlabeled multi-view data are easy to obtain while few labeled data are available. Accordingly, how to utilize the relationship between labeled and unlabeled data is of significance in multi-view semi-supervised learning. In this paper, we propose an auto-weighted manifold embedding model to address multi-view semi-supervised classification problems, where only a small percentage of labeled data points are used for model training. In the proposed model, data points close to each other in the feature space will be assigned to similar classes in the label space through manifold embedding. Accordingly, the class information of labeled data will be employed in the prediction process for unlabeled data. Moreover, an optimal weight for each view of multi-view data is learned automatically, which enables an encouraging fusion quality for multi-view manifold embedding. To further speed up the calculation process and reduce the computational complexity of the proposed model, an effective accelerated auto-weighted manifold embedding scheme is developed. Besides, theoretical analyses are then provided to indicate a tight approximation bound for the primary manifold embedding method, while the accelerated method is designed by an order decrease of magnitude of computational complexity. Finally, comprehensive experiments on eight publicly available data sets demonstrate the superiority of the proposed models over compared state-of-the-art semi-supervised methods and fully supervised classifiers. Furthermore, the experimental results are suggestive of positive robustness and promising generalization capacity of the proposed methods.

Holomorphic maps into Grassmann manifolds (harmonic maps into Grassmann manifolds III)

A well-known Calabi’s rigidity theorem on holomorphic isometric immersions into the complex projective space is generalized to the case that the target is the complex Grassmann manifolds. Our strategy is to use the differential geometry of vector bundles and a generalization of do Carmo and Wallach theory developed in Nagatomo (Harmonic maps into Grassmann manifolds. arXiv:mathDG/1408.1504 ). We introduce the associated maps with holomorphic maps to obtain a general rigidity theorem (Theorem 5.6). As applications, several rigidity results on Einstein–Hermitian holomorphic maps are exhibited and we also give an interpretation of the existence of a Kahler structure with an $$S^1$$ -action on the moduli spaces of holomorphic isometric embeddings of a compact Kahler manifold into complex quadrics. Theorem 5.6 also implies classification theorems for equivariant holomorphic maps.

Connected Fundamental Groups and Homotopy Contacts in Fibered Topological (C, R) Space

The algebraic as well as geometric topological constructions of manifold embeddings and homotopy offer interesting insights about spaces and symmetry. This paper proposes the construction of 2-quasinormed variants of locally dense p-normed 2-spheres within a non-uniformly scalable quasinormed topological (C, R) space. The fibered space is dense and the 2-spheres are equivalent to the category of 3-dimensional manifolds or three-manifolds with simply connected boundary surfaces. However, the disjoint and proper embeddings of covering three-manifolds within the convex subspaces generates separations of p-normed 2-spheres. The 2-quasinormed variants of p-normed 2-spheres are compact and path-connected varieties within the dense space. The path-connection is further extended by introducing the concept of bi-connectedness, preserving Urysohn separation of closed subspaces. The local fundamental groups are constructed from the discrete variety of path-homotopies, which are interior to the respective 2-spheres. The simple connected boundaries of p-normed 2-spheres generate finite and countable sets of homotopy contacts of the fundamental groups. Interestingly, a compact fibre can prepare a homotopy loop in the fundamental group within the fibered topological (C, R) space. It is shown that the holomorphic condition is a requirement in the topological (C, R) space to preserve a convex path-component. However, the topological projections of p-normed 2-spheres on the disjoint holomorphic complex subspaces retain the path-connection property irrespective of the projective points on real subspace. The local fundamental groups of discrete-loop variety support the formation of a homotopically Hausdorff (C, R) space.

Discriminant Tensor-Based Manifold Embedding for Medical Hyperspectral Imagery.

Medical hyperspectral imagery has recentlyattracted considerable attention. However, for identification tasks, the high dimensionality of hyperspectral images usually leads to poor performance. Thus, dimensionality reduction (DR) is crucial in hyperspectral image analysis. Motivated by exploiting the underlying structure information of medical hyperspectral images and enhancing the discriminant ability of features, a discriminant tensor-based manifold embedding (DTME) is proposed for discriminant analysis of medical hyperspectral images. Based on the idea of manifold learning, a new discriminant similarity metric is designed, which takes into account the tensor representation, sparsity, low-rank and distribution characteristics. Then, an inter-class tensor graph and an intra-class tensor graph are constructed using the new similarity metric to reveal intrinsic manifold of hyperspectral data. Dimensionality reduction is achieved by embedding this supervised tensor graphs into the low-dimensional tensor subspace. Experimental results on membranous nephropathy and white bloodcells identification tasks demonstrate the potential clinical value of the proposed DTME.

On compact subsets of Sobolev spaces on manifolds

The paper considers compactness of Sobolev embeddings of non-compact manifolds, restricted to subsets (typically subspaces) defined either by conditions of symmetry (or quasisymmetry) relative to actions of compact groups, or by restriction in the number of variables, i.e. consisting of functions of the form f ∘ φ f\circ \varphi with a fixed φ \varphi . The manifolds are assumed to satisfy general common conditions under which Sobolev embeddings exist. We provide sufficient conditions for compactness of the embeddings, which in many situations are also necessary.

Geometric machine learning algorithm helps sieve through structural data of tiny biological objects

By fusing manifold embedding with machine learning, the researchers create an algorithm that can pick out X-ray diffraction patterns of single particles from an ocean of data.

Hand motion recognition via multi-kernel manifold learning

In the context of smart city, hand motion recognition has once again become a research hotspot, especially the surface electromyographic (sEMG) based hand motion recognition. When sEMG signals contain noise, the overfitting problem always exists. To solve this problem, a multi-kernel label relaxation ridge regression with manifold embedding is proposed. Firstly, a non-negative matrix is introduced to relax the label matrix of the training data, so that the proposed model has more freedom in label fitting. At the same time, label relaxation can enlarge the margins between classes as much as possible. Secondly, based on the idea of manifold learning, the compactness graph is introduced to ensure that the training data in the same class should be as close as possible in the conversion space when they are converted from the original feature space to the label space so as to improve the generalization ability. Finally, 15 UCI datasets and the hand motion dataset Ninapro are used to verify the proposed model. The experimental results show that the proposed model has certain advantages over comparative ones in terms of classification accuracy and generalization ability.

Sparse non-negative transition subspace learning for image classification

Recently many variations of least squares regression (LSR) have been developed to address the problem of over-fitting that widely exists in the task of image classification. Among these methods, the most prevalent two means, such as label relaxation and graph manifold embedding, have been demonstrated to be highly effective. In this paper, we present a new strategy named sparse non-negative transition subspace learning (SN-TSL) based least squares regression algorithm which aims to avoid over-fitting by learning a transition subspace between the multifarious high-dimensional inputs and low-dimensional binary labels. Moreover, considering the final regression targets are sparse binary positive matrices, we use the l1-norm and the non-negativity constraint to enforce the transition subspace to be sparse and non-negative. The resulting subspace features can be viewed as intermediate representations between the inputs and labels. Because SN-TSL can simultaneously learn two projection matrices in one regression model and the dimensionality of the transition subspace can be set to any integer, SN-TSL has the potential to obtain more distinct projections for classification. It is also suitable for classification problems involving a small number of classes. Extensive experiments on public datasets have shown the proposed SN-TSL outperforms other state-of-the-art LSR based image classification methods.