Low-dimensional Embedding Research Articles

Dimensionality reduction for visualizing high-dimensional biological data

High throughput technologies used in experimental biological sciences produce data with a vast number of variables at a rapid pace, making large volumes of high-dimensional data available. The exploratory analysis of such high-dimensional data can be aided by human interpretable low-dimensional visualizations. This work investigates how both discrete and continuous structures in biological data can be captured using the recently proposed dimensionality reduction method SONG, and compares the results with commonly used methods UMAP and PHATE. Using simulated and real-world datasets, we observe that SONG produces insightful visualizations by preserving various patterns, including discrete clusters, continuums, and branching structures in all considered datasets. More importantly, for datasets containing both discrete and continuous structures, SONG performs better at preserving both the structures compared to UMAP and PHATE. Furthermore, our quantitative evaluation of the three methods using downstream analysis validates the on par quality of the SONG’s low-dimensional embeddings compared to the other methods.

Knowledge Graph Entity Alignment Using Relation Structural Similarity

Embedding-based entity alignment, which represents knowledge graphs as low-dimensional embeddings and finds entities in different knowledge graphs that semantically represent the same real-world entity by measuring the similarities between entity embeddings, has achieved promising results. However, existing methods are still challenged by the error accumulation of embeddings along multi-step paths and the semantic information loss. This paper proposes a novel embedding-based entity alignment method that iteratively aligns both entities and relations with high similarities as training data. Newly-aligned entities and relations are used to calibrate the corresponding embeddings in the unified embedding space, which reduces the error accumulation. To reduce the negative impact of semantic information loss, the authors propose to use relation structural similarity instead of embedding similarity to align relations. Experimental results on five widely used real-world datasets show that the proposed method significantly outperforms several state-of-the-art methods for entity alignment.

Consensus graph learning for auto-weighted multi-view projection clustering

Graph-based learning approaches have achieved remarkable success in clustering prevalent multi-view data owing to their capacities to reveal the relation between data and discover its underlying structure. However, real multi-view data is not only simply high-dimensional, but also contains noise and redundant information, so the learned affinity graphs may be unreliable, let alone optimal, and produce inaccurate clustering results. Moreover, existing graph learning based multi-view projection models only learn a common graph or a shared low-dimensional embedding matrix, which fails to preserve the flexible local manifold geometry of each view. To alleviate these problems, a novel consensus graph-based auto-weighted multi-view projection clustering (CGAMPC) is developed, which performs dimensionality reduction, manifold structure preservation and consensus structured graph learning simultaneously. To be specific, the ℓ2,1-norm is leveraged to resist noise and adaptively select discriminative features. Meanwhile, to preserve the manifold structure information of all views, we construct informative similarity graphs for the projection data, and fuse them into a consensus structured graph via an auto-weighted synthesis strategy. Furthermore, an effective alternating iterative algorithm is presented to optimize our CGAMPC. Finally, numerical studies on several multi-view benchmark datasets justify the superiority of the proposed approach over other state-of-the-art clustering approaches.

Entity alignment based on relational semantics augmentation for multilingual knowledge graphs

Seeking similar entity pairs pointing to their counterpart real-world objects in knowledge graphs (KGs) is one of the most challenging and critical steps for entity alignment (EA), also known as KG integration. EA has prompted the development of knowledge-based technologies such as recommender systems, semantic search engines, chatbot systems, and knowledge reasoning. Recent years have witnessed increasing interest in representation learning-based entity alignment methods. Most of these represent different KG entities as low-dimensional vector embeddings via their neighborhood structure and then find counterpart entities by estimating the similarities between entity representations. However, most of the studies on the mainstream methods for entity alignment have paid little attention to abundant information on entity one-hop bidirectional neighbors and relational semantics centrality in KGs. Due to the heterogeneity of the knowledge graph, it is difficult to achieve satisfactory alignment results for entity alignment methods based on structure encoding. This paper investigates multilingual entity alignment strategies and proposes a novel relational semantics augmentation (RSA) model to alleviate these issues. RSA can fuse the entity’s bidirectional neighborhood path information and their connected relational semantics contexts. Comprehensive experiments on five benchmark EA datasets with the comparison of state-of-the-art entity alignment methods demonstrate the better performance of our RSA model.

Automating t-SNE parameterization with prototype-based learning of manifold connectivity

We harness topological information about a data manifold revealed through neural prototype-based learning to automate t-SNE parameterization. This information is contained in the CONN (CONNectivity) similarity of neural prototypes, which grades the strength (weakness) of topological connectivity at various points within a data manifold. CONN suggests a data-driven specification of localized versions (varying across the manifold) of t-SNE’s perplexity parameter which, in turn, defines the high-dimensional similarities P that t-SNE attempts to preserve. We further imbue P with CONN’s graded similarity to reduce mismatch between the topology of the manifold and its embedded representation. Experiments show these improvements, collectively called CONNt-SNE, are capable of producing meaningful and trustworthy low-dimensional embeddings without the need to heuristically optimize over (i.e., grid search) t-SNE’s perplexity space. Data-driven t-SNE parameterization improves our confidence that any structure appearing in the embeddings is valid and not merely an artifact of spurious parameterization.

Identifying multicellular spatiotemporal organization of cells with SpaceFlow

One major challenge in analyzing spatial transcriptomic datasets is to simultaneously incorporate the cell transcriptome similarity and their spatial locations. Here, we introduce SpaceFlow, which generates spatially-consistent low-dimensional embeddings by incorporating both expression similarity and spatial information using spatially regularized deep graph networks. Based on the embedding, we introduce a pseudo-Spatiotemporal Map that integrates the pseudotime concept with spatial locations of the cells to unravel spatiotemporal patterns of cells. By comparing with multiple existing methods on several spatial transcriptomic datasets at both spot and single-cell resolutions, SpaceFlow is shown to produce a robust domain segmentation and identify biologically meaningful spatiotemporal patterns. Applications of SpaceFlow reveal evolving lineage in heart developmental data and tumor-immune interactions in human breast cancer data. Our study provides a flexible deep learning framework to incorporate spatiotemporal information in analyzing spatial transcriptomic data.

Using Global t-SNE to Preserve Intercluster Data Structure.

The t-distributed stochastic neighbor embedding (t-SNE) method is one of the leading techniques for data visualization and clustering. This method finds lower-dimensional embedding of data points while minimizing distortions in distances between neighboring data points. By construction, t-SNE discards information about large-scale structure of the data. We show that adding a global cost function to the t-SNE cost function makes it possible to cluster the data while preserving global intercluster data structure. We test the new global t-SNE (g-SNE) method on one synthetic and two real data sets on flower shapes and human brain cells. We find that significant and meaningful global structure exists in both the plant and human brain data sets. In all cases, g-SNE outperforms t-SNE and UMAP in preserving the global structure. Topological analysis of the clustering result makes it possible to find an appropriate trade-off of data distribution across scales. We find differences in how data are distributed across scales between the two subjects that were part of the human brain data set. Thus, by striving to produce both accurate clustering and positioning between clusters, the g-SNE method can identify new aspects of data organization across scales.

Self-Organizing Maps for Contrastive Embeddings of Sleep Recordings.

Nowadays, high amounts of data can be acquired in various applications, spurring the need for interpretable data representations that provide actionable insights. Algorithms that yield such representations ideally require as little a priori knowledge about the data or corresponding annotations as possible. To this end, we here investigate the use of Kohonen's Self-Organizing Map (SOM) in combination with data-driven low-dimensional embeddings obtained through self-supervised Contrastive Predictive Coding. We compare our approach to embeddings found with an auto-encoder and, moreover, investigate three ways to deal with node selection during SOM optimization. As a challenging experiment we analyze nocturnal sleep recordings of healthy subjects, and conclude that - for this noisy real-life data - contrastive learning yields a better low-dimensional embedding for the purpose of SOM training, compared to an auto-encoder. In addition, we show that a stochastic temperature-annealed SOM-training outperforms both a deterministic and a non-temperature-annealed stochastic approach. Clinical relevance - The hypnogram has for decades been the clinical standard in sleep medicine despite the fact that it is a highly simplified representation of a polysomnography recording. We propose a sensor-agnostic algorithm that is able to reveal more intricate patterns in sleep recordings which might teach us about sleep structure and sleep disorders.

Spatiotemporal patterns of firearm acquisition in the United States in different presidential terms.

This study develops mathematical tools and approaches to investigate spatiotemporal patterns of firearm acquisition in the U.S. complemented by hypothesis testing and statistical analysis. First, state-level and nation-level instant background check (BC) data are employed as proxy of firearm acquisition corresponding to 1999-2021. The relative-phase time-series of BC in each U.S. state is recovered and utilized to calculate the time-series of the U.S. states' synchronization degree. We reveal that U.S. states present a high-level degree of synchronization except in 2010-2011 and after 2018. Comparing these results with respect to a sitting U.S. president provides additional information: specifically, any two presidential terms are characterized by statistically different synchronization degrees except G. W. Bush's first term and B. H. Obama's second term. Next, to detail variations of BC, short-time Fourier transform, dimensionality reduction techniques, and diffusion maps are implemented within a time-frequency representation. Firearm acquisition in the high frequency band is described by a low-dimensional embedding, in the form of a plane with two embedding coordinates. Data points on the embedding plane identify separate clusters that signify state transitions in the original BC data with respect to different time windows. Through this analysis, we reveal that the frequency content of the BC data has a time-dependent characteristic. By comparing the diffusion map at hand with respect to a presidential term, we find that at least one of the embedding coordinates presents statistically significant variations between any two presidential terms except B. H. Obama's first term and D. J. Trump's pre-COVID term. The results point at a possible interplay between firearm acquisition in the U.S. and a presidential term.

Topological data analysis of truncated contagion maps.

The investigation of dynamical processes on networks has been one focus for the study of contagion processes. It has been demonstrated that contagions can be used to obtain information about the embedding of nodes in a Euclidean space. Specifically, one can use the activation times of threshold contagions to construct contagion maps as a manifold-learning approach. One drawback of contagion maps is their high computational cost. Here, we demonstrate that a truncation of the threshold contagions may considerably speed up the construction of contagion maps. Finally, we show that contagion maps may be used to find an insightful low-dimensional embedding for single-cell RNA-sequencing data in the form of cell-similarity networks and so reveal biological manifolds. Overall, our work makes the use of contagion maps as manifold-learning approaches on empirical network data more viable.

A Survey on High-dimensional Gaussian Process Modeling with Application to Bayesian Optimization

Bayesian Optimization (BO), the application of Bayesian function approximation to finding optima of expensive functions, has exploded in popularity in recent years. In particular, much attention has been paid to improving its efficiency on problems with many parameters to optimize. This attention has trickled down to the workhorse of high-dimensional BO, high-dimensional Gaussian process regression, which is also of independent interest. The great flexibility that the Gaussian process prior implies is a boon when modeling complicated, low-dimensional surfaces but simply says too little when dimension grows too large. A variety of structural model assumptions have been tested to tame high dimensions, from variable selection and additive decomposition to low-dimensional embeddings and beyond. Most of these approaches in turn require modifications of the acquisition function optimization strategy as well. Here, we review the defining structural model assumptions and discuss the benefits and drawbacks of these approaches in practice.

Simple Unsupervised Graph Representation Learning

In this paper, we propose a simple unsupervised graph representation learning method to conduct effective and efficient contrastive learning. Specifically, the proposed multiplet loss explores the complementary information between the structural information and neighbor information to enlarge the inter-class variation, as well as adds an upper bound loss to achieve the finite distance between positive embeddings and anchor embeddings for reducing the intra-class variation. As a result, both enlarging inter-class variation and reducing intra-class variation result in small generalization error, thereby obtaining an effective model. Furthermore, our method removes widely used data augmentation and discriminator from previous graph contrastive learning methods, meanwhile available to output low-dimensional embeddings, leading to an efficient model. Experimental results on various real-world datasets demonstrate the effectiveness and efficiency of our method, compared to state-of-the-art methods. The source codes are released at https://github.com/YujieMo/SUGRL.

DURIAN: an integrative deconvolution and imputation method for robust signaling analysis of single-cell transcriptomics data.

Single-cell RNA sequencing trades read-depth for dimensionality, often leading to loss of critical signaling gene information that is typically present in bulk data sets. We introduce DURIAN (Deconvolution and mUltitask-Regression-based ImputAtioN), an integrative method for recovery of gene expression in single-cell data. Through systematic benchmarking, we demonstrate the accuracy, robustness and empirical convergence of DURIAN using both synthetic and published data sets. We show that use of DURIAN improves single-cell clustering, low-dimensional embedding, and recovery of intercellular signaling networks. Our study resolves several inconsistent results of cell-cell communication analysis using single-cell or bulk data independently. The method has broad application in biomarker discovery and cell signaling analysis using single-cell transcriptomics data sets.

A Novel Bus Arrival Time Prediction Method Based on Spatio-Temporal Flow Centrality Analysis and Deep Learning

This paper presents a method for predicting bus stop arrival times based on a unique approach that extracts the spatio-temporal dynamics of bus flows. Using a new technique called Bus Flow Centrality Analysis (BFC), we obtain the low-dimensional embedding of short-term bus flow patterns in the form of IID (Individual In Degree) and IOD (Individual Out Degree) and TOD (Total Out Degree) at every station in the bus network. The embedding using BFC analysis well captures the characteristics of every individual flow and aggregate pattern. The latent vector returned by the BFC analysis is combined with other essential information such as bus speed, travel time, wait time, dispatch intervals, the distance between stations, seasonality, holiday status, and climate information. We employed a family of recurrent neural networks such as LSTM, GRU, and ALSTM to model how these features change over time and to predict the time the bus takes to reach the next stop in subsequent time windows. We experimented with our solution using logs of bus operations in the Seoul Metropolitan area offered by the Bus Management System (BMS) and the Bus Information System (BIS) of Korea. We predicted arrival times for more than 100 bus routes with a MAPE of 1.19%. This margin of error is 74% lower than the latest work based on ALSTM. We also learned that LSTM performs better than GRU with a 40.5% lower MAPE. This result is even remarkable considering the irregularity in the bus flow patterns and the fact that we did not rely on real-time GPS information. Moreover, our approach scales at a city-wide level by analyzing more than 100 bus routes, while previous studies showed limited experiments on much fewer bus routes.

Feature extraction and fault detection scheme via improved locality preserving projection and SVDD

Manifold learning is widely adopted for the fault detection of industrial processes. However, the quality of low-dimensional embedding coordinates can be adversely affected by ill-constructed graph Laplacian. An improved locality preserving projection (ILPP) scheme is proposed. ILPP is built on a geometrically inspired Laplacian, and the Riemannian metric is used to find the suitable bandwidth parameter. The proposed approach combines the advantages of ILPP in preserving manifold data structures and those of support vector data description (SVDD) in handling complex process data distributions. Case studies on helix data, hot strip mill, and Pensim benchmark processes demonstrate the utility and feasibility of the proposed approach. The average fault detection rate for proposed ILPP is 99%, which is higher than locality preserving projection (LPP; 87.8%), local tangent space alignment (LTSA; 74.9%), and principal component analysis (PCA; 90.6%).

DeepForge for astronomy: Deep learning SDSS redshifts from images

Dedicated survey telescopes provide multicolor imaging observations every night, whose scientific impact hinges on the subsequent data analysis. Deep learning has recently provided breakthroughs in many areas of astronomy due to its ability to extract optimal features from the input data during training. Photometric redshift estimation in particular seems like a perfect application. Directly analyzing the images instead of traditional summary catalogs holds the promise of naturally combining spectral and morphological information. Here we introduce a complete solution called DeepForge for creating, training, and running deep networks for astronomical analyses using a convenient web portal and Python. Following the approach by Pasquet et al. (2019) we demonstrate unprecedented accuracy on the Sloan Digital Sky Survey measurements. We study variants of their network and analyze the optimal feature space using low-dimensional embedding, which clearly reveal the power of the custom features. With DeepForge the process of experimentation is straightforward and has the potential to further improve the current state of the art in photometric redshift estimation.

ScGAD: single-cell gene associating domain scores for exploratory analysis of scHi-C data.

Quantitative tools are needed to leverage the unprecedented resolution of single-cell high-throughput chromatin conformation (scHi-C) data and integrate it with other single-cell data modalities. We present single-cell gene associating domain (scGAD) scores as a dimension reduction and exploratory analysis tool for scHi-C data. scGAD enables summarization at the gene unit while accounting for inherent gene-level genomic biases. Low-dimensional projections with scGAD capture clustering of cells based on their 3D structures. Significant chromatin interactions within and between cell types can be identified with scGAD. We further show that scGAD facilitates the integration of scHi-C data with other single-cell data modalities by enabling its projection onto reference low-dimensional embeddings. This multi-modal data integration provides an automated and refined cell-type annotation for scHi-C data. scGAD is part of the BandNorm R package at https://sshen82.github.io/BandNorm/articles/scGAD-tutorial.html. Supplementary data are available at Bioinformatics online.

A Bayesian model of dose-response for cancer drug studies

Exploratory cancer drug studies test multiple tumor cell lines against multiple candidate drugs. The goal in each paired (cell line, drug) experiment is to map out the dose-response curve of the cell line as the dose level of the drug increases. We propose Bayesian tensor filtering (BTF), a hierarchical Bayesian model for dose-response modeling in multisample, multitreatment cancer drug studies. BTF uses low-dimensional embeddings to share statistical strength between similar drugs and similar cell lines. Structured shrinkage priors in BTF encourage smoothness in the dose-response curves while remaining adaptive to sharp jumps when the data call for it. We focus on a pair of cancer drug studies exhibiting a particular pathology in their experimental design, leading us to a nonconjugate monotone mixture-of-gammas likelihood. To perform posterior inference, we develop a variant of the elliptical slice sampling algorithm for sampling from linearly-constrained multivariate normal priors with nonconjugate likelihoods. In benchmarks, BTF outperforms state-of-the-art methods for covariance regression and dynamic Poisson matrix factorization. On the two cancer drug studies, BTF outperforms the current standard approach in biology and reveals potential new biomarkers of drug sensitivity in cancer. Code is available at https://github.com/tansey/functionalmf.

Discovery of Slow Variables in a Class Of Multiscale Stochastic Systems Via Neural Networks

Finding a reduction of complex, high-dimensional dynamics to its essential, low-dimensional “heart” remains a challenging yet necessary prerequisite for designing efficient numerical approaches. Machine learning methods have the potential to provide a general framework to automatically discover such representations. In this paper, we consider multiscale stochastic systems with local slow-fast timescale separation and propose a new method to encode in an artificial neural network a map that extracts the slow representation from the system. The architecture of the network consists of an encoder–decoder pair that we train in a supervised manner to learn the appropriate low-dimensional embedding in the bottleneck layer. We test the method on a number of examples that illustrate the ability to discover a correct slow representation. Moreover, we provide an error measure to assess the quality of the embedding and demonstrate that pruning the network can pinpoint essential coordinates of the system to build the slow representation.

GLIDER: function prediction from GLIDE-based neighborhoods.

MotivationProtein function prediction, based on the patterns of connection in a protein–protein interaction (or association) network, is perhaps the most studied of the classical, fundamental inference problems for biological networks. A highly successful set of recent approaches use random walk-based low-dimensional embeddings that tend to place functionally similar proteins into coherent spatial regions. However, these approaches lose valuable local graph structure from the network when considering only the embedding. We introduce GLIDER, a method that replaces a protein–protein interaction or association network with a new graph-based similarity network. GLIDER is based on a variant of our previous GLIDE method, which was designed to predict missing links in protein–protein association networks, capturing implicit local and global (i.e. embedding-based) graph properties.ResultsGLIDER outperforms competing methods on the task of predicting GO functional labels in cross-validation on a heterogeneous collection of four human protein–protein association networks derived from the 2016 DREAM Disease Module Identification Challenge, and also on three different protein–protein association networks built from the STRING database. We show that this is due to the strong functional enrichment that is present in the local GLIDER neighborhood in multiple different types of protein–protein association networks. Furthermore, we introduce the GLIDER graph neighborhood as a way for biologists to visualize the local neighborhood of a disease gene. As an application, we look at the local GLIDER neighborhoods of a set of known Parkinson’s Disease GWAS genes, rediscover many genes which have known involvement in Parkinson’s disease pathways, plus suggest some new genes to study.Availability and implementationAll code is publicly available and can be accessed here: https://github.com/kap-devkota/GLIDER.Supplementary information Supplementary data are available at Bioinformatics online.