High-dimensional Landscapes Research Articles

Multimodal Learning for Mapping the Genotype-Phenotype Dynamics.

How complex phenotypes emerge from intricate gene expression patterns is a fundamental question in biology. Quantitative characterization of this relationship, however, is challenging due to the vast combinatorial possibilities and dynamic interplay between genotype and phenotype landscapes. Integrating high-content genotyping approaches such as single-cell RNA sequencing and advanced learning methods such as language models offers an opportunity for dissecting this complex relationship. Here, we present a computational integrated genetics framework designed to analyze and interpret the high-dimensional landscape of genotypes and their associated phenotypes simultaneously. We applied this approach to develop a multimodal foundation model to explore the genotype-phenotype relationship manifold for human transcriptomics at the cellular level. Analyzing this joint manifold showed a refined resolution of cellular heterogeneity, enhanced precision in phenotype annotating, and uncovered potential cross-tissue biomarkers that are undetectable through conventional gene expression analysis alone. Moreover, our results revealed that the gene networks are characterized by scale-free patterns and show context-dependent gene-gene interactions, both of which result in significant variations in the topology of the gene network, particularly evident during aging. Finally, utilizing contextualized embeddings, we investigated gene polyfunctionality which illustrates the multifaceted roles that genes play in different biological processes, and demonstrated that for VWF gene in endothelial cells. Overall, this study advances our understanding of the dynamic interplay between gene expression and phenotypic manifestation and demonstrates the potential of integrated genetics in uncovering new dimensions of cellular function and complexity.

Comparative phenomics: a new approach to study heterochrony.

Understanding the links between development and evolution is one of the major challenges of biology. 'Heterochronies', evolutionary alterations in the timings of development are posited as a key mechanism of evolutionary change, but their quantification requires gross simplification of organismal development. Consequently, how changes in event timings influence development more broadly is poorly understood. Here, we measure organismal development as spectra of energy in pixel values of video, creating high-dimensional landscapes integrating development of all visible form and function. This approach we termed 'Energy proxy traits' (EPTs) is applied alongside previously identified heterochronies in three freshwater pulmonate molluscs (Lymnaea stagnalis, Radix balthica and Physella acuta). EPTs were calculated from time-lapse video of embryonic development to construct a continuous functional time series. High-dimensional transitions in phenotype aligned with major sequence heterochronies between species. Furthermore, differences in event timings between conspecifics were associated with changes in high-dimensional phenotypic space. We reveal EPTs as a powerful approach to considering the evolutionary importance of alterations to developmental event timings. Reimagining the phenotype as energy spectra enabled continuous quantification of developmental changes in high-dimensional phenotypic space, rather than measurement of timings of discrete events. This approach has the possibility to transform how we study heterochrony and development more generally.

Exploring canyons in glassy energy landscapes using metadynamics

The complex physics of glass-forming systems is controlled by the structure of the low-energy portions of their potential energy landscapes. Here we report that a modified metadynamics algorithm efficiently explores and samples low-energy regions of such high-dimensional landscapes. In the energy landscape for a model foam, our algorithm finds and descends meandering canyons in the landscape, which contain dense clusters of energy minima along their floors. Similar canyon structures in the energy landscapes of two model glass formers-hard sphere fluids and the Kob-Andersen glass-allow us to reach high densities and low energies, respectively. In the hard sphere system, fluid configurations are found to form continuous regions that cover the canyon floors up to densities well above the jamming transition. For the Kob-Andersen glass former, our technique samples low-energy states with modest computational effort, with the lowest energies found approaching the predicted Kauzmann limit.

Interconnected high-dimensional landscapes of epithelial-mesenchymal plasticity and stemness in cancer.

Establishing macrometastases at distant organs is a highly challenging process for cancer cells, with extremely high attrition rates. A very small percentage of disseminated cells have the ability to dynamically adapt to their changing micro-environments through reversibly switching to another phenotype, aiding metastasis. Such plasticity can be exhibited along one or more axes-epithelial-mesenchymal plasticity (EMP) and cancer stem cells (CSCs) being the two most studied, and often tacitly assumed to be synonymous. Here, we review the emerging concepts related to EMP and CSCs across multiple cancers. Both processes are multi-dimensional in nature; for instance, EMP can be defined on morphological, molecular and functional changes, which may or may not be synchronized. Similarly, self-renewal, multi-lineage potential, and resistance to anoikis and/or therapy may not all occur simultaneously in CSCs. Thus, understanding the complexity in defining EMP and CSCs is essential if we are to understand their contribution to cancer metastasis. This will require a more comprehensive understanding of the non-linearity of these processes. These processes are dynamic, reversible, and semi-independent in nature; cells traverse the inter-connected high-dimensional EMP and CSC landscapes in diverse paths, each of which may exhibit a distinct EMP-CSC coupling. Our proposed model offers a potential unifying framework for elucidating the coupled decision-making along these dimensions and highlights a key set of open questions to be answered.

Spherical Distributions Used in Evolutionary Algorithms

Performance of evolutionary algorithms in real space is evaluated by local measures such as success probability and expected progress. In high-dimensional landscapes, most algorithms rely on the normal multi-variate, easy to assemble from independent, identically distributed components. This paper analyzes a different distribution, also spherical, yet with dependent components and compact support: uniform in the sphere. Under a simple setting of the parameters, two algorithms are compared on a quadratic fitness function. The success probability and the expected progress of the algorithm with uniform distribution are proved to dominate their normal mutation counterparts by order n!!.

SGOOP-d: Estimating Kinetic Distances and Reaction Coordinate Dimensionality for Rare Event Systems from Biased/Unbiased Simulations.

Understanding kinetics including reaction pathways and associated transition rates is an important yet difficult problem in numerous chemical and biological systems, especially in situations with multiple competing pathways. When these high-dimensional systems are projected on low-dimensional coordinates, which are often needed for enhanced sampling or for interpretation of simulations and experiments, one can end up losing the kinetic connectivity of the underlying high-dimensional landscape. Thus, in the low-dimensional projection, metastable states might appear closer or further than they actually are. To deal with this issue, in this work, we develop a formalism that learns a multidimensional yet minimally complex reaction coordinate (RC) for generic high-dimensional systems. When projected along this RC, all possible kinetically relevant pathways can be demarcated and the true high-dimensional connectivity is maintained. One of the defining attributes of our method lies in that it can work on long unbiased simulations as well as biased simulations often needed for rare event systems. We demonstrate the utility of the method by studying a range of model systems including conformational transitions in a small peptide Ace-Ala3-Nme, where we show how two-dimensional and three-dimensional RCs found by our previously published spectral gap optimization method "SGOOP" [Tiwary, P. and Berne, B. J. Proc. Natl. Acad. Sci. 2016, 113, 2839] can capture the kinetics for 23 and all 28 out of the 28 dominant state-to-state transitions, respectively.

Optimizing String Method's Reproducibility Using Generalized Solute Tempering Replica Exchange.

Obtaining accurate and reproducible free energies from molecular simulations is somewhat tricky due to incomplete knowledge of crucial slow degrees of freedom leading to hidden barriers that can stymie sampling. Employing a sufficiently large number of collective variables (CV) and ensuring ergodic sampling in orthogonal CV space, perhaps via tempering methods, can reduce these issues to some extent. For complex systems with high-dimensional free energy landscapes, both these approaches become computationally expensive. For high-dimensional landscapes, efficient exploration can be enabled by using temperature-accelerated MD (TAMD) and identification and characterization of minimum free energy pathways connecting minima can be found by using the string method (SM). Both TAMD and SM use mean-force estimates from finite MD simulations and are thus susceptible to sampling restrictions from hidden variables. A recent development in parallel tempering methods, "generalized replica exchange solute tempering" (gREST), can enhance sampling at a reasonable computational cost with its flexibility to target very specific "solutes" which can include arbitrary independent variables. Considering the advantages of both methods, we implement gREST-enabled TAMD and SM. By considering two different collective variable representations of the pentapeptide neurotransmitter met-enkephalin, we show that both gREST-enabled TAMD and SM yield more accurate and reproducible free energy predictions than TAMD and SM alone. Given the moderate computational cost of gREST compared with other replica-exchange methods, gREST-enabled SM represents a more attractive method for characterizing free energy minima and pathways among them for a large variety of systems.

Surfing on minima of isostatic landscapes: avalanches and unjamming transition

Recently, we showed that optimization problems, both in infinite as well as in finite dimensions, for continuous variables and soft excluded volume constraints, can display entire isostatic phases where local minima of the cost function are marginally stable configurations endowed with non-linear excitations [, ]. In this work we describe an athermal adiabatic algorithm to explore with continuity the corresponding rough high-dimensional landscape. We concentrate on a prototype problem of this kind, the spherical perceptron optimization problem with linear cost function (hinge loss). This algorithm allows to ‘surf’ between isostatic marginally stable configurations and to investigate some properties of such landscape. In particular we focus on the statistics of avalanches occurring when local minima are destabilized. We show that when perturbing such minima, the system undergoes plastic rearrangements whose size is power law distributed and we characterize the corresponding critical exponent. Finally we investigate the critical properties of the unjamming transition, showing that the linear interaction potential gives rise to logarithmic behavior in the scaling of energy and pressure as a function of the distance from the unjamming point. For some quantities, the logarithmic corrections can be gauged out. This is the case of the number of soft constraints that are violated as a function of the distance from jamming which follows a non-trivial power law behavior.

Large deviations of extreme eigenvalues of generalized sample covariance matrices

We present an analytical technique to compute the probability of rare events in which the largest eigenvalue of a random matrix is atypically large (i.e., the right tail of its large deviations). The results also transfer to the left tail of the large deviations of the smallest eigenvalue. The technique improves upon past methods by not requiring the explicit law of the eigenvalues, and we apply it to a large class of random matrices that were previously out of reach. In particular, we solve an open problem related to the performance of principal components analysis on highly correlated data, and open the way towards analyzing the high-dimensional landscapes of complex inference models. We probe our results using an importance sampling approach, effectively simulating events with probability as small as .

Algorithmic thresholds for tensor PCA

We study the algorithmic thresholds for principal component analysis of Gaussian $k$-tensors with a planted rank-one spike, via Langevin dynamics and gradient descent. In order to efficiently recover the spike from natural initializations, the signal-to-noise ratio must diverge in the dimension. Our proof shows that the mechanism for the success/failure of recovery is the strength of the “curvature” of the spike on the maximum entropy region of the initial data. To demonstrate this, we study the dynamics on a generalized family of high-dimensional landscapes with planted signals, containing the spiked tensor models as specific instances. We identify thresholds of signal-to-noise ratios above which order 1 time recovery succeeds; in the case of the spiked tensor model, these match the thresholds conjectured for algorithms such as approximate message passing. Below these thresholds, where the curvature of the signal on the maximal entropy region is weak, we show that recovery from certain natural initializations takes at least stretched exponential time. Our approach combines global regularity estimates for spin glasses with pointwise estimates to study the recovery problem by a perturbative approach.

Adaptive walks on high-dimensional fitness landscapes and seascapes with distance-dependent statistics

The dynamics of evolution is intimately shaped by epistasis — interactions between genetic elements which cause the fitness-effect of combinations of mutations to be non-additive. Analyzing evolutionary dynamics that involves large numbers of epistatic mutations is intrinsically difficult. A crucial feature is that the fitness landscape in the vicinity of the current genome depends on the evolutionary history. A key step is thus developing models that enable study of the effects of past evolution on future evolution. In this work, we introduce a broad class of high-dimensional random fitness landscapes for which the correlations between fitnesses of genomes are a general function of genetic distance. Their Gaussian character allows for tractable computational as well as analytic understanding. We study the properties of these landscapes focusing on the simplest evolutionary process: random adaptive (uphill) walks. Conventional measures of “ruggedness” are shown to not much affect such adaptive walks. Instead, the long-distance statistics of epistasis cause all properties to be highly conditional on past evolution, determining the statistics of the local landscape (the distribution of fitness-effects of available mutations and combinations of these), as well as the global geometry of evolutionary trajectories. In order to further explore the effects of conditioning on past evolution, we model the effects of slowly changing environments. At long times, such fitness “seascapes” cause a statistical steady state with highly intermittent evolutionary dynamics: populations undergo bursts of rapid adaptation, interspersed with periods in which adaptive mutations are rare and the population waits for more new directions to be opened up by changes in the environment. Finally, we discuss prospects for studying more complex evolutionary dynamics and on broader classes of high-dimensional landscapes and seascapes.

Complexity of energy barriers in mean-field glassy systems

We analyze the energy barriers that allow escapes from a given local minimum in a complex high-dimensional landscape. We perform this study by using the Kac-Rice method and computing the typical number of critical points of the energy function at a given distance from the minimum. We analyze their Hessian in terms of random matrix theory and show that for a certain regime of energies and distances critical points are index-one saddles, or transition states, and are associated to barriers. We find that the transition state of lowest energy, important for the activated dynamics at low temperature, is strictly below the “threshold” level above which saddles proliferate. We characterize how the quenched complexity of transition states, important for the activated processes at finite temperature, depends on the energy of the state, the energy of the initial minimum, and the distance between them. The overall picture gained from this study is expected to hold generically for mean-field models of the glass transition.

Complex Energy Landscapes in Spiked-Tensor and Simple Glassy Models: Ruggedness, Arrangements of Local Minima, and Phase Transitions

We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the statistical properties of the corresponding landscapes and characterize the associated geometrical phase transitions. In order to perform our study, we develop a framework based on the Kac-Rice method that allows to compute the complexity of the landscape, i.e. the logarithm of the typical number of stationary points and their Hessian. This approach generalizes the one used to compute rigorously the annealed complexity of mean-field glass models. We discuss its advantages with respect to previous frameworks, in particular the thermodynamical replica method which is shown to lead to partially incorrect predictions.

Cytofast: A workflow for visual and quantitative analysis of flow and mass cytometry data to discover immune signatures and correlations

Multi-parametric flow and mass cytometry allows exceptional high-resolution exploration of the cellular composition of the immune system. A large panel of computational tools have been developed to analyze the high-dimensional landscape of the data generated. Analysis frameworks such as FlowSOM or Cytosplore incorporate clustering and dimensionality reduction techniques and include algorithms allowing visualization of multi-parametric cytometric analysis. To additionally provide means to quantify specific cell clusters and correlations between samples, we developed an R-package, called cytofast, for further downstream analysis. Specifically, cytofast enables the visualization and quantification of cell clusters for an efficient discovery of cell populations associated with diseases or physiology. We used cytofast on mass and flow cytometry datasets based on the modulation of the immune system upon immunotherapy. With cytofast, we rapidly generated visual representations of group-related immune cell clusters and showed correlations with the immune system composition. We discovered macrophage subsets that significantly decrease upon cancer immunotherapy and distinct prime-boost effects of prophylactic vaccines on the myeloid compartment. Cytofast is a time-efficient tool for comprehensive cytometric analysis to reveal immune signatures and correlations. Cytofast is available at Bioconductor.

Sequential Convex Programming for Computing Information-Theoretic Minimal Partitions: Nonconvex Nonsmooth Optimization

We consider an unsupervised image segmentation problem---from figure-ground separation to multiregion partitioning---that consists of maximal distribution separation (in terms of mutual information) with spatial regularity (total variation regularization), which is what we call information-theoretic minimal partitioning. Adopting the bounded variation framework, we provide an in-depth analysis of the problem which establishes theoretical foundations and investigates the structure of the associated energy from a variational perspective. In doing so, we show that the objective exhibits a form of difference of convex functionals, which leads us to a class of large-scale nonconvex optimization problems where convex optimization techniques can be successfully applied. In this regard, we propose sequential convex programming based on the philosophy of stochastic optimization and the Chambolle--Pock primal-dual algorithm. The key idea behind it is to construct a stochastic family of convex approximations of the original nonconvex function and sequentially minimize the associated subproblems. Indeed, its stochastic nature makes it possible to often escape from bad local minima toward near-optimal solutions, where such optimality can be justified in terms of the recent findings in statistical physics regarding a striking characteristic of the high-dimensional landscapes. We experimentally demonstrate such a favorable ability of the proposed algorithm as well as show the capacity of our approach in numerous experiments. The preliminary conference paper can be found in [Y. Kee, M. Souiai, D. Cremers, and J. Kim, Proceedings of the IEEE Conference on Computer Vision and Pattern, Recognition, 2014].

Nonuniqueness of classical inflationary trajectories on a high-dimensional landscape

Motivated by the string landscape, the inflationary potential may be high dimensional and complicated. We propose a new way to construct some high-dimensional random potentials, and simulate inflationary dynamics on top of that, for up to 50 dimensions in field space. Especially, random bifurcations of classical inflationary trajectory are studied. It is shown that the bifurcation probability increases as a function of the number of dimensions. Those random bifurcations are not consistent with observations and dramatically limit the parameter space of inflation on a complicated landscape. For example, in 10 dimensions, only $1{0}^{\ensuremath{-}3}--1{0}^{\ensuremath{-}6}$ of the parameter space volume leads to unique classical trajectories. The rest is ruled out by random bifurcations.

Epigenetic landscapes explain partially reprogrammed cells and identify key reprogramming genes.

A common metaphor for describing development is a rugged “epigenetic landscape” where cell fates are represented as attracting valleys resulting from a complex regulatory network. Here, we introduce a framework for explicitly constructing epigenetic landscapes that combines genomic data with techniques from spin-glass physics. Each cell fate is a dynamic attractor, yet cells can change fate in response to external signals. Our model suggests that partially reprogrammed cells are a natural consequence of high-dimensional landscapes, and predicts that partially reprogrammed cells should be hybrids that co-express genes from multiple cell fates. We verify this prediction by reanalyzing existing datasets. Our model reproduces known reprogramming protocols and identifies candidate transcription factors for reprogramming to novel cell fates, suggesting epigenetic landscapes are a powerful paradigm for understanding cellular identity.

How dominant is the most efficient pathway through the potential energy landscape of a slowly diffusing disordered system?

It has been suggested that the most-efficient pathway taken by a slowly diffusing many-body system is its geodesic path through the parts of the potential energy landscape lying below a prescribed value of the potential energy. From this perspective, slow diffusion occurs just because these optimal paths become particularly long and convoluted. We test this idea here by applying it to diffusion in two kinds of well-studied low-dimensional percolation problems: the 2d overlapping Lorentz model, and square and simple-cubic bond-dilute lattices. Although the most efficient path should be at its most dominant with the high-dimensional landscapes associated with many-body problems, it is useful to examine simpler, low-dimensional, constant-potential-energy problems such as these ones, both because the simpler models lend themselves to more accurate geodesic-path-finding approaches, and because they offer a significant contrast to many of the models used in the traditional energy-landscape literature. Neither the continuum nor the lattice percolation examples are adequately described by our geodesic-path formalism in the weakly disordered (relatively-fast-diffusion) limit, but in both cases the formalism successfully predicts the existence of the percolation transition and (to a certain extent) the slow diffusion characteristic of near-percolation behavior. The numerical results for these models are not nearly accurate enough near their transitions to describe critical exponents, but the models do showcase the qualitative validity of the geodesic perspective in that they allow us to see explicitly how tortuous and sparse the optimal pathways become as the diffusion constants begin to vanish.

Determination of noncovalent docking by infrared spectroscopy of cold gas-phase complexes.

Multidentate, noncovalent interactions between small molecules and biopolymer fragments are central to processes ranging from drug action to selective catalysis. We present a versatile and sensitive spectroscopic probe of functional groups engaged in hydrogen bonding in such contexts. This involves measurement of the frequency changes in specific covalent bonds upon complex formation, information drawn from otherwise transient complexes that have been extracted from solution and conformationally frozen near 10 kelvin in gas-phase clusters. Resonances closely associated with individual oscillators are easily identified through site-specific isotopic labeling, as demonstrated by application of the method to an archetypal system involving a synthetic tripeptide known to bind biaryl substrates through tailored hydrogen bonding to catalyze their asymmetric bromination. With such data, calculations readily converge on the plausible operative structures in otherwise computationally prohibitive, high-dimensionality landscapes.

Exploring high-dimensional energy landscapes

Maps are not reserved for geography. Chemical reactions, atomic diffusion and protein folding all involve atomic displacements determined by the topography of a complex energy landscape. These landscapes are largely unexplored, and our first priority is to identify their key features: the energy minima and the connecting paths between them. Such a study represents a formidable task. The effort needed to map a space increases exponentially with its dimensionality and becomes rapidly out of reach for the high-dimensional problems of interest in physics, chemistry and biology. Therefore, we have to satisfy ourselves with only a very crude knowledge of these energy landscapes. Recently, many researchers have been developing algorithms for exploring and mapping the potential energy landscapes of systems as diverse as polypeptides, chemical reactions, Lennard-Jones clusters and silica glass. In this article, we address some of the general issues and present an algorithm, called the activation-relaxation technique (ART), which we developed for mapping high-dimensional landscapes.