High-dimensional Dynamics Research Articles

Multiframe Evolving Dynamic Functional Network Connectivity Motifs (Evodfncs) from Continuity-Preserving Planar Embedding.

The study of brain network connectivity as a time-varying property began relatively recently and to date has remained primarily concerned with capturing a handful of discrete static states that characterize connectivity as measured on a timescale shorter than that of the full scan. Capturing group- level representations of temporally evolving patterns of connectivity is a challenging and important next step in fully leveraging the information available in large resting state functional magnetic resonance imaging (rs-fMRI) studies. We introduce a flexible, extensible data-driven framework for the identification of group-level multiframe (movie-style) dynamic functional network connectivity (dFNC) states. Our approach employs uniform manifold approximation and embedding (UMAP) to produce a planar embedding of the high-dimensional whole-brain connectivity dynamics that preserves important features, such as trajectory continuity, characterizing dynamics in the native high dimensional state space. The method is validated in application to a large rs- fMRI study of schizophrenia where it extracts naturalistic fluidly-varying connectivity motifs that differ between schizophrenia patients (SZs) and healthy controls (HC).Functional Magnetic Resonance Imaging, Functional Network Connectivity, Dynamic Functional Network Connectivity, Schizophrenia.

Nonadiabatic quantum dynamics of the coherent excited state intramolecular proton transfer of 10-hydroxybenzo[h]quinoline

The photoinduced nonadiabatic dynamics of the enol-keto isomerization of 10-hydroxybenzo[h]quinoline (HBQ) are studied computationally using high-dimensional quantum dynamics. The simulations are based on a diabatic vibronic coupling Hamiltonian, which includes the two lowest pi pi ^* excited states and a npi ^* state, which has high energy in the Franck–Condon zone, but significantly stabilizes upon excited state intramolecular proton transfer. A procedure, applicable to large classes of excited state proton transfer reactions, is presented to parametrize this model using potential energies, forces and force constants, which, in this case, are obtained by time-dependent density functional theory. The wave packet calculations predict a time scale of 10–15 fs for the photoreaction, and reproduce the time constants and the coherent oscillations observed in time-resolved spectroscopic studies performed on HBQ. In contrast to the interpretation given to the most recent experiments, it is found that the reaction initiated by 1pi pi ^* longleftarrow S_0 photoexcitation proceeds essentially on a single potential energy surface, and the observed coherences bear signatures of Duschinsky mode-mixing along the reaction path. The dynamics after the 2pi pi ^* longleftarrow S_0 excitation are instead nonadiabatic, and the npi ^* state plays a major role in the relaxation process. The simulations suggest a mainly active role of the proton in the isomerization, rather than a passive migration assisted by the vibrations of the benzoquinoline backbone.Graphic

Exact symmetries and threshold states in two-dimensional models for QCD

Two-dimensional SU(N) gauge theory coupled to a Majorana fermion in the adjoint representation is a nice toy model for higher-dimensional gauge dynamics. It possesses a multitude of “gluinoball” bound states whose spectrum has been studied using numerical diagonalizations of the light-cone Hamiltonian. We extend this model by coupling it to Nf flavors of fundamental Dirac fermions (quarks). The extended model also contains meson-like bound states, both bosonic and fermionic, which in the large-N limit decouple from the gluinoballs. We study the large-N meson spectrum using the Discretized Light-Cone Quantization (DLCQ). When all the fermions are massless, we exhibit an exact mathfrak{osp} (1|4) symmetry algebra that leads to an infinite number of degeneracies in the DLCQ approach. More generally, we show that many single-trace states in the theory are threshold bound states that are degenerate with multi-trace states. These exact degeneracies can be explained using the Kac-Moody algebra of the SU(N) current. We also present strong numerical evidence that additional threshold states appear in the continuum limit. Finally, we make the quarks massive while keeping the adjoint fermion massless. In this case too, we observe some exact degeneracies that show that the spectrum of mesons becomes continuous above a certain threshold. This demonstrates quantitatively that the fundamental string tension vanishes in the massless adjoint QCD2 without explicit four-fermion operators.

Hierarchical learning from human preferences and curiosity

AbstractRecent success in scaling deep reinforcement algorithms (DRL) to complex problems has been driven by well-designed extrinsic rewards, which limits their applicability to many real-world tasks where rewards are naturally extremely sparse. One solution to this problem is to introduce human guidance to drive the agent’s learning. Although low-level demonstrations is a promising approach, it was shown that such guidance may be difficult for experts to demonstrate since some tasks require a large amount of high-quality demonstrations. In this work, we explore human guidance in the form of high-level preferences between sub-goals, leading to drastic reductions in both human effort and cost of exploration. We design a novel hierarchical reinforcement learning method that introduces non-expert human preferences at the high-level, and curiosity to drastically speed up the convergence of subpolicies to reach any sub-goals. We further propose a strategy based on curiosity to automatically discover sub-goals. We evaluate the proposed method on 2D navigation tasks, robotic control tasks, and image-based video games (Atari 2600), which have high-dimensional observations, sparse rewards, and complex state dynamics. The experimental results show that the proposed method can learn significantly faster than traditional hierarchical RL methods and drastically reduces the amount of human effort required over standard imitation learning approaches.

Between synchrony and turbulence: intricate hierarchies of coexistence patterns

Coupled oscillators, even identical ones, display a wide range of behaviours, among them synchrony and incoherence. The 2002 discovery of so-called chimera states, states of coexisting synchronized and unsynchronized oscillators, provided a possible link between the two and definitely showed that different parts of the same ensemble can sustain qualitatively different forms of motion. Here, we demonstrate that globally coupled identical oscillators can express a range of coexistence patterns more comprehensive than chimeras. A hierarchy of such states evolves from the fully synchronized solution in a series of cluster-splittings. At the far end of this hierarchy, the states further collide with their own mirror-images in phase space – rendering the motion chaotic, destroying some of the clusters and thereby producing even more intricate coexistence patterns. A sequence of such attractor collisions can ultimately lead to full incoherence of only single asynchronous oscillators. Chimera states, with one large synchronized cluster and else only single oscillators, are found to be just one step in this transition from low- to high-dimensional dynamics.

Networks of Causal Linkage Between Eigenmodes Characterize Behavioral Dynamics of Caenorhabditis elegans.

Behavioral phenotyping of model organisms has played an important role in unravelling the complexities of animal behavior. Techniques for classifying behavior often rely on easily identified changes in posture and motion. However, such approaches are likely to miss complex behaviors that cannot be readily distinguished by eye (e.g., behaviors produced by high dimensional dynamics). To explore this issue, we focus on the model organism Caenorhabditis elegans, where behaviors have been extensively recorded and classified. Using a dynamical systems lens, we identify high dimensional, nonlinear causal relationships between four basic shapes that describe worm motion (eigenmodes, also called “eigenworms”). We find relationships between all pairs of eigenmodes, but the timescales of the interactions vary between pairs and across individuals. Using these varying timescales, we create “interaction profiles” to represent an individual’s behavioral dynamics. As desired, these profiles are able to distinguish well-known behavioral states: i.e., the profiles for foraging individuals are distinct from those of individuals exhibiting an escape response. More importantly, we find that interaction profiles can distinguish high dimensional behaviors among divergent mutant strains that were previously classified as phenotypically similar. Specifically, we find it is able to detect phenotypic behavioral differences not previously identified in strains related to dysfunction of hermaphrodite-specific neurons.

Neuroinformatic Tool to Study High Dimensional Dynamics with Distributed Delays in Neural Mass Models

Neuroinformatic Tool to Study High Dimensional Dynamics with Distributed Delays in Neural Mass Models

Sparse identification of nonlinear dynamics with low-dimensionalized flow representations

We perform a sparse identification of nonlinear dynamics (SINDy) for low-dimensionalized complex flow phenomena. We first apply the SINDy with two regression methods, the thresholded least square algorithm and the adaptive least absolute shrinkage and selection operator which show reasonable ability with a wide range of sparsity constant in our preliminary tests, to a two-dimensional single cylinder wake at$Re_D=100$, its transient process and a wake of two-parallel cylinders, as examples of high-dimensional fluid data. To handle these high-dimensional data with SINDy whose library matrix is suitable for low-dimensional variable combinations, a convolutional neural network-based autoencoder (CNN-AE) is utilized. The CNN-AE is employed to map a high-dimensional dynamics into a low-dimensional latent space. The SINDy then seeks a governing equation of the mapped low-dimensional latent vector. Temporal evolution of high-dimensional dynamics can be provided by combining the predicted latent vector by SINDy with the CNN decoder which can remap the low-dimensional latent vector to the original dimension. The SINDy can provide a stable solution as the governing equation of the latent dynamics and the CNN-SINDy-based modelling can reproduce high-dimensional flow fields successfully, although more terms are required to represent the transient flow and the two-parallel cylinder wake than the periodic shedding. A nine-equation turbulent shear flow model is finally considered to examine the applicability of SINDy to turbulence, although without using CNN-AE. The present results suggest that the proposed scheme with an appropriate parameter choice enables us to analyse high-dimensional nonlinear dynamics with interpretable low-dimensional manifolds.

Recurrent dynamics in the cerebral cortex: Integration of sensory evidence with stored knowledge

Current concepts of sensory processing in the cerebral cortex emphasize serial extraction and recombination of features in hierarchically structured feed-forward networks in order to capture the relations among the components of perceptual objects. These concepts are implemented in convolutional deep learning networks and have been validated by the astounding similarities between the functional properties of artificial systems and their natural counterparts. However, cortical architectures also display an abundance of recurrent coupling within and between the layers of the processing hierarchy. This massive recurrence gives rise to highly complex dynamics whose putative function is poorly understood. Here a concept is proposed that assigns specific functions to the dynamics of cortical networks and combines, in a unifying approach, the respective advantages of recurrent and feed-forward processing. It is proposed that the priors about regularities of the world are stored in the weight distributions of feed-forward and recurrent connections and that the high-dimensional, dynamic space provided by recurrent interactions is exploited for computations. These comprise the ultrafast matching of sensory evidence with the priors covertly represented in the correlation structure of spontaneous activity and the context-dependent grouping of feature constellations characterizing natural objects. The concept posits that information is encoded not only in the discharge frequency of neurons but also in the precise timing relations among the discharges. Results of experiments designed to test the predictions derived from this concept support the hypothesis that cerebral cortex exploits the high-dimensional recurrent dynamics for computations serving predictive coding.

Approximate dynamics leading to more optimal control: Efficient exact derivatives

Accurate derivatives are important for efficiently locally traversing and converging in quantum optimization landscapes. By deriving analytically exact control derivatives (gradient and Hessian) for unitary control tasks, we show here that the computational feasibility of meeting this accuracy requirement depends on the choice of propagation scheme and problem representation. Even when exact propagation is sufficiently cheap it is, perhaps surprisingly, much more efficient to optimize the (appropriately) approximate propagators: approximations in the dynamics are traded off for significant complexity reductions in the exact derivative calculations. Importantly, past the initial analytical considerations, only standard numerical techniques are explicitly required with straightforward application to realistic systems. These results are numerically verified for two concrete problems of increasing Hilbert space dimensionality. The best schemes obtain unit fidelity to machine precision whereas the results for other schemes are separated consistently by orders of magnitude in computation time and in worst case 10 orders of magnitude in achievable fidelity. Since these gaps continually increase with system size and complexity, this methodology allows numerically efficient optimization of very high-dimensional dynamics, e.g. in many-body contexts, operating in the high-fidelity regime which will be published separately.

A top-down measure of gene-to-gene coordination for analyzing cell-to-cell variability

Recent technological advances, such as single-cell RNA sequencing (scRNA-seq), allow the measurement of gene expression profiles of individual cells. These expression profiles typically exhibit substantial variations even across seemingly homogeneous populations of cells. Two main different sources contribute to this measured variability: actual differences between the biological activity of the cells and technical measurement errors. Analysis of the biological variability may provide information about the underlying gene regulation of the cells, yet distinguishing it from the technical variability is a challenge. Here, we apply a recently developed computational method for measuring the global gene coordination level (GCL) to systematically study the cell-to-cell variability in numerical models of gene regulation. We simulate ‘biological variability’ by introducing heterogeneity in the underlying regulatory dynamic of different cells, while ‘technical variability’ is represented by stochastic measurement noise. We show that the GCL decreases for cohorts of cells with increased ‘biological variability’ only when it is originated from the interactions between the genes. Moreover, we find that the GCL can evaluate and compare—for cohorts with the same cell-to-cell variability—the ratio between the introduced biological and technical variability. Finally, we show that the GCL is robust against spurious correlations that originate from a small sample size or from the compositionality of the data. The presented methodology can be useful for future analysis of high-dimensional ecological and biochemical dynamics.

High-Dimensional Quantum Dynamics Study on Excitation-Specific Surface Scattering Including Lattice Effects of a Five-Atom Surface Cell.

In this work, high-dimensional (21D) quantum dynamics calculations on the mode-specific surface scattering of a carbon monoxide molecule on a copper(100) surface with lattice effects of a five-atom surface cell are performed through the multilayer multiconfiguration time-dependent Hartree (ML-MCTDH) method. We employ a surface model in which five surface atoms near the impact site are treated as fully flexible quantum particles, while all other more distant atoms are kept at fixed locations. To efficiently perform the 21D ML-MCTDH wave packet propagation, the potential energy surface is transferred to a canonical polyadic decomposition form with the aid of a Monte Carlo-based method. Excitation-specific sticking probabilities of CO on Cu(100) are computed, and lattice effects caused by the flexible surface atoms are demonstrated by comparison with sticking probabilities computed for a rigid surface. The dependence of the sticking probability of the initial state of the system is studied, and it is found that the sticking probability is reduced when the surface atom on the impact site is initially vibrationally excited.

Unifying Large- and Small-Scale Theories of Coordination

Coordination is a ubiquitous feature of all living things. It occurs by virtue of informational coupling among component parts and processes and can be quite specific (as when cells in the brain resonate to signals in the environment) or nonspecific (as when simple diffusion creates a source–sink dynamic for gene networks). Existing theoretical models of coordination—from bacteria to brains to social groups—typically focus on systems with very large numbers of elements (N→∞) or systems with only a few elements coupled together (typically N = 2). Though sharing a common inspiration in Nature’s propensity to generate dynamic patterns, both approaches have proceeded largely independent of each other. Ideally, one would like a theory that applies to phenomena observed on all scales. Recent experimental research by Mengsen Zhang and colleagues on intermediate-sized ensembles (in between the few and the many) proves to be the key to uniting large- and small-scale theories of coordination. Disorder–order transitions, multistability, order–order phase transitions, and especially metastability are shown to figure prominently on multiple levels of description, suggestive of a basic Coordination Dynamics that operates on all scales. This unified coordination dynamics turns out to be a marriage of two well-known models of large- and small-scale coordination: the former based on statistical mechanics (Kuramoto) and the latter based on the concepts of Synergetics and nonlinear dynamics (extended Haken–Kelso–Bunz or HKB). We show that models of the many and the few, previously quite unconnected, are thereby unified in a single formulation. The research has led to novel topological methods to handle the higher-dimensional dynamics of coordination in complex systems and has implications not only for understanding coordination but also for the design of (biorhythm inspired) computers.

A tale of two rhythms: Locked clocks and chaos in biology.

The fundamental mechanisms that control and regulate biological organisms exhibit a surprising level of complexity. Oscillators are perhaps the simplest motifs that produce time-varying dynamics and are ubiquitous in biological systems. It is also known that such biological oscillators interact with each other-for instance, circadian oscillators affect the cell cycle, and somitogenesis clock proteins in adjacent cells affect each other in developing embryos. Therefore, it is vital to understand the effects that can emerge from non-linear interaction between oscillations. Here, we show how oscillations typically arise in biology and take the reader on a tour through the great variety in dynamics that can emerge even from a single pair of coupled oscillators. We explain how chaotic dynamics can emerge and outline the methods of detecting this in experimental time traces. Finally, we discuss the potential role of such complex dynamical features in biological systems.

Direct product-type grid representations for angular coordinates in extended space and their application in the MCTDH approach.

Multi-configurational time-dependent Hartree (MCTDH) calculations using time-dependent grid representations can be used to accurately simulate high-dimensional quantum dynamics on general ab initio potential energy surfaces. Employing the correlation discrete variable representation, sets of direct product type grids are employed in the calculation of the required potential energy matrix elements. This direct product structure can be a problem if the coordinate system includes polar and azimuthal angles that result in singularities in the kinetic energy operator. In the present work, a new direct product-type discrete variable representation (DVR) for arbitrary sets of polar and azimuthal angles is introduced. It employs an extended coordinate space where the range of the polar angles is taken to be [-π, π]. The resulting extended space DVR resolves problems caused by the singularities in the kinetic energy operator without generating a very large spectral width. MCTDH calculations studying the F·CH4 complex are used to investigate important properties of the new scheme. The scheme is found to allow for more efficient integration of the equations of motion compared to the previously employed cot-DVR approach [G. Schiffel and U. Manthe, Chem. Phys. 374, 118 (2010)] and decreases the required central processing unit times by about an order of magnitude.

Convolutional neural network and long short-term memory based reduced order surrogate for minimal turbulent channel flow

We investigate the applicability of the machine learning based reduced order model (ML-ROM) to three-dimensional complex flows. As an example, we consider a turbulent channel flow at the friction Reynolds number of Reτ=110 in a minimum domain, which can maintain coherent structures of turbulence. Training datasets are prepared by direct numerical simulation (DNS). The present ML-ROM is constructed by combining a three-dimensional convolutional neural network autoencoder (CNN-AE) and a long short-term memory (LSTM). The CNN-AE works to map high-dimensional flow fields into a low-dimensional latent space. The LSTM is, then, utilized to predict a temporal evolution of the latent vectors obtained by the CNN-AE. The combination of the CNN-AE and LSTM can represent the spatiotemporal high-dimensional dynamics of flow fields by only integrating the temporal evolution of the low-dimensional latent dynamics. The turbulent flow fields reproduced by the present ML-ROM show statistical agreement with the reference DNS data in time-ensemble sense, which can also be found through an orbit-based analysis. Influences of the population of vortical structures contained in the domain and the time interval used for temporal prediction on the ML-ROM performance are also investigated. The potential and limitation of the present ML-ROM for turbulence analysis are discussed at the end of our presentation.

Higher dimensional localized and periodic wave dynamics in an integrable (2+1)-dimensional deep water oceanic wave model

In this article, a new integrable (2+1)-dimensional Kundu–Mukherjee–Naskar model which is a variant of the well-known nonlinear Schrödinger equation is investigated. Bright–dark optical solitons along with periodic waves, complexiton, and rational solutions are constructed by employing a generalized traveling wave analysis, Jacobian-elliptic function, Riccati equation, and ansatz approach. Further, the dynamics of these bright/dark optical solitons and complexiton/periodic waves are studied by exploring the importance of arbitrary physical parameters with graphical demonstrations for a clear understanding and stability of the considered model is also investigated. The obtained higher dimensional nonlinear wave solutions of this integrable system shall have useful applications in different physical systems including the dynamics of beam propagation in optical fibers, ion-acoustic waves in magnetized plasma, and deep water oceanic rogue waves.

Multiscale measures of phase-space trajectories.

Characterizing the multiscale nature of fluctuations from nonlinear and nonstationary time series is one of the most intensively studied contemporary problems in nonlinear sciences. In this work, we address this problem by combining two established concepts-empirical mode decomposition (EMD) and generalized fractal dimensions-into a unified analysis framework. Specifically, we demonstrate that the intrinsic mode functions derived by EMD can be used as a source of local (in terms of scales) information about the properties of the phase-space trajectory of the system under study, allowing us to derive multiscale measures when looking at the behavior of the generalized fractal dimensions at different scales. This formalism is applied to three well-known low-dimensional deterministic dynamical systems (the Hénon map, the Lorenz '63 system, and the standard map), three realizations of fractional Brownian motion with different Hurst exponents, and two somewhat higher-dimensional deterministic dynamical systems (the Lorenz '96 model and the on-off intermittency model). These examples allow us to assess the performance of our formalism with respect to practically relevant aspects like additive noise, different initial conditions, the length of the time series under study, low- vs high-dimensional dynamics, and bursting effects. Finally, by taking advantage of two real-world systems whose multiscale features have been widely investigated (a marine stack record providing a proxy of the global ice volume variability of the past 5×106 years and the SYM-H geomagnetic index), we also illustrate the applicability of this formalism to real-world time series.

CNN-LSTM based reduced order modeling of two-dimensional unsteady flows around a circular cylinder at different Reynolds numbers

We investigate the capability of machine learning (ML) based reduced order model (ML-ROM) for two-dimensional unsteady flows around a circular cylinder at different Reynolds numbers. The present ML-ROM is constructed by two ML schemes: a convolutional neural network-based autoencoder (CNN-AE) and a long short-term memory (LSTM). The CNN-AE is utilized to map high-dimensional flow fields obtained by direct numerical simulation (DNS) into a low-dimensional latent space while keeping their spatially coherent information. The LSTM is then trained to learn the temporal evolution of the mapped latent vectors together with the information on the Reynolds number. Using the trained LSTM model, the high-dimensional dynamics of flow fields can be reproduced with the aid of the decoder part of CNN-AE, which can map the predicted low-dimensional latent vector to the high-dimensional space. We find that the flow fields generated by the present ML-ROM show statistical agreement with the reference DNS data. The dependence of the accuracy of the proposed model on the Reynolds number is also examined in detail. The present results demonstrate that the ML-ROM can reconstruct flows at the Reynolds numbers that were not used in the training process unless the flow regime drastically changes.

Threefold way to the dimension reduction of dynamics on networks: An application to synchronization

Several complex systems can be modeled as large networks in which the state of the nodes continuously evolves through interactions among neighboring nodes, forming a high-dimensional nonlinear dynamical system. One of the main challenges of Network Science consists in predicting the impact of network topology and dynamics on the evolution of the states and, especially, on the emergence of collective phenomena, such as synchronization. We address this problem by proposing a Dynamics Approximate Reduction Technique (DART) that maps high-dimensional (complete) dynamics unto low-dimensional (reduced) dynamics while preserving the most salient features, both topological and dynamical, of the original system. DART generalizes recent approaches for dimension reduction by allowing the treatment of complex-valued dynamical variables, heterogeneities in the intrinsic properties of the nodes as well as modular networks with strongly interacting communities. Most importantly, we identify three major reduction procedures whose relative accuracy depends on whether the evolution of the states is mainly determined by the intrinsic dynamics, the degree sequence, or the adjacency matrix. We use phase synchronization of oscillator networks as a benchmark for our threefold method. We successfully predict the synchronization curves for three phase dynamics (Winfree, Kuramoto, theta) on the stochastic block model. Moreover, we obtain the bifurcations of the Kuramoto-Sakaguchi model on the mean stochastic block model with asymmetric blocks and we show numerically the existence of periphery chimera state on the two-star graph. This allows us to highlight the critical role played by the asymmetry of community sizes on the existence of chimera states. Finally, we systematically recover well-known analytical results on explosive synchronization by using DART for the Kuramoto-Sakaguchi model on the star graph.