High-dimensional Dynamics Research Articles

Ordinal pattern-based complexity analysis of high-dimensional chaotic time series.

The ordinal pattern-based complexity-entropy plane is a popular tool in nonlinear dynamics for distinguishing stochastic signals (noise) from deterministic chaos. Its performance, however, has mainly been demonstrated for time series from low-dimensional discrete or continuous dynamical systems. In order to evaluate the usefulness and power of the complexity-entropy (CE) plane approach for data representing high-dimensional chaotic dynamics, we applied this method to time series generated by the Lorenz-96 system, the generalized Hénon map, the Mackey-Glass equation, the Kuramoto-Sivashinsky equation, and to phase-randomized surrogates of these data. We find that both the high-dimensional deterministic time series and the stochastic surrogate data may be located in the same region of the complexity-entropy plane, and their representations show very similar behavior with varying lag and pattern lengths. Therefore, the classification of these data by means of their position in the CE plane can be challenging or even misleading, while surrogate data tests based on (entropy, complexity) yield significant results in most cases.

Airfoil-based convolutional autoencoder and long short-term memory neural network for predicting coherent structures evolution around an airfoil

Airfoil stall induced by unsteady flow separation is a major inhibitor that constrains the aircraft design. Hence, it is critical to have a better understanding of the flow dynamics around the airfoil near stall conditions. Recently, machine learning (ML) has been widely used to predict the spatiotemporal evolutions of coherent structures over the airfoil. Nonetheless, due to the existence of strong nonlinearity introduced by turbulence, most of the existing approaches focus on prediction of the flow fields in the laminar flow regime. In this work, we seek to examine the applicability of deep neural network (DNN) to prediction of turbulent flows around a NACA (National Advisory Committee for Aeronautics) 4412 airfoil. Large eddy simulations (LES) are performed to prepare the training dataset. Both proper orthogonal decomposition (POD) and DNN-based reduced-order models (ROM) are employed for reconstructing the flow fields. A convolutional autoencoder (CAE) is applied to map the high-dimensional flow dynamics into the low-dimensional nonlinear manifold. The geometric features are considered in the input matrix of the CAE network by setting physical quantities inside the airfoil to zero to reflect the physics. Then, the multi-head attention-strengthened long short-term memory (MH-LSTM) is utilized to predict the evolutions of latent vectors. Finally, the high-dimensional flow dynamics can be reconstructed by utilizing the decoder part of CAE. We believe that the MH-LSTM methodology provides a promising path towards predicting the unsteady flow field over the airfoil by capitalizing on the capability of multi-head attention allowing for attending to distinct parts of input sequence.

Analytical high-dimensional operators in canonical polyadic finite basis representation (CP-FBR).

In the present work, we introduce a simple means of obtaining an analytical (i.e., grid-free) canonical polyadic (CP) representation of a multidimensional function that is expressed in terms of a set of discrete data. For this, we make use of an initial CP guess, even not fully converged, and a set of auxiliary basis functions [finite basis representation (FBR)]. The resulting CP-FBR expression constitutes the CP counterpart of our previous Tucker sum-of-products-FBR approach. However, as is well-known, CP expressions are much more compact. This has obvious advantages in high-dimensional quantum dynamics. The power of CP-FBR lies in the fact that it requires a grid much coarser than the one needed for the dynamics. In a subsequent step, the basis functions can be interpolated to any desired density of grid points. This is useful, for instance, when different initial conditions (e.g., energy content) of a system are to be considered. We show the application of the method to bound systems of increased dimensionality: H2 (3D), HONO (6D), and CH4 (9D).

Mimicking Gait Dynamics: A Step Toward Precision Learning of Human Activities

We develop a computational protocol for mimicking personal gait dynamics with 12-dimensional time series derived from 4 accelerometer sensors found in the MAREA database and then explore its utilities in line with precision learning of human activities. The foundation of mimicking high dimensional rhythmic dynamics is explicitly established upon deterministic and stochastic structures found on structural representations of evolving biomechanical states hidden within all computed gait cycles. Such a technique enables practitioners to detect and confirm minute structural changes that could last for only a few cycles with high precision. Our computational developments are step-by-step illustrated via one subject’s data, while the other 8 subjects’ data are also analyzed and compared accordingly. A common cyclic composition of evolving biomechanical states of various temporal scales emerges from the 9 subjects’ comparisons. We conclude that mimicking an individual’s gait dynamics offers precise detections of potential multiscale minute differences against gait dynamics of different time periods or of different persons, and further offers clues of efficiency on personal walking activity. This mimicking-based capability is a cornerstone for the proof of concept: dynamics mimicking enables precision learning by improving the efficiency of learning and performing human activities in competitive sports, social dancing, and physical rehabilitation, among many others.

Reconstructing in-depth activity for chaotic 3D spatiotemporal excitable media models based on surface data.

Motivated by potential applications in cardiac research, we consider the task of reconstructing the dynamics within a spatiotemporal chaotic 3D excitable medium from partial observations at the surface. Three artificial neural network methods (a spatiotemporal convolutional long-short-term-memory, an autoencoder, and a diffusion model based on the U-Net architecture) are trained to predict the dynamics in deeper layers of a cube from observational data at the surface using data generated by the Barkley model on a 3D domain. The results show that despite the high-dimensional chaotic dynamics of this system, such cross-prediction is possible, but non-trivial and as expected, its quality decreases with increasing prediction depth.

Discernibility of Topological Variations for Networked LTI Systems

In this article, the discernibility of topological variations for networked linear time-invariant (LTI) systems is investigated, where the network topology is general, and the nodes have identical higher dimensional dynamics. A necessary and sufficient condition on the discernibility is derived, revealing how the topological variations, node-system dynamics, and inner interactions altogether affect the discernibility of the network. Compared with the existing conditions in (Patil et al., 2019) and (Roy et al., 2020), which require the network topology to be undirected, this condition is more general. Furthermore, the discernibility of topological variations for multiagent systems is revisited. A new necessary and sufficient condition is established, and the indiscernible space is completely characterized. Differing from the condition provided in (Roy et al., 2020), this condition removes the requirements on the multiagent system and has broader applicability. The effectiveness of the results is demonstrated by several examples.

Minimal frustration underlies the usefulness of incomplete regulatory network models in biology

Regulatory networks as large and complex as those implicated in cell-fate choice are expected to exhibit intricate, very high-dimensional dynamics. Cell-fate choice, however, is a macroscopically simple process. Additionally, regulatory network models are almost always incomplete and/or inexact, and do not incorporate all the regulators and interactions that may be involved in cell-fate regulation. In spite of these issues, regulatory network models have proven to be incredibly effective tools for understanding cell-fate choice across contexts and for making useful predictions. Here, we show that minimal frustration-a feature of biological networks across contexts but not of random networks-can compel simple, low-dimensional steady-state behavior even in large and complex networks. Moreover, the steady-state behavior of minimally frustrated networks can be recapitulated by simpler networks such as those lacking many of the nodes and edges and those that treat multiple regulators as one. The present study provides a theoretical explanation for the success of network models in biology and for the challenges in network inference.

Adaptive Learning and Optimization of High-Speed Sailing Maneuvers for America's Cup

Abstract This paper presents an adaptive control strategy to optimize the sailing maneuvers of an AC75 foiling sailboat competing in America's Cup. Foiling yachts have nonlinear, high-dimensional, and unstable dynamics due to several articulations for fast motions and maneuverability. Achieving aggressive and optimal maneuvers requires taking these complex dynamics into account instead of analytical optimizations using reduced-order models. We compared extremum-seeking and Jacobian learning (JL) control approaches on a full-order model to achieve optimal maneuvers and used JL to optimize articulations. The controllers were integrated with a high-fidelity sailboat simulator for safe and efficient maneuver optimization. The optimal solutions were subject to physical/actuator constraints and those enforced to ensure the feasibility of the maneuvers by humans (sailors). The close-hauled and tacking maneuvers were optimized to achieve maximum velocity made good (VMG) and minimum loss of VMG, respectively. The optimal maneuvers boast a marginal VMG loss of less than 1.5%, which enables exploiting areas of good wind conditions in the racing environment.

Identifying behavioral structure from deep variational embeddings of animal motion

Quantification and detection of the hierarchical organization of behavior is a major challenge in neuroscience. Recent advances in markerless pose estimation enable the visualization of high-dimensional spatiotemporal behavioral dynamics of animal motion. However, robust and reliable technical approaches are needed to uncover underlying structure in these data and to segment behavior into discrete hierarchically organized motifs. Here, we present an unsupervised probabilistic deep learning framework that identifies behavioral structure from deep variational embeddings of animal motion (VAME). By using a mouse model of beta amyloidosis as a use case, we show that VAME not only identifies discrete behavioral motifs, but also captures a hierarchical representation of the motif’s usage. The approach allows for the grouping of motifs into communities and the detection of differences in community-specific motif usage of individual mouse cohorts that were undetectable by human visual observation. Thus, we present a robust approach for the segmentation of animal motion that is applicable to a wide range of experimental setups, models and conditions without requiring supervised or a-priori human interference.

A local particle filter and its Gaussian mixture extension implemented with minor modifications to the LETKF

Abstract. A particle filter (PF) is an ensemble data assimilation method that does not assume Gaussian error distributions. Recent studies proposed local PFs (LPFs), which use localization, as in the ensemble Kalman filter, to apply the PF efficiently for high-dimensional dynamics. Among others, Penny and Miyoshi (2016) developed an LPF in the form of the ensemble transform matrix of the local ensemble transform Kalman filter (LETKF). The LETKF has been widely accepted for various geophysical systems, including numerical weather prediction (NWP) models. Therefore, implementing the LPF consistently with an existing LETKF code is useful. This study develops a software platform for the LPF and its Gaussian mixture extension (LPFGM) by making slight modifications to the LETKF code with a simplified global climate model known as Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY). A series of idealized twin experiments were accomplished under the ideal-model assumption. With large inflation by the relaxation to prior spread, the LPF showed stable filter performance with dense observations but became unstable with sparse observations. The LPFGM showed a more accurate and stable performance than the LPF with both dense and sparse observations. In addition to the relaxation parameter, regulating the resampling frequency and the amplitude of Gaussian kernels was important for the LPFGM. With a spatially inhomogeneous observing network, the LPFGM was superior to the LETKF in sparsely observed regions, where the background ensemble spread and non-Gaussianity were larger. The SPEEDY-based LETKF, LPF, and LPFGM systems are available as open-source software on GitHub (https://github.com/skotsuki/speedy-lpf, last access: 16 November 2022) and can be adapted to various models relatively easily, as in the case of the LETKF.

The Weierstrass–Durand–Kerner root finder is not generally convergent

Finding roots of univariate polynomials is one of the fundamental tasks of numerics, and there is still a wide gap between root finders that are well understood in theory and those that perform well in practice. We investigate the root-finding method of Weierstrass, also known as the Durand–Kerner-method: this is a root finder that tries to approximate all roots of a given polynomial in parallel. This method has been introduced 130 years ago and has since then a good reputation for finding all roots in practice except in obvious cases of symmetry. Nonetheless, very little is known about its global dynamics and convergence properties. We show that the Weierstrass method, like the well-known Newton method, is not generally convergent: there are open sets of polynomials p p of every degree d ≥ 3 d \ge 3 such that the dynamics of the Weierstrass method applied to p p exhibits attracting periodic orbits. Specifically, all polynomials sufficiently close to Z 3 + Z + 175 Z^3 + Z + 175 have attracting cycles of period 4 4 . Here, period 4 4 is minimal: we show that for cubic polynomials, there are no periodic orbits of length 2 2 or 3 3 that attract open sets of starting points. We also establish another convergence problem for the Weierstrass method: for almost every polynomial of degree d ≥ 3 d\ge 3 there are orbits that are defined for all iterates but converge to ∞ \infty ; this is a problem that does not occur for Newton’s method. Our results are obtained by first interpreting the original problem coming from numerical mathematics in terms of higher-dimensional complex dynamics, then phrasing the question in algebraic terms in such a way that we could finally answer it by applying methods from computer algebra. The main innovation here is the translation into an algebraic question, which is amenable to (exact) computational methods close to the limits of current computer algebra systems.

A Multi-Population Mean-Field Game Approach for Large-Scale Agents Cooperative Attack-Defense Evolution in High-Dimensional Environments

As one of the important issues of multi-agent collaboration, the large-scale agents’ cooperative attack–defense evolution requires a large number of agents to make stress-effective strategies to achieve their goals in complex environments. Multi-agent attack and defense in high-dimensional environments (3D obstacle scenarios) present the challenge of being able to accurately control high-dimensional state quantities. Moreover, the large scale makes the dynamic interactions in the attack and defense problems increase dramatically, which, using traditional optimal control techniques, can cause a dimensional explosion. How to model and solve the cooperative attack–defense evolution problem of large-scale agents in high-dimensional environments have become a challenge. We jointly considered energy consumption, inter-group attack and defense, intra-group collision avoidance, and obstacle avoidance in their cost functions. Meanwhile, the high-dimensional state dynamics were used to describe the motion of agents under environmental interference. Then, we formulated the cooperative attack–defense evolution of large-scale agents in high-dimensional environments as a multi-population high-dimensional stochastic mean-field game (MPHD-MFG), which significantly reduced the communication frequency and computational complexity. We tractably solved the MPHD-MFG with a generative-adversarial-network (GAN)-based method using the MFGs’ underlying variational primal–dual structure. Based on our approach, we carried out an integrative experiment in which we analytically showed the fast convergence of our cooperative attack–defense evolution algorithm by the convergence of the Hamilton–Jacobi–Bellman equation’s residual errors. The experiment also showed that a large number of drones can avoid obstacles and smoothly evolve their attack and defense behaviors while minimizing their energy consumption. In addition, the comparison with the baseline methods showed that our approach is advanced.

Synthesized spatiotemporal mode-locking and photonic flywheel in multimode mesoresonators

Dissipative Kerr soliton (DKS) frequency combs—also known as microcombs—have arguably created a new field in cavity nonlinear photonics, with a strong cross-fertilization between theoretical, experimental, and technological research. Spatiotemporal mode-locking (STML) not only adds new degrees of freedom to ultrafast laser technology, but also provides new insights for implementing analogue computers and heuristic optimizers with photonics. Here, we combine the principles of DKS and STML to demonstrate the STML DKS by developing an unexplored ultrahigh-quality-factor Fabry–Pérot (FP) mesoresonator based on graded index multimode fiber (GRIN-MMF). Complementing the two-step pumping scheme with a cavity stress tuning method, we can selectively excite either the eigenmode DKS or the STML DKS. Furthermore, we demonstrate an ultralow noise microcomb that enhances the photonic flywheel performance in both the fundamental comb linewidth and DKS timing jitter. The demonstrated fundamental comb linewidth of 400 mHz and DKS timing jitter of 500 attosecond (averaging times up to 25 μs) represent improvements of 25× and 2.5×, respectively, from the state-of-the-art. Our results show the potential of GRIN-MMF FP mesoresonators as an ideal testbed for high-dimensional nonlinear cavity dynamics and photonic flywheel with ultrahigh coherence and ultralow timing jitter.

Learn one size to infer all: Exploiting translational symmetries in delay-dynamical and spatiotemporal systems using scalable neural networks

We design scalable neural networks adapted to translational symmetries in dynamical systems, capable of inferring untrained high-dimensional dynamics for different system sizes. We train these networks to predict the dynamics of delay-dynamical and spatiotemporal systems for a single size. Then, we drive the networks by their own predictions. We demonstrate that by scaling the size of the trained network, we can predict the complex dynamics for larger or smaller system sizes. Thus, the network learns from a single example and by exploiting symmetry properties infers entire bifurcation diagrams.

Adjoint-based phase reduction analysis of incompressible periodic flows

We establish the theoretical framework for adjoint-based phase reduction analysis for incompressible periodic flows. Through this adjoint-based method, we obtain spatiotemporal phase sensitivity fields through a single pair of forward and backward direct numerical simulations, as opposed to the impulse-based method that requires a very large number of simulations. Phase-based analysis involves perturbation analysis about a periodically varying base state and hence is tailored for the analysis of periodic flows. We formulate the phase description of periodic flows with respect to the potential and vortical perturbations in the flow field. The current phase-reduction analysis can also be implemented consistently in the immersed boundary projection method, which facilitates the analysis over arbitrarily-shaped bodies. We demonstrate the strength of the phase-based analysis for periodic flows over circular cylinder and symmetric airfoils at high incidence angles. The critical regions for phase modification in the cylinder flow are investigated and the locations of flow separation are shown to be the most sensitive regions. Further, the results reveal the influence of the angle of attack and airfoil thickness on the phase-sensitivity distribution of flows over various airfoils. The phase for such flows is defined based on the lift coefficient, and hence is influenced by the vortical structures responsible for lift production. The present framework sheds light on the connection between phase-sensitivity and vortex formation dynamics.

Deep Recurrent Encoder: an end-to-end network to model magnetoencephalography at scale

Understanding how the brain responds to sensory inputs from non-invasive brain recordings like magnetoencephalography (MEG) can be particularly challenging: (i) the high-dimensional dynamics of mass neuronal activity are notoriously difficult to model, (ii) signals can greatly vary across subjects and trials and (iii) the relationship between these brain responses and the stimulus features is non-trivial. These challenges have led the community to develop a variety of preprocessing and analytical (almost exclusively linear) methods, each designed to tackle one of these issues. Instead, we propose to address these challenges through a specific end-to-end deep learning architecture, trained to predict the MEG responses of multiple subjects at once. We successfully test this approach on a large cohort of MEG recordings acquired during a one-hour reading task. Our Deep Recurrent Encoder (DRE) reliably predicts MEG responses to words with a three-fold improvement over classic linear methods. We further describe a simple variable importance analysis to investigate the MEG representations learnt by our model and recover the expected evoked responses to word length and word frequency. Last, we show that, contrary to linear encoders, our model captures modulations of the brain response in relation to baseline fluctuations in the alpha frequency band. The quantitative improvement of the present deep learning approach paves the way to a better characterization of the complex dynamics of brain activity from large MEG datasets.

Neural population clocks: Encoding time in dynamic patterns of neural activity.

The ability to predict and prepare for near- and far-future events is among the most fundamental computations the brain performs. Because of the importance of time for prediction and sensorimotor processing, the brain has evolved multiple mechanisms to tell and encode time across scales ranging from microseconds to days and beyond. Converging experimental and computational data indicate that, on the scale of seconds, timing relies on diverse neural mechanisms distributed across different brain areas. Among the different encoding mechanisms on the scale of seconds, we distinguish between neural population clocks and ramping activity as distinct strategies to encode time. One instance of neural population clocks, neural sequences, represents in some ways an optimal and flexible dynamic regime for the encoding of time. Specifically, neural sequences comprise a high-dimensional representation that can be used by downstream areas to flexibly generate arbitrarily simple and complex output patterns using biologically plausible learning rules. We propose that high-level integration areas may use high-dimensional dynamics such as neural sequences to encode time, providing downstream areas information to build low-dimensional ramp-like activity that can drive movements and temporal expectation. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Joint ocean physics-acoustics inversion and mutual-information based experiment design in realistic ocean fields

Reliable acoustic exploration and navigation in the ocean requires precise knowledge of the environmental fields (e.g. ocean physics, bathymetry, seabed) and acoustic parameters (e.g. source location and frequencies). However, such knowledge is typically incomplete due to the sparse and heterogeneous observations, and to the complex high-dimensional multiscale dynamics. In this work, we use our stochastic Dynamically Orthogonal Parabolic Equation (DO-ParEq) framework to: (i) model the ocean environment uncertainties computed from a large realistic ocean ensemble forecast, and (ii) predict the resulting stochastic acoustic fields and their probability distributions. We then employ new nonlinear Bayesian learning algorithms that use Gaussian Mixture Models to assimilate ocean-acoustic measurements and jointly infer the unknown properties of the ocean environment, the bathymetry, the acoustic field and parameters, and even the model parameterizations themselves. We showcase the developed techniques in realistic data-assimilative sea experiments in the New York Bight Region and the Mediterranean Sea. Finally, we demonstrate the impact of selecting measurement locations using a mutual-information-based algorithm that maximizes the information content in the learned fields based on the measurement constraints.

Hybrid quantum-classical reservoir computing of thermal convection flow

We simulate the nonlinear chaotic dynamics of Lorenz-type models for a classical two-dimensional thermal convection flow with three and eight degrees of freedom by a hybrid quantum-classical reservoir computing model. The high-dimensional quantum reservoir dynamics are established by universal quantum gates that rotate and entangle the individual qubits of the tensor product quantum state. A comparison of the quantum reservoir computing model with its classical counterpart shows that the same prediction and reconstruction capabilities of classical reservoirs with thousands of perceptrons can be obtained by a few strongly entangled qubits. We demonstrate that the mean squared error between model output and ground truth in the test phase of the quantum reservoir computing algorithm increases when the reservoir is decomposed into separable subsets of qubits. Furthermore, the quantum reservoir computing model is implemented on a real noisy IBM quantum computer for up to seven qubits. Our work thus opens the door to model the dynamics of classical complex systems in a high-dimensional phase space effectively with an algorithm that requires a small number of qubits.

Dynamics-disentangled deep learning model for multi-cycle prediction of unsteady flow field

The prediction of an unsteady flow field inherently involving high-dimensional dynamics is challenging. The multi-cycle prediction is especially difficult due to the inevitably accumulated errors over time. A novel deep learning model is proposed in this paper to disentangle the high-dimensional dynamics into two separate attributes that, respectively, represent spatial and temporal dynamics. A continuous mapping of temporal dynamics is subsequently constructed, which alleviates the error accumulation and, thus, contributes to the long-term prediction of the unsteady flow field. The dynamics-disentangled deep learning model (D3LM) processes sequential image data of the unsteady flow field and is constituted by three sub-networks, an encoder introducing a stochastic latent variable to explicitly model the low-order temporal dynamics (called varying attribute herein) and extracting multi-level representations of spatial dynamics (called consistent attribute herein), a decoder integrating the disentangled attributes and generating a future flow field, and a discriminator improving the quality of the predicted flow field. The proposed model is evaluated by two simulated datasets of unsteady flows around a circular cylinder at divergent Reynolds numbers. Benefiting from modeling the continuous distribution of temporal dynamics with the stochastic latent variable, the proposal can give multi-cycle future predictions with high accuracy both spatially and temporally on the two datasets with a small amount of training data. Our work demonstrates the potential practicability of deep learning techniques for modeling the long-term nonlinear laws of unsteady flow.