High-dimensional Dynamical Systems Research Articles

Versatile Control of Fluid-directed Solid Objects Using Multi-task Reinforcement Learning

We propose a learning-based controller for high-dimensional dynamic systems with coupled fluid and solid objects. The dynamic behaviors of such systems can vary across different simulators and the control tasks subject to changing requirements from users. Our controller features high versatility and can adapt to changing dynamic behaviors and multiple tasks without re-training, which is achieved by combining two training strategies. We use meta-reinforcement learning to inform the controller of changing simulation parameters. We further design a novel task representation, which allows the controller to adapt to continually changing tasks via hindsight experience replay. We highlight the robustness and generality of our controller on a row of dynamic-rich tasks, including scooping up solid balls from a water pool, in-air ball acrobatics using fluid spouts, and zero-shot transferring to unseen simulators and constitutive models. In all the scenarios, our controller consistently outperforms the plain multi-task reinforcement-learning baseline.

Asymptotic Rules of Equilibrium Desingularization

A local bifurcation analysis of a high-dimensional dynamical system dxdt=f(x) is performed using a good deformation of the polynomial mapping P:Cn→Cn. This theory is used to construct geometric aspects of the resolution of multiple zeros of the polynomial vector field P(x). Asymptotic bifurcation rules are derived from Grothendieck’s theory of residuals. Following the Coxeter–Dynkin classification, the singularity graph is constructed. A detailed study of three types of multidimensional mappings with a large symmetry group has been carried out, namely: 1. A linear singularity (behaves similarly to a one-dimensional complex analysis theory); 2. The lattice singularity (generalized the linear and resembling regular crystal growth models); 3. The fan-shaped singularity (can be split radially like nuclear fission and fusion models).

Near-optimal control of dynamical systems with neural ordinary differential equations

Optimal control problems naturally arise in many scientific applications where one wishes to steer a dynamical system from an initial state x 0 to a desired target state in finite time T. Recent advances in deep learning and neural network–based optimization have contributed to the development of numerical methods that can help solve control problems involving high-dimensional dynamical systems. In particular, the framework of neural ordinary differential equations (neural ODEs) provides an efficient means to iteratively approximate continuous-time control functions associated with analytically intractable and computationally demanding control tasks. Although neural ODE controllers have shown great potential in solving complex control problems, the understanding of the effects of hyperparameters such as network structure and optimizers on learning performance is still very limited. Our work aims at addressing some of these knowledge gaps to conduct efficient hyperparameter optimization. To this end, we first analyze how truncated and non-truncated backpropagation through time affect both runtime performance and the ability of neural networks to learn optimal control functions. Using analytical and numerical methods, we then study the role of parameter initializations, optimizers, and neural-network architecture. Finally, we connect our results to the ability of neural ODE controllers to implicitly regularize control energy.

Blue-Sky Catastrophic Bifurcations Behind Emergence and Disappearance of Quasiperiodic Rubbing Oscillations in a Piecewise Smooth Rotor–Stator System

In this paper, an approach combining the methods of multigrid Point Mapping under Cell Reference and Stagger-and-step is proposed, which can efficiently determine both attractors and unstable saddle-type invariant sets in higher-dimensional dynamical systems. Then, the bifurcations around quasiperiodic partial rub oscillations of a piecewise smooth rotor–stator system are investigated, which are categorized into two classes depending on whether Nonlinear Normal Modes (NNMs) exist or not. It is found that the quasiperiodic rubbing oscillations, within the parameter region with an NNM, emerge through a Hopf bifurcation from a periodic rubbing oscillation and disappear through a collision with a saddle torus with an accompanying variation of system parameters. Within the parameter region without NNM, the bifurcation scenarios of two sequences of quasiperiodic rubbing oscillations, which emerge subsequently with the increasing of the rotating speed, are similar but somewhat complicated. That is, the quasiperiodic rubbing oscillation emerges through an inverse doubling of torus from a chaotic attractor, which is produced from a chaotic saddle after chaotic crisis. The quasiperiodic rubbing oscillation disappears through a collision between the stable and the unstable quasiperiodic tori. The insight into blue-sky catastrophic bifurcations in the quasiperiodic rubbing oscillations of the piecewise smooth rotor–stator rubbing system is thus obtained through this study.

First-passage probability estimation of high-dimensional nonlinear stochastic dynamic systems by a fractional moments-based mixture distribution approach

First-passage probability estimation of high-dimensional nonlinear stochastic dynamic systems is a significant task to be solved in many science and engineering fields, but remains still an open challenge. The present paper develops a novel approach, termed ‘fractional moments-based mixture distribution’, to address such challenge. This approach is implemented by capturing the extreme value distribution (EVD) of the system response with the concepts of fractional moment and mixture distribution. In our context, the fractional moment itself is by definition a high-dimensional integral with a complicated integrand. To efficiently compute the fractional moments, a parallel adaptive sampling scheme that allows for sample size extension is developed using the refined Latinized stratified sampling (RLSS). In this manner, both variance reduction and parallel computing are possible for evaluating the fractional moments. From the knowledge of low-order fractional moments, the EVD of interest is then expected to be reconstructed. Based on introducing an extended inverse Gaussian distribution and a log extended skew-normal distribution, one flexible mixture distribution model is proposed, where its fractional moments are derived in analytic form. By fitting a set of fractional moments, the EVD can be recovered via the proposed mixture model. Accordingly, the first-passage probabilities under different thresholds can be obtained from the recovered EVD straightforwardly. The performance of the proposed method is verified by three examples consisting of two test examples and one engineering problem.

Learning spatiotemporal chaos using next-generation reservoir computing.

Forecasting the behavior of high-dimensional dynamical systems using machine learning requires efficient methods to learn the underlying physical model. We demonstrate spatiotemporal chaos prediction using a machine learning architecture that, when combined with a next-generation reservoir computer, displays state-of-the-art performance with a computational time 10- 10 times faster for training process and training data set ∼ 10 times smaller than other machine learning algorithms. We also take advantage of the translational symmetry of the model to further reduce the computational cost and training data, each by a factor of ∼10.

Globally-evolving-based generalized density evolution equation for nonlinear systems involving randomness from both system parameters and excitations

It has long been one of the main challenges in science and engineering to capture the probabilistic response of high-dimensional nonlinear stochastic dynamic systems involving double randomness, i.e. randomness in both system parameters and excitations. For this purpose, a globally-evolving-based generalized density evolution equation (GE-GDEE) is established. Generally, for a multi-dimensional nonlinear system involving double randomness, if one single physical quantity as a response of the system is of interest, a GE-GDEE, as a two-dimensional partial differential equation (PDE) governing the probability density function (PDF), can be derived. The effective drift coefficients, which represent the physically driving force function in the GE-GDEE, can be determined based on the data from some representative deterministic dynamic analyses of the underlying physical system. A new estimator for effective drift coefficients is developed based on the vine copulas. Once the effective drift coefficients are determined, the GE-GDEE can be solved to capture the probability distributions of the quantities of interest. Several numerical examples, including linear and nonlinear multi-degree-of-freedom (MDOF) systems subjected to white noise or non-stationary earthquake ground motions, are presented to verify the effectiveness of the proposed method. Finally, problems for future investigations are discussed.

Heteroclinic Cycles Generated by Piecewise Affine Systems in ℝn

It is essential yet challenging to investigate heteroclinic trajectories in chaotic dynamics of high-dimensional systems. This paper presents an efficient approach for precise prediction of heteroclinic cycles in [Formula: see text]-dimensional ([Formula: see text]) piecewise affine systems. Moreover, it systemically investigates different types of heteroclinic cycles in the considered systems, and proposes the corresponding existence conditions with rigorous proofs by analyzing the dynamics of stable and unstable manifolds. In addition, three numerical examples with both a heteroclinic cycle and a chaotic set are provided to verify the established results.

A Stochastic Approximation-Langevinized Ensemble Kalman Filter Algorithm for State Space Models with Unknown Parameters

Inference for high-dimensional, large scale and long series dynamic systems is a challenging task in modern data science. The existing algorithms, such as particle filter or sequential importance sampler, do not scale well to the dimension of the system and the sample size of the dataset, and often suffers from the sample degeneracy issue for long series data. The recently proposed Langevinized ensemble Kalman filter (LEnKF) addresses these difficulties in a coherent way. However, it cannot be applied to the case that the dynamic system contains unknown parameters. This paper proposes the so-called stochastic approximation-LEnKF for jointly estimating the states and unknown parameters of the dynamic system, where the parameters are estimated on the fly based on the state variables simulated by the LEnKF under the framework of stochastic approximation Markov chain Monte Carlo (MCMC). Under mild conditions, we prove its consistency in parameter estimation and ergodicity in state variable simulations. The proposed algorithm can be used in uncertainty quantification for long series, large scale, and high-dimensional dynamic systems. Numerical results indicate its superiority over the existing algorithms. We employ the proposed algorithm in state-space modeling of the sea surface temperature with a long short term memory (LSTM) network, which indicates its great potential in statistical analysis of complex dynamic systems encountered in modern data science. A supplement to the paper is available online.

Proper Orthogonal Decomposition Analysis Reveals Cell Migration Directionality During Wound Healing.

A proper orthogonal decomposition (POD) order reduction method was implemented to reduce the full high dimensional dynamical system associated with a wound healing cell migration assay to a low-dimensional approximation that identified the prevailing cell trajectories. The POD analysis generated POD modes that were representative of the prevalent cell trajectories. The shapes of the POD modes depended on the location of the cells with respect to the wound and exposure to disturbed (DF) or undisturbed (UF) fluid flow where the net flow was in the antegrade direction with a retrograde component or fully antegrade, respectively. For DF and UF, the POD modes of the downstream cells identified trajectories that moved upstream against the flow, while upstream POD modes exhibited sideways cell migrations. In the absence of flow, no major shape differences were observed in the POD modes on either side of the wound. The POD modes also served to reconstruct the cell migration of individual cells. With as few as three modes, the predominant cell migration trajectories were reconstructed, while the level of accuracy increased with the inclusion of more POD modes. The POD order reduction method successfully identified the predominant cell migratory trajectories under static and varying pulsatile fluid flow conditions serving as a first step in the development of artificial intelligence models of cell migration in disease and development.

Recurrent neural networks enable design of multifunctional synthetic human gut microbiome dynamics.

Predicting the dynamics and functions of microbiomes constructed from the bottom-up is a key challenge in exploiting them to our benefit. Current models based on ecological theory fail to capture complex community behaviors due to higher order interactions, do not scale well with increasing complexity and in considering multiple functions. We develop and apply a long short-term memory (LSTM) framework to advance our understanding of community assembly and health-relevant metabolite production using a synthetic human gut community. A mainstay of recurrent neural networks, the LSTM learns a high dimensional data-driven non-linear dynamical system model. We show that the LSTM model can outperform the widely used generalized Lotka-Volterra model based on ecological theory. We build methods to decipher microbe-microbe and microbe-metabolite interactions from an otherwise black-box model. These methods highlight that Actinobacteria, Firmicutes and Proteobacteria are significant drivers of metabolite production whereas Bacteroides shape community dynamics. We use the LSTM model to navigate a large multidimensional functional landscape to design communities with unique health-relevant metabolite profiles and temporal behaviors. In sum, the accuracy of the LSTM model can be exploited for experimental planning and to guide the design of synthetic microbiomes with target dynamic functions.

Information-theoretic formulation of dynamical systems: Causality, modeling, and control

The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of information in the states of the system. Within this framework, causality is quantified by the information flux among the variables of interest in the dynamical system. Reduced-order modeling is posed as a problem related to the conservation of information in which models aim at preserving the maximum amount of relevant information from the original system. Similarly, control theory is cast in information-theoretic terms by envisioning the tandem sensor-actuator as a device reducing the unknown information of the state to be controlled. The new formulation is used to address three problems about the causality, modeling, and control of turbulence, which stands as a primary example of a chaotic, high-dimensional dynamical system. The applications include the causality of the energy transfer in the turbulent cascade, subgrid-scale modeling for large-eddy simulation, and flow control for drag reduction in wall-bounded turbulence.

Time-varying Group Lasso Granger Causality Graph for High Dimensional Dynamic system

Feature selection is a crucial preprocessing step in data analysis and machine learning. Since causal relationships imply the underlying mechanism of a system, causality-based feature selection methods have gradually attracted great attentions. For a high dimensional system undergoing dynamic transformation, because of the non-stationarity and sample scarcity, modeling the causal structure among these features is difficult. In this paper, we propose a time-varying Granger causal networks to capture the causal relations underlying high dimensional time-varying vector autoregressive models with high order lagged dependence. A kernel reweighted group lasso method is proposed, which overcomes the limitations of sample scarcity and transforms the problem of Granger causal structural learning into a group variable selection problem. The asymptotic consistency of the proposed algorithm is proved. We apply the time-varying Granger causal networks to simulation experiments and real data in the financial market. The study demonstrates that the method provides an efficient tool to detect changes and analysis characters of causal dependency structure in network evolution.

Graph covers of higher dimensional dynamical systems

We generalize the notion of an inverse sequence of graph covers from the zero-dimensional dynamical systems [8] to any dynamical system.

Surrogate modelling for an aircraft dynamic landing loads simulation using an LSTM AutoEncoder-based dimensionality reduction approach

Surrogate modelling can alleviate the computational burden of design activities as they rely on multiple evaluations of high-fidelity models. However, the learning task can be adversely affected by the high-dimensionality of the system, complex non-linearities and temporal dependencies, leading to an inaccurate surrogate model. In this paper we present an innovative dual-phase Long-Short Term Memory (LSTM) Autoencoder-based surrogate model applied in an industrial context for the prediction of aircraft dynamic landing response over time, conditioned by an exogenous set of design parameters. The LSTM-Autoencoder is adopted as a dimensionality-reduction tool that extracts the temporal features and the nonlinearities of the high-dimensional dynamical system response, and learns a low-dimensional representation of it. Then, a Fully Connected Neural Network is trained to learn the simplified relationship between the input parameters and the reduced representation of the output. For our application, the results demonstrate that our LSTM-AE based model outperforms both Principal Component Analysis and Convolutional-Autoencoder based surrogate models, in predicting the parameters-dependent high-dimensional temporal system response.

Experimental Implementation of Energy Closure Analysis for Reduced Order Modeling

Abstract Reduced order models (ROMs) provide an efficient, kinematically condensed representation of computationally expensive high-dimensional dynamical systems; however, their accuracy depends crucially on the accurate estimation of their dimension. We here demonstrate how the energy closure criterion, developed in our prior work, can be experimentally implemented to accurately estimate the dimension of ROMs obtained using the proper orthogonal decomposition (POD). We examine the effect of using discrete data with and without measurement noise, as will typically be gathered in an experiment or numerical simulation, on estimating the degree of energy closure on a candidate reduced subspace. To this end, we used a periodically kicked Euler–Bernoulli beam with Rayleigh damping as the model system and studied ROMs obtained by applying POD to discrete displacement field data obtained from simulated numerical experiments. An improved method for quantifying the degree of energy closure is presented: the convergence of energy input to or dissipated from the system is obtained as a function of the subspace dimension, and the dimension capturing a predefined percentage of either energy is selected as the ROM dimension. This method was found to be more robust to data discretization error and measurement noise. The data-processing necessary for the experimental application of energy closure analysis is discussed in detail. We show how ROMs formulated from the simulated data using our approach accurately capture the dynamics of the beam for different sets of parameter values.

Non-hyperbolicity at large scales of a high-dimensional chaotic system

The dynamics of many important high-dimensional dynamical systems are both chaotic and complex, meaning that strong reducing hypotheses are required to understand the dynamics. The highly influential chaotic hypothesis of Gallavotti and Cohen states that the large-scale dynamics of high-dimensional systems are effectively uniformly hyperbolic, which implies many felicitous statistical properties. We obtain direct and reliable numerical evidence, contrary to the chaotic hypothesis, of the existence of non-hyperbolic large-scale dynamical structures in a mean-field coupled system. To do this, we reduce the system to its thermodynamic limit, which we approximate numerically with a Chebyshev basis transfer operator discretization. This enables us to obtain a high-precision estimate of a homoclinic tangency, implying a failure of uniform hyperbolicity. Robust non-hyperbolic behaviour is expected under perturbation. As a result, the chaotic hypothesis should not be a priori assumed to hold in all systems, and a better understanding of the domain of its validity is required.

Unsteady flow prediction from sparse measurements by compressed sensing reduced order modeling

Prediction of complex fluid flows from sparse and noisy sensor measurements is widely applied to many engineering fields. In the present study, a novel compressed sensing reduced-order modeling framework has been proposed to predict unsteady flow fields from sparse and noisy sensor observations, which includes offline learning and online forecasting. In the offline learning stage, the Long Short Term Memory (LSTM) model is used for modeling the dynamic evolution of the sensor signals. Moreover, the sparsity-promoting Dynamic Mode Decomposition (DMD) algorithm is applied to extract spatio-temporal coherent structures from high-dimensional fluid dynamic system and choose optimal DMD modes for flow reconstruction. In order to establish the correlation between sensor space and feature-based coherent structures space, the Deep Neural Network (DNN) is employed. In the online forecasting stage, the trained framework is applied to predict the flow fields from limited sensor measurements in the actual experiment setup. In the current work, the proposed framework is applied in two numerical examples, including the flow past a circle cylinder at Re=100 characterized by laminar flow and at Re=3900 characterized by turbulent flow. It is shown that the proposed framework performs accurately and robustly in both prediction of sensor measurements and unsteady flow fields. Especially, the important physical features and quantities can be well maintained for both laminar and turbulent flow, even if high level noise is involved. Therefore, the proposed framework is a promising tool for sparse representation from complex flow fields in realistic experiments.

Transient chaos in bidirectional encoder representations from transformers

Language is an outcome of our complex and dynamic human-interactions and the technique of natural language processing (NLP) is hence built on human linguistic activities. Bidirectional Encoder Representations from Transformers (BERT) has recently gained its popularity by establishing the state-of-the-art scores in several NLP benchmarks. A Lite BERT (ALBERT) is literally characterized as a lightweight version of BERT, in which the number of BERT parameters is reduced by repeatedly applying the same neural network called Transformer's encoder layer. By pre-training the parameters with a massive amount of natural language data, ALBERT can convert input sentences into versatile high-dimensional vectors potentially capable of solving multiple NLP tasks. In that sense, ALBERT can be regarded as a well-designed high-dimensional dynamical system whose operator is the Transformer's encoder, and essential structures of human language are thus expected to be encapsulated in its dynamics. In this study, we investigated the embedded properties of ALBERT to reveal how NLP tasks are effectively solved by exploiting its dynamics. We thereby aimed to explore the nature of human language from the dynamical expressions of the NLP model. Our short-term analysis clarified that the pre-trained model stably yields trajectories with higher dimensionality, which would enhance the expressive capacity required for NLP tasks. Also, our long-term analysis revealed that ALBERT intrinsically shows transient chaos, a typical nonlinear phenomenon showing chaotic dynamics only in its transient, and the pre-trained ALBERT model tends to produce the chaotic trajectory for a significantly longer time period compared to a randomly-initialized one. Our results imply that local chaoticity would contribute to improving NLP performance, uncovering a novel aspect in the role of chaotic dynamics in human language behaviors.

Estimating covariant Lyapunov vectors from data.

Covariant Lyapunov vectors characterize the directions along which perturbations in dynamical systems grow. They have also been studied as predictors of critical transitions and extreme events. For many applications, it is necessary to estimate these vectors from data since model equations are unknown for many interesting phenomena. We propose an approach for estimating covariant Lyapunov vectors based on data records without knowing the underlying equations of the system. In contrast to previous approaches, our approach can be applied to high-dimensional datasets. We demonstrate that this purely data-driven approach can accurately estimate covariant Lyapunov vectors from data records generated by several low- and high-dimensional dynamical systems. The highest dimension of a time series from which covariant Lyapunov vectors are estimated in this contribution is 128.