High-dimensional Dynamical Systems Research Articles

Tipping point and noise-induced transients in ecological networks.

A challenging and outstanding problem in interdisciplinary research is to understand the interplay between transients and stochasticity in high-dimensional dynamical systems. Focusing on the tipping-point dynamics in complex mutualistic networks in ecology constructed from empirical data, we investigate the phenomena of noise-induced collapse and noise-induced recovery. Two types of noise are studied: environmental (Gaussian white) noise and state-dependent demographic noise. The dynamical mechanism responsible for both phenomena is a transition from one stable steady state to another driven by stochastic forcing, mediated by an unstable steady state. Exploiting a generic and effective two-dimensional reduced model for real-world mutualistic networks, we find that the average transient lifetime scales algebraically with the noise amplitude, for both environmental and demographic noise. We develop a physical understanding of the scaling laws through an analysis of the mean first passage time from one steady state to another. The phenomena of noise-induced collapse and recovery and the associated scaling laws have implications for managing high-dimensional ecological systems.

Deep neural networks for nonlinear model order reduction of unsteady flows

Unsteady fluid systems are nonlinear high-dimensional dynamical systems that may exhibit multiple complex phenomena in both time and space. Reduced Order Modeling (ROM) of fluid flows has been an active research topic in the recent decade with the primary goal to decompose complex flows into a set of features most important for future state prediction and control, typically using a dimensionality reduction technique. In this work, a novel data-driven technique based on the power of deep neural networks for ROM of the unsteady fluid flows is introduced. An autoencoder network is used for nonlinear dimension reduction and feature extraction as an alternative for singular value decomposition (SVD). Then, the extracted features are used as an input for a long short-term memory (LSTM) network to predict the velocity field at future time instances. The proposed autoencoder-LSTM method is compared with non-intrusive reduced order models based on dynamic mode decomposition (DMD) and proper orthogonal decomposition. Moreover, an autoencoder-DMD algorithm is introduced for ROM, which uses the autoencoder network for dimensionality reduction rather than SVD rank truncation. The results show that the autoencoder-LSTM method is considerably capable of predicting fluid flow evolution, where higher values for the coefficient of determination R2 are obtained using autoencoder-LSTM compared to other models.

Application of adaptive multilevel splitting to high-dimensional dynamical systems

Stochastic nonlinear dynamical systems can undergo rapid transitions relative to the change in their forcing, for example due to the occurrence of multiple equilibrium solutions for a specific interval of parameters. In this paper, we modify one of the methods developed to compute probabilities of such transitions, Trajectory-Adaptive Multilevel Sampling (TAMS), to be able to apply it to high-dimensional systems. The key innovation is a projected time-stepping approach, which leads to a strong reduction in computational costs, in particular memory usage. The performance of this new implementation of TAMS is studied through an example of the collapse of the Atlantic Ocean Circulation.

Backward Reachability Analysis for Nonlinear Dynamical Systems via Pseudospectral Method

In this paper, we propose a new approach to solving the backward reachability problem for nonlinear dynamical systems. Previously, this class of problems has been studied within frameworks of optimal control and zero-sum differential games, where a backward reachable set can be expressed as the zero sublevel set of the value function that can be characterized by solving the Hamilton-Jacobi-Bellman (HJB) partial differential equation (PDE). In many cases, however, a high computational cost is incurred in numerically solving such HJB PDEs due to the curse of dimensionality. We use the pseudospectral method to convert the associated optimal control problem into nonlinear programs (NLPs). We then show that the zero sublevel set obtained by the optimal cost of the NLP is the corresponding backward reachable set. Note that our approach does not require solving complex HJB PDEs. Therefore, it can reduce computation time and handle high-dimensional dynamical systems, compared with the numerical software package developed by I. Mitchell, which has been used widely in the literature to obtain backward reachable sets by solving HJB equations. We provide several examples to validate the effectiveness of the proposed approach.

Enhancing subdivision technique with an adaptive interpolation sampling method for global attractors of nonlinear dynamical systems

The cell mapping method is a prominent one for global analysis of nonlinear dynamical systems, with which multiple invariant sets can be obtained. However, it is a continuous challenge to enhance the efficiency of the cell mapping method, especially when dealing with high-dimensional nonlinear dynamical systems. In this paper, the subdivision technique commonly used in the cell mapping method is incorporated with an interpolation sampling method, which can further enhance the efficiency over the set-oriented method with subdivision for global attractors of nonlinear dynamical systems. In the present method, a new lattice of interpolating nodes is adopted to fit into the nesting structures of the subdivided cells, portions of which are sequentially removed when resolution goes from low to high. An improved interpolation method is developed to obtain one-step sample mappings, instead of integrations when error bounds are met, in the process of subdivision iterations. Furthermore, a Hash table is introduced in order to fast search and locate the coordinates of cells that cover the invariant sets. Three examples of nonlinear dynamical systems are presented to demonstrate the enhancement in efficiency and effectiveness of the proposed method, indicating that a computational cost is one half down to one sixth of the previous methods.

Path length statistics in reservoir computers.

Because reservoir computers are high dimensional dynamical systems, designing a good reservoir computer is difficult. In many cases, the designer must search a large nonlinear parameter space, and each step of the search requires simulating the full reservoir computer. In this work, I show that a simple statistic based on the mean path length between nodes in the reservoir computer is correlated with better reservoir computer performance. The statistic predicts the diversity of signals produced by the reservoir computer, as measured by the covariance matrix of the reservoir computer. This statistic by itself is not sufficient to predict reservoir computer performance because not only must the reservoir computer produce a diverse set of signals, it must be well matched to the training signals. Nevertheless, this path length statistic allows the designer to eliminate some network configurations from consideration without having to actually simulate the reservoir computer, reducing the complexity of the design process.

A new scalable algorithm for computational optimal control under uncertainty

We address the design of optimal control strategies for high-dimensional stochastic dynamical systems. Such systems may be deterministic nonlinear systems evolving from random initial states, or systems driven by random parameters or random noise. The objective is to provide a validated new computational capability for optimal control which will be achieved more efficiently than current state-of-the-art methods. The new framework utilizes direct single or multi-shooting discretization, and is based on efficient vectorized gradient computation with adaptable memory management. The algorithm is demonstrated to be scalable to high-dimensional nonlinear control systems with random initial condition and unknown parameters. Numerical applications are presented and discussed for stochastic path planning problems involving models of unmanned aerial and ground vehicles, and for distributed control of a nonlinear advection-reaction-diffusion equation.

Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems

Data-based discovery of effective, coarse-grained (CG) models of high-dimensional dynamical systems presents a unique challenge in computational physics and particularly in the context of multiscale problems. The present paper offers a data-based, probabilistic perspective that enables the quantification of predictive uncertainties. One of the outstanding problems has been the introduction of physical constraints in the probabilistic machine learning objectives. The primary utility of such constraints stems from the undisputed physical laws such as conservation of mass, energy etc. that they represent. Furthermore and apart from leading to physically realistic predictions, they can significantly reduce the requisite amount of training data which for high-dimensional, multiscale systems are expensive to obtain (Small Data regime). We formulate the coarse-graining process by employing a probabilistic state-space model and account for the aforementioned equality constraints as virtual observables in the associated densities. We demonstrate how deep neural nets in combination with probabilistic inference tools can be employed to identify the coarse-grained variables and their evolution model without ever needing to define a fine-to-coarse (restriction) projection and without needing time-derivatives of state variables.We advocate a sparse Bayesian learning perspective which avoids overfitting and reveals the most salient features in the CG evolution law. The formulation adopted enables the quantification of a crucial, and often neglected, component in the CG process, i.e. the predictive uncertainty due to information loss. Furthermore, it is capable of reconstructing the evolution of the full, fine-scale system and therefore the observables of interest need not be selected a priori. We demonstrate the efficacy of the proposed framework by applying it to systems of interacting particles and a series of images of a nonlinear pendulum. In both cases we identify the underlying coarse dynamics and can generate extrapolative predictions including the forming and propagation of a shock for the particle systems and a stable trajectory in the phase space for the pendulum.

A General Framework to Increase Safety of Learning Algorithms for Dynamical Systems Based on Region of Attraction Estimation

Although the state-of-the-art learning approaches exhibit impressive results for dynamical systems, only a few applications on real physical systems have been presented. One major impediment is that the intermediate policy during the training procedure may result in behaviors that are not only harmful to the system itself but also to the environment. In essence, imposing safety guarantees for learning algorithms is vital for autonomous systems acting in the real world. In this article, we propose a computationally effective and general safe learning framework, specifically for complex dynamical systems. With a proper definition of the safe region, a supervisory control strategy, which switches the actions applied on the system between the learning-based controller and a predefined corrective controller, is given. A simplified system facilitates the estimation of the safe region for the high-dimensional dynamical system. During the learning phase, the belief of the safe region is updated with the actual execution results of the corrective controller, which in turn enables the learning-based controller to have more freedom in choosing its actions. Two examples are given to demonstrate the performance of the proposed framework, one simple inverted pendulum to illustrate the online adaptation method, and one quadcopter control task to show the overall performance.

Response and reliability analysis of a high-dimensional stochastic system

The efficiency of the probability density evolution method (PDEM) is improved in this paper by embedding the Kullback–Leibler (K–L) relative sensitivity in the response analysis of a stochastic dynamic system. The response reliability obtained and the probability density function of the response peaks are used for ranking to get a reduced set of random variables for the PDEM analysis. The need of complicated point selection technique with the high-dimensional uncertain variables is therefore alleviated. The proposed method is illustrated with the response analysis of a random crowd-structure system where the load randomness is considered. The acceleration response induced by the presence of the crowd is evaluated with the proposed method. Results obtained highlight the significant improvements in the computation efficiency of the probabilistic response analysis of a high-dimensional dynamic system.

Semi-Implicit and Semi-Explicit Adams-Bashforth-Moulton Methods

Multistep integration methods are widespread in the simulation of high-dimensional dynamical systems due to their low computational costs. However, the stability of these methods decreases with the increase of the accuracy order, so there is a known room for improvement. One of the possible ways to increase stability is implicit integration, but it consequently leads to sufficient growth in computational costs. Recently, the development of semi-implicit techniques achieved great success in the construction of highly efficient single-step ordinary differential equations (ODE) solvers. Thus, the development of multistep semi-implicit integration methods is of interest. In this paper, we propose the simple solution to increase the numerical efficiency of Adams-Bashforth-Moulton predictor-corrector methods using semi-implicit integration. We present a general description of the proposed methods and explicitly show the superiority of ODE solvers based on semi-implicit predictor-corrector methods over their explicit and implicit counterparts. To validate this, performance plots are given for simulation of the van der Pol oscillator and the Rossler chaotic system with fixed and variable stepsize. The obtained results can be applied in the development of advanced simulation software.

Path Integral Policy Improvement With Population Adaptation.

Path integral policy improvement (PI2) is known to be an efficient reinforcement learning algorithm, particularly, if the target system is a high-dimensional dynamical system. However, PI2, and its existing extensions, have adjustable parameters, on which the efficiency depends significantly. This article proposes an extension of PI2 that adjusts all of the critical parameters automatically. Motion acquisition tasks for three different types of simulated legged robots were performed to test the efficacy of the proposed algorithm. The results show that the proposed method cannot only eliminate the burden on the user to set the parameters appropriately but also improve the optimization performance significantly. For one of the acquired motions, a real robot experiment was conducted to show the validity of the motion.

A highly-efficient method for stationary response of multi-degree-of-freedom nonlinear stochastic systems

Analytical and numerical studies of multi-degree-of-freedom (MDOF) nonlinear stochastic or deterministic dynamic systems have long been a technical challenge. This paper presents a highly-efficient method for determining the stationary probability density functions (PDFs) of MDOF nonlinear systems subjected to both additive and multiplicative Gaussian white noises. The proposed method takes advantages of the sufficient conditions of the reduced Fokker-Planck-Kolmogorov (FPK) equation when constructing the trial solution. The assumed solution consists of the analytically constructed trial solutions satisfying the sufficient conditions and an exponential polynomial of the state variables, and delivers a high accuracy of the solution because the analytically constructed trial solutions capture the main characteristics of the nonlinear system. We also make use of the concept from the data-science and propose a symbolic integration over a hypercube to replace the numerical integrations in higher-dimensional space, which has been regarded as the insurmountable difficulty in the classical method of weighted residuals or stochastic averaging for high-dimensional dynamic systems. Three illustrative examples of MDOF nonlinear systems are analyzed in detail. The accuracy of the numerical results is validated by comparison with the Monte Carlo simulation (MCS) or the available exact solution. Furthermore, we also show the substantial gain in the computational efficiency of the proposed method compared with the MCS.

The Applications of Order Reduction Methods in Nonlinear Dynamic Systems

Two different order reduction methods of the deterministic and stochastic systems are discussed in this paper. First, the transient proper orthogonal decomposition (T-POD) method is introduced based on the high-dimensional nonlinear dynamic system. The optimal order reduction conditions of the T-POD method are provided by analyzing the rotor-bearing system with pedestal looseness fault at both ends. The efficiency of the T-POD method is verified via comparing with the results of the original system. Second, the polynomial dimensional decomposition (PDD) method is applied to the 2 DOFs spring system considering the uncertain stiffness to study the amplitude-frequency response. The numerical results obtained by the PDD method agree well with the Monte Carlo simulation (MCS) method. The results of the PDD method can approximate to MCS better with the increasing of the polynomial order. Meanwhile, the Uniform-Legendre polynomials can eliminate perturbation of the PDD method to a certain extent via comparing it with the Gaussian-Hermite polynomials.

System Identification of High-Dimensional Linear Dynamical Systems With Serially Correlated Output Noise Components

We consider identification of linear dynamical systems comprising of high-dimensional signals, where the output noise components exhibit strong serial, and cross-sectional correlations. Although such settings occur in many modern applications, such dependency structure has not been fully incorporated in existing approaches in the literature. In this paper, we explicitly incorporate the dependency structure present in the output noise through lagged values of the observed multivariate signals. We formulate a constrained optimization problem to identify the space spanned by the latent states, and the transition matrices of the lagged values simultaneously , wherein the constraints reflect the low rank nature of the state information, and the sparsity of the transition matrices. We establish theoretical properties of the estimators, and introduce an easy-to-implement computational procedure for empirical applications. The performance of the proposed approach, and the implementation procedure is evaluated on synthetic data, and compared with competing approaches, and further illustrated on a data set involving weekly stock returns of 75 US large financial institutions for the 2001–2017 period.

Diffusion Maps Kalman Filter for a Class of Systems With Gradient Flows

In this paper, we propose a non-parametric method for state estimation of high-dimensional nonlinear stochastic dynamical systems, which evolve according to gradient flows with isotropic diffusion. We combine diffusion maps, a manifold learning technique, with a linear Kalman filter and with concepts from Koopman operator theory. More concretely, using diffusion maps, we construct data-driven virtual state coordinates, which linearize the system model. Based on these coordinates, we devise a data-driven framework for state estimation using the Kalman filter. We demonstrate the strengths of our method with respect to both parametric and non-parametric algorithms in three tracking problems. In particular, applying the approach to actual recordings of hippocampal neural activity in rodents directly yields a representation of the position of the animals. We show that the proposed method outperforms competing non-parametric algorithms in the examined stochastic problem formulations. Additionally, we obtain results comparable to classical parametric algorithms, which, in contrast to our method, are equipped with model knowledge.

Sampling Low-Dimensional Markovian Dynamics for Preasymptotically Recovering Reduced Models from Data with Operator Inference

This work introduces a method for learning low-dimensional models from data of high-dimensional black-box dynamical systems. The novelty is that the learned models are exactly the reduced models that are traditionally constructed with classical projection-based model reduction techniques. Thus, the proposed approach learns models that are guaranteed to have the well-studied properties of reduced models known from model reduction, without requiring full knowledge of the governing equations and without requiring the operators of the high-dimensional systems. The key ingredient is a new data sampling scheme to obtain re-projected trajectories of high-dimensional systems that correspond to Markovian dynamics in low-dimensional subspaces. The exact recovery of reduced models from these re-projected trajectories is guaranteed preasymptotically under certain conditions for finite amounts of data and for a large class of systems with polynomial nonlinear terms. Numerical results demonstrate that the low-dimensional models learned with the proposed approach match reduced models from traditional model reduction up to numerical errors in practice. The numerical results further indicate that low-dimensional models fitted to re-projected trajectories are predictive even in situations where models fitted to trajectories without re-projection are inaccurate and unstable.

Photonic reservoir computing based on nonlinear wave dynamics at microscale

High-dimensional nonlinear dynamical systems, including neural networks, can be utilized as computational resources for information processing. In this sense, nonlinear wave systems are good candidates for such computational resources. Here, we propose and numerically demonstrate information processing based on nonlinear wave dynamics in microcavity lasers, i.e., optical spatiotemporal systems at microscale. A remarkable feature is its ability of high-dimensional and nonlinear mapping of input information to the wave states, enabling efficient and fast information processing at microscale. We show that the computational capability for nonlinear/memory tasks is maximized at the edge of dynamical stability. Moreover, we show that computational capability can be enhanced by applying a time-division multiplexing technique to the wave dynamics. Thus, the computational potential of the wave dynamics can sufficiently be extracted even when the number of detectors to monitor the wave states is limited. In addition, we discuss the merging of optical information processing with optical sensing, revealing a novel method for model-free sensing by using a microcavity reservoir as a sensing element. These results pave a way for on-chip photonic computing with high-dimensional dynamics and a model-free sensing method.

Power System Time Domain Simulation Using a Differential Transformation Method

This paper proposes a novel approach for power system dynamic simulation based on the differential transformation (DT). The DT is introduced to study power systems as high-dimensional nonlinear dynamical systems for the first time, and is able to avoid computations of high-order derivatives with nonlinear differential equations by its transform rules. This paper, first, proposes and proves several new transform rules for generic nonlinear functions that often appear in power system models, and then uses these rules to transform representative power system models such as the synchronous machine model with trigonometric functions and the exciter model with exponential and square root functions. This paper also designs a DT-based simulation scheme that allows significantly prolonged time steps to reduce simulation time compared to a traditional numerical approach. The numerical stability, accuracy, and time performance of the proposed new DT-based simulation approach are compared with widely used numerical methods on the IEEE 39-bus system and Polish 2383-bus system.

Smoothing and parameter estimation by soft-adherence to governing equations

The analysis of high-dimensional dynamical systems generally requires the integration of simulation data with experimental measurements. Experimental data often has substantial amounts of measurement noise that compromises the ability to produce accurate dimensionality reduction, parameter estimation, reduced order models, and/or balanced models for control. Data assimilation attempts to overcome the deleterious effects of noise by producing a set of algorithms for state estimation from noisy and possibly incomplete measurements. Indeed, methods such as Kalman filtering and smoothing are vital tools for scientists in fields ranging from electronics to weather forecasting. In this work we develop a novel framework for smoothing data based on known or partially known nonlinear governing equations. The method yields superior results to current techniques when applied to problems with known deterministic dynamics. By exploiting the numerical time-stepping constraints of the deterministic system, an optimization formulation can readily extract the noise from the nonlinear dynamics in a principled manner. The superior performance is due in part to the fact that it optimizes global state estimates. We demonstrate the efficiency and efficacy of the method on a number of canonical examples, thus demonstrating its viability for the wide range of potential applications stated above.