High-dimensional Dynamical Systems Research Articles

Representing turbulent statistics with partitions of state space. Part 2. The compressible Euler equations

This is the second part of a two-part paper. We apply the methodology of the first paper (Souza, J. Fluid Mech., vol. 997, 2024, A1) to construct a data-driven finite-volume discretization of the Liouville/Fokker–Planck equation of a high-dimensional dynamical system, i.e. the compressible Euler equations with gravity and rotation evolved on a thin spherical shell. We show that the method recovers a subset of the statistical properties of the underlying system, steady-state distributions of observables and autocorrelations of particular observables, as well as revealing the global Koopman modes of the system. We employ two different strategies for the partitioning of a high-dimensional state space, and explore their consequences.

Humans actively reconfigure neural task states.

The ability to switch between different tasks is a core component of adaptive cognition, but a mechanistic understanding of this capacity has remained elusive. Longstanding questions over whether task switching requires active preparation remain hotly contested, in large part due to the difficulty of inferring preparatory dynamics from behavior or time-locked neuroimaging. We make progress on this debate by quantifying neural task representations using high-dimensional linear dynamical systems fit to human electroencephalographic recordings. We find that these dynamical systems have high predictive accuracy and reveal neural signatures of active preparation that are shared with task-optimized neural networks. These findings inform a classic debate about how we control our cognition, and offer a promising new paradigm for neuroimaging analysis.

Quantum algorithm for dynamic mode decomposition integrated with a quantum differential equation solver

We present a quantum algorithm that analyzes time series data simulated by a quantum differential equation solver. The proposed algorithm is a quantum version of a dynamic mode decomposition algorithm used in diverse fields such as fluid dynamics, molecular dynamics, and epidemiology. Our quantum algorithm can also compute matrix eigenvalues and eigenvectors by analyzing the corresponding linear dynamical system. Our algorithm handles a broad range of matrices, particularly those with complex eigenvalues. The complexity of our quantum algorithm is O(polylogN) for an N-dimensional system. This is an exponential speedup over known classical algorithms with at least O(N) complexity. Thus, our quantum algorithm is expected to enable high-dimensional dynamical systems analysis and large matrix eigenvalue decomposition, intractable for classical computers. Published by the American Physical Society 2024

Reduced Markovian models of dynamical systems

Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract the most salient reduced-order dynamics for a given timescale by using a modified clustering algorithm from network theory. The second problem is to provide an alternative construction for the infinitesimal generator of a Markov process that respects statistical features over a large range of time scales. We demonstrate the methodology on three low-dimensional dynamical systems with stochastic and chaotic dynamics. We then apply the method to two high-dimensional dynamical systems, the Kuramoto-Sivashinky equations and data sampled from fluid-flow experiments via Particle Image Velocimetry. We show that the methodology presented herein provides a robust reduced-order statistical representation of the underlying system.

Achieving liquid processors by colloidal suspensions for reservoir computing

The increasing use of machine learning, with its significant computational and environmental costs, has motivated the exploration of unconventional computing substrates. Liquid substrates, such as colloids, are of particular interest due to their ability to conform to various shapes while exhibiting complex dynamics resulting from the collective behaviour of the constituent colloidal particles. This study explores the potential of using a PEDOT:PSS colloidal suspension as a physical reservoir for reservoir computing in spoken digit recognition. Reservoir computing uses high-dimensional dynamical systems to perform tasks with different substrates, including physical ones. Here, a physical reservoir is implemented that encodes temporal data by exploiting the rich dynamics inherent in colloidal suspensions, thus avoiding reliance on conventional computing hardware. The reservoir processes audio input encoded as spike sequences, which are then classified using a trained readout layer to identify spoken digits. Evaluation across different speaker scenarios shows that the colloidal reservoir achieves high accuracy in classification tasks, demonstrating its viability as a physical reservoir substrate.

Controllability of higher-order networks

Higher-order networks can comprehensively describe interactions among groups, thus emerging as a novel area of exploration in network science. This paper aims to delve into the controllability of higher-order networks, where the network topology is characterized by higher-order interactions and the nodes are higher-dimensional dynamical systems. The collective effects on the network controllability from the dynamics of higher-order interactions, node dynamics, inner interactions, and external control inputs are extensively explored. By applying matrix theory and control theory, some necessary and/or sufficient conditions are developed to determine the controllability of hypergraph networks and simplicial complex networks. Through simulated examples, it becomes evident that the controllability of higher-order networked system is far more complicated than that of traditional networked systems and the higher-order topological structures facilitate the controllability. Remarkably, the integrated network can achieve controllability even when the corresponding traditional network is uncontrollable by external inputs.

Reservoir computing with logistic map.

Recent studies on reservoir computing essentially involve a high-dimensional dynamical system as the reservoir, which transforms and stores the input as a higher-dimensional state for temporal and nontemporal data processing. We demonstrate here a method to predict temporal and nontemporal tasks by constructing virtual nodes as constituting a reservoir in reservoir computing using a nonlinear map, namely, the logistic map, and a simple finite trigonometric series. We predict three nonlinear systems, namely, Lorenz, Rössler, and Hindmarsh-Rose, for temporal tasks and a seventh-order polynomial for nontemporal tasks with great accuracy. Also, the prediction is made in the presence of noise and found to closely agree with the target. Remarkably, the logistic map performs well and predicts close to the actual or target values. The low values of the root mean square error confirm the accuracy of this method in terms of efficiency. Our approach removes the necessity of continuous dynamical systems for constructing the reservoir in reservoir computing. Moreover, the accurate prediction for the three different nonlinear systems suggests that this method can be considered a general one and can be applied to predict many systems. Finally, we show that the method also accurately anticipates the time series of the all the three variable of Rössler system for the future (self-prediction).

Featurizing Koopman mode decomposition for robust forecasting.

This article introduces an advanced Koopman mode decomposition (KMD) technique-coined Featurized Koopman Mode Decomposition (FKMD)-that uses delay embedding and a learned Mahalanobis distance to enhance analysis and prediction of high-dimensional dynamical systems. The delay embedding expands the observation space to better capture underlying manifold structures, while the Mahalanobis distance adjusts observations based on the system's dynamics. This aids in featurizing KMD in cases where good features are not a priori known. We show that FKMD improves predictions for a high-dimensional linear oscillator, a high-dimensional Lorenz attractor that is partially observed, and a cell signaling problem from cancer research.

Waddington landscape for prototype learning in generalized Hopfield networks

Networks in machine learning offer examples of complex high-dimensional dynamical systems inspired by and reminiscent of biological systems. Here, we study the learning dynamics of generalized Hopfield networks, which permit visualization of internal memories. These networks have been shown to proceed through a “feature-to-prototype” transition, as the strength of network nonlinearity is increased, wherein the learned, or terminal, states of internal memories transition from mixed to pure states. Focusing on the prototype learning dynamics of the internal memories, we observe stereotypical dynamics of memories wherein similar subgroups of memories sequentially split at well-defined saddles. The splitting order is interpretable and reproducible from one simulation to the other. The dynamics prior to splits are robust to variations in many features of the system. To develop a more rigorous understanding of these global dynamics, we study smaller subsystems that exhibit similar properties to the full system. Within these smaller systems, we combine analytical calculations with numerical simulations to study the dynamics of the feature-to-prototype transition, and the emergence of saddle points in the learning landscape. We exhibit regimes where saddles appear and disappear through saddle-node bifurcations, qualitatively changing the distribution of learned memories as the strength of the nonlinearity is varied—allowing us to systematically investigate the mechanisms that underlie the emergence of the learning dynamics. Several features of the learning dynamics are reminiscent of the Waddington's caricature of cellular differentiation, and we attempt to make this analogy more precise. Memories can thus differentiate in a predictive and controlled way, revealing bridges between experimental biology, dynamical systems theory, and machine learning. Published by the American Physical Society 2024

Fake turbulence

High-dimensional dynamical systems projected onto a lower-dimensional manifold cease to be deterministic and are best described by probability distributions in the projected state space. Their equations of motion map onto an evolution operator with a deterministic component, describing the projected dynamics, and a stochastic one representing the neglected dimensions. This is illustrated with data-driven models for a moderate-Reynolds-number turbulent channel. It is shown that, for projections in which the deterministic component is dominant, relatively ‘physics-free’ stochastic Markovian models can be constructed that mimic many of the statistics of the real flow, even for fairly crude operator approximations, and this is related to general properties of Markov chains. Deterministic models converge to steady states, but the simplified stochastic models can be used to suggest what is essential to the flow and what is not.

GIẢM BẬC MÔ HÌNH ĐỘNG HỌC SỬ DỤNG CẮT NGẮN PHƯƠNG THỨC TRONG BÀI TOÁN CHUYỂN ĐỘNG TÒA NHÀ

Modal truncation, an advanced algorithm for model reduction in dynamic systems, efficiently simplifies complex models by selectively discarding less influential eigenmodes, maintaining a balance between computational efficiency and model accuracy. This paper explores the algorithm's application to a 48th order building model. Proceed to reduce this model to lower orders, then analyze errors in time and frequency domains. Modal truncation algorithm systematically reduces model dimensions while preserving critical dynamic attributes. Numerical simulations reveal a favorable reduction order range (from order 6th to order 25th) for optimal balance, with sensitivity observed at order 25th. From the results obtained, depending on specific requirements, users can use a lower-order model corresponding to the allowed error to replace the original system. Recommendations include iterative refinement for adaptive reduction orders and in-depth analysis around critical points. This algorithm becomes an effective method for researchers dealing with high-dimensional dynamic systems, offering simpler yet accurate model representations. As technology develops, continued refinements and applications of modal truncation are expected, solidifying its role in the realm of model reduction.

The fluctuational transition mechanism of non-hyperbolic chaotic invariant sets

In order to reveal the general escape mechanism of non-hyperbolic chaotic invariant sets under both weak noise limit and finite noise intensity, the simplest example of Hénon map, which represents the stretching and folding of the phase space, is taken to study the escape mechanism under two kinds of global bifurcation: the fractal boundary crisis and the attractor contact crisis. In this paper, we revealed the general exit mechanism by analyzing the escape paths derived from the shooting method and Monte Carlo simulation. Finally, to further demonstrate universality, an example of high-dimensional differential dynamical system, namely the transient chaos Duffing oscillator, is examined, which underscores the main idea of this paper analyzing specific deterministic structures not only enhances the comprehensibility of the escape process, but also allows for predictions of general escape behavior under weak noise intensity.

Fixed Attracting Closed Surfaces in Three and Higher Dimensional Dynamical Systems

Fixed Attracting Closed Surfaces in Three and Higher Dimensional Dynamical Systems

Probabilistic response determination of high-dimensional nonlinear dynamical systems enforced by parametric multiple Poisson white noises

Probabilistic response determination of high-dimensional nonlinear dynamical systems enforced by parametric multiple Poisson white noises

A novel method for response probability density of nonlinear stochastic dynamic systems

AbstractThis paper presents a novel method for analyzing high-dimensional nonlinear stochastic dynamic systems. In particular, we attempt to obtain the solution of the Fokker–Planck–Kolmogorov (FPK) equation governing the response probability density of the system without using the FPK equation directly. The method consists of several important components including the radial basis function neural networks (RBFNN), Feynman–Kac formula and the short-time Gaussian property of the response process. In the area of solving partial differential equations (PDEs) using neural networks, known as physics-informed neural network (PINN), the proposed method presents an effective alternative for obtaining solutions of PDEs without directly dealing with the equation, thus avoids evaluating the derivatives of the equation. This approach has a potential to make the neural network-based solution more efficient and accurate. Several highly challenging examples of nonlinear stochastic systems are presented in the paper to illustrate the effectiveness of the proposed method in comparison to the equation-based RBFNN approach.

Boosting reservoir computer performance with multiple delays.

Time delays play a significant role in dynamical systems, as they affect their transient behavior and the dimensionality of their attractors. The number, values, and spacing of these time delays influences the eigenvalues of a nonlinear delay-differential system at its fixed point. Here we explore a multidelay system as the core computational element of a reservoir computer making predictions on its input in the usual regime close to fixed point instability. Variations in the number and separation of time delays are first examined to determine the effect of such parameters of the delay distribution on the effectiveness of time-delay reservoirs for nonlinear time series prediction. We demonstrate computationally that an optoelectronic device with multiple different delays can improve the mapping of scalar input into higher-dimensional dynamics, and thus its memory and prediction capabilities for input time series generated by low- and high-dimensional dynamical systems. In particular, this enhances the suitability of such reservoir computers for predicting input data with temporal correlations. Additionally, we highlight the pronounced harmful resonance condition for reservoir computing when using an electro-optic oscillator model with multiple delays. We illustrate that the resonance point may shift depending on the task at hand, such as cross prediction or multistep ahead prediction, in both single delay and multiple delay cases.

Hermite kernel surrogates for the value function of high-dimensional nonlinear optimal control problems

Numerical methods for the optimal feedback control of high-dimensional dynamical systems typically suffer from the curse of dimensionality. In the current presentation, we devise a mesh-free data-based approximation method for the value function of optimal control problems, which partially mitigates the dimensionality problem. The method is based on a greedy Hermite kernel interpolation scheme and incorporates context knowledge by its structure. Especially, the value function surrogate is elegantly enforced to be 0 in the target state, non-negative and constructed as a correction of a linearized model. The algorithm allows formulation in a matrix-free way which ensures efficient offline and online evaluation of the surrogate, circumventing the large-matrix problem for multivariate Hermite interpolation. Additionally, an incremental Cholesky factorization is utilized in the offline generation of the surrogate. For finite time horizons, both convergence of the surrogate to the value function and for the surrogate vs. the optimal controlled dynamical system are proven. Experiments support the effectiveness of the scheme, using among others a new academic model with an explicitly given value function. It may also be useful for the community to validate other optimal control approaches.

A novel approach for dimensionality reduction of high-dimensional stochastic dynamical systems using symbolic regression

This paper introduces an innovative data-driven approach for dimensionality reduction in stochastic dynamical systems. The method combines stochastic averaging theory with symbolic regression. First, this approach generates simulation data from the high-dimensional stochastic dynamical system that needs to be reduced. Then, a loss function is designed based on stochastic averaging theory, and symbolic regression is used to obtain a parameterized one-dimensional equation from the simulation data. Finally, the effectiveness of the approach proposed in this paper is demonstrated through validation on Hamiltonian systems with 5, 10, and 20 degrees of freedom (DOF) subjected to noisy excitation. The primary advantage of this method is that it ultimately provides a closed form rather than a black-box process. Furthermore, it overcomes the numerical integration challenges that stochastic averaging methods often face when dealing with high-dimensional systems, therefore it performs effectively with high-dimensional stochastic dynamical systems.

Learning Reduced Fluid Dynamics

Predicting the state evolution of ultra high-dimensional, time-reversible fluid dynamic systems is a crucial but computationally expensive task. Existing physics-informed neural networks either incur high inference cost or cannot preserve the time-reversible nature of the underlying dynamics system. We propose a model-based approach to identify low-dimensional, time reversible, nonlinear fluid dynamic systems. Our method utilizes the symplectic structure of reduced Eulerian fluid and use stochastic Riemann optimization to obtain a low-dimensional bases that minimize the expected trajectory-wise dimension-reduction error over a given distribution of initial conditions. We show that such minimization is well-defined since the reduced trajectories are differentiable with respect to the subspace bases over the entire Grassmannian manifold, under proper choices of timestep sizes and numerical integrators. Finally, we propose a loss function measuring the trajectory-wise discrepancy between the original and reduced models. By tensor precomputation, we show that gradient information of such loss function can be evaluated efficiently over a long trajectory without time-integrating the high-dimensional dynamic system. Through evaluations on a row of simulation benchmarks, we show that our method reduces the discrepancy by 50-90 percent over conventional reduced models and we outperform PINNs by exactly preserving the time reversibility.

Predicting multiple observations in complex systems through low-dimensional embeddings

Forecasting all components in complex systems is an open and challenging task, possibly due to high dimensionality and undesirable predictors. We bridge this gap by proposing a data-driven and model-free framework, namely, feature-and-reconstructed manifold mapping (FRMM), which is a combination of feature embedding and delay embedding. For a high-dimensional dynamical system, FRMM finds its topologically equivalent manifolds with low dimensions from feature embedding and delay embedding and then sets the low-dimensional feature manifold as a generalized predictor to achieve predictions of all components. The substantial potential of FRMM is shown for both representative models and real-world data involving Indian monsoon, electroencephalogram (EEG) signals, foreign exchange market, and traffic speed in Los Angeles Country. FRMM overcomes the curse of dimensionality and finds a generalized predictor, and thus has potential for applications in many other real-world systems.