Multiscale Dynamical Systems Research Articles

Disentangling nonlinear geomagnetic variability during magnetic storms and quiescence by timescale dependent recurrence properties

Understanding the complex behavior of the near-Earth electromagnetic environment is one of the main challenges of Space Weather studies. This includes both the correct characterization of the different physical mechanisms responsible for its configuration and dynamics as well as the efforts which are needed for a correct forecasting of several phenomena. By using a nonlinear multi-scale dynamical systems approach, we provide here new insights into the scale-to-scale dynamical behavior of both quiet and disturbed periods of geomagnetic activity. The results show that a scale-dependent dynamical transition occurs when moving from short to long timescales, i.e., from fast to slow dynamical processes, the latter being characterized by a more regular behavior, while more dynamical anomalies are found in the behavior of the fast component. This suggests that different physical processes are typical for both dynamical regimes: the fast component, being characterized by a more chaotic and less predictable behavior, can be related to the internal dynamical state of the near-Earth electromagnetic environment, while the slow component seems to be less chaotic and associated with the directly driven processes related to the interplanetary medium variability. Moreover, a clear difference has been found between quiet and disturbed periods, the former being more complex than the latter. These findings support the view that, for a correct forecasting in the framework of Space Weather studies, more attention needs to be devoted to the identification of proxies describing the internal dynamical state of the near-Earth electromagnetic environment.

Discovery of Nonlinear Multiscale Systems: Sampling Strategies and Embeddings

A major challenge in the study of dynamical systems is that of model discovery: turning data into models that are not just predictive, but provide insight into the nature of the underlying dynamical system that generated the data. This problem is made more difficult by the fact that many systems of interest exhibit diverse behaviors across multiple time scales. We introduce a number of data-driven strategies for discovering nonlinear multiscale dynamical systems and their embeddings from data. We consider two canonical cases: (i) systems for which we have full measurements of the governing variables, and (ii) systems for which we have incomplete measurements. For systems with full state measurements, we show that the recent sparse identification of nonlinear dynamical systems (SINDy) method can discover governing equations with relatively little data and introduce a sampling method that allows SINDy to scale efficiently to problems with multiple time scales. Specifically, we can discover distinct governing equations at slow and fast scales. For systems with incomplete observations, we show that the Hankel alternative view of Koopman (HAVOK) method, based on time-delay embedding coordinates, can be used to obtain a linear model and Koopman invariant measurement system that nearly perfectly captures the dynamics of nonlinear quasiperiodic systems. We introduce two strategies for using HAVOK on systems with multiple time scales. Together, our approaches provide a suite of mathematical strategies for reducing the data required to discover and model nonlinear multiscale systems.

Averaging principle for multiscale stochastic reaction‐diffusion‐advection equations

Stochastic averaging principle is a powerful tool for studying qualitative analysis of multiscale stochastic dynamical systems. In this paper, we will establish an averaging principle for stochastic reaction‐diffusion‐advection equations with slow and fast time scales. Under suitable conditions, we show that the slow component strongly converges to the solution of the corresponding averaged equation.

A study of the regularized lid-driven cavity’s progression to chaos

Computational simulations of a two-dimensional incompressible regularized lid-driven cavity flow were performed and analyzed to identify the dynamic behavior of the flow through multiple bifurcations which ultimately result in Eulerian chaotic flow. Semi-implicit, pseudo-spectral numerical simulations were performed at Reynolds numbers from 1000 to 25,000. Poincaré maps were used to identify transitions as the Reynolds number increased from stable-laminar flow to periodic flow, periodic flow to quasi-periodic flow, quasi-periodic flow to chaotic flow, and a sudden, brief return from chaotic flow to periodic flow. The first critical Reynolds number, near 10,250, is found in agreement with existing literature. An additional bifurcation is observed near a Reynolds number of 15,500. A power spectrum analysis, in which the novel concepts of frequency shredding and power capacity are introduced, was performed with the conclusion that no further bifurcations occurred at Reynolds numbers above 15,500 even though Eulerian chaos was not formally observed until Reynolds numbers above 18,000. While qualitative changes in the fluid flow’s attractor were apparent from trajectories in the phase space, mechanisms by which such changes can occur were apparent from power spectra of flow time histories. The novel power spectrum analysis may also serve as a new approach for characterizing multi-scale nonlinear dynamical systems.

Efficient data-driven reduced-order models for high-dimensional multiscale dynamical systems

We present two data-driven reduced-order models for high-dimensional multiscale dynamical systems. The methodologies proposed assume that the observed high-dimensional dynamic response is generated from some hidden low-dimensional processes. We argue that a single hidden process may not have the representative capability to accurately approximate the dynamical system exhibiting multiple timescales. Hence, the concept of mixture of experts is used where we represent the high-dimensional response as a function of multiple low-dimensional processes. The weight associated with each low-dimensional process is again governed by a separate dynamical process. The primary difference between the two proposed models resides in the procedure adopted for computing the parameters associated with the hidden processes. The first algorithm proposed combines expectation propagation (EP) with expectation maximization (EM) for computing the unknown parameters associated with the latent processes. While some of the parameters are treated in a deterministic sense (point estimates), the others are treated in a Bayesian manner. On the contrary, the second algorithm proposed uses variational Bayes expectation maximization (VBEM) and treats all the parameters in a Bayesian sense. Within VBEM, EP is used as an inference engine. Both the algorithms presented are data-driven. Several numerical examples are presented to certify the accuracy and efficiency of the proposed algorithms.

On the Selection of Localization Radius in Ensemble Filtering for Multiscale Quasigeostrophic Dynamics

Covariance localization remedies sampling errors due to limited ensemble size in ensemble data assimilation. Previous studies suggest that the optimal localization radius depends on ensemble size, observation density and accuracy, as well as the correlation length scale determined by model dynamics. A comprehensive localization theory for multiscale dynamical systems with varying observation density remains an active area of research. Using a two-layer quasigeostrophic (QG) model, this study systematically evaluates the sensitivity of the best Gaspari–Cohn localization radius to changes in model resolution, ensemble size, and observing networks. Numerical experiment results show that the best localization radius is smaller for smaller-scale components of a QG flow, indicating its scale dependency. The best localization radius is rather insensitive to changes in model resolution, as long as the key dynamical processes are reasonably well represented by the low-resolution model with inflation methods that account for representation errors. As ensemble size decreases, the best localization radius shifts to smaller values. However, for nonlocal correlations between an observation and state variables that peak at a certain distance, decreasing localization radii further within this distance does not reduce analysis errors. Increasing the density of an observing network has two effects that both reduce the best localization radius. First, the reduced observation error spectral variance further constrains prior ensembles at large scales. Less large-scale contribution results in a shorter overall correlation length, which favors a smaller localization radius. Second, a denser network provides more independent pieces of information, thus a smaller localization radius still allows the same number of observations to constrain each state variable.

Discrete-Time Statistical Inference for Multiscale Diffusions

We study statistical inference for small-noise-perturbed multiscale dynamical systems under the assumption that we observe a single time series from the slow process only. We construct estimators for both averaging and homogenization regimes, based on an appropriate misspecified model motivated by a second-order stochastic Taylor expansion of the slow process with respect to a function of the time-scale separation parameter. In the case of a fixed number of observations, we establish consistency, asymptotic normality, and asymptotic statistical efficiency of a minimum contrast estimator (MCE), the limiting variance having been identified explicitly; we furthermore establish consistency and asymptotic normality of a simplified MCE, which is, however, not, in general, efficient. These results are then extended to the case of high-frequency observations under a condition restricting the rate at which the number of observations may grow vis-a-vis the separation of scales. Numerical simulations illustrate the ...

Statistical inference for perturbed multiscale dynamical systems

We study statistical inference for small-noise-perturbed multiscale dynamical systems. We prove consistency, asymptotic normality, and convergence of all scaled moments of an appropriately constructed maximum likelihood estimator (MLE) for a parameter of interest, identifying precisely its limiting variance. We allow full dependence of coefficients on both slow and fast processes, which take values in the full Euclidean space; coefficients in the equation for the slow process need not be bounded and there is no assumption of periodic dependence. The results provide a theoretical basis for calibration of small-noise-perturbed multiscale dynamical systems. Data from numerical simulations are presented to illustrate the theory.

Statistical and dynamical properties of covariant lyapunov vectors in a coupled atmosphere-ocean model—multiscale effects, geometric degeneracy, and error dynamics

We study a simplified coupled atmosphere-ocean model using the formalism of covariant Lyapunov vectors (CLVs), which link physically-based directions of perturbations to growth/decay rates. The model is obtained via a severe truncation of quasi-geostrophic equations for the two fluids, and includes a simple yet physically meaningful representation of their dynamical/thermodynamical coupling. The model has 36 degrees of freedom, and the parameters are chosen so that a chaotic behaviour is observed. There are two positive Lyapunov exponents (LEs), sixteen negative LEs, and eighteen near-zero LEs. The presence of many near-zero LEs results from the vast time-scale separation between the characteristic time scales of the two fluids, and leads to nontrivial error growth properties in the tangent space spanned by the corresponding CLVs, which are geometrically very degenerate. Such CLVs correspond to two different classes of ocean/atmosphere coupled modes. The tangent space spanned by the CLVs corresponding to the positive and negative LEs has, instead, a non-pathological behaviour, and one can construct robust large deviations laws for the finite time LEs, thus providing a universal model for assessing predictability on long to ultra-long scales along such directions. Interestingly, the tangent space of the unstable manifold has substantial projection on both atmospheric and oceanic components. The results show the difficulties in using hyperbolicity as a conceptual framework for multiscale chaotic dynamical systems, whereas the framework of partial hyperbolicity seems better suited, possibly indicating an alternative definition for the chaotic hypothesis. They also suggest the need for an accurate analysis of error dynamics on different time scales and domains and for a careful set-up of assimilation schemes when looking at coupled atmosphere-ocean models.

Distributed Optimal Control of Multiscale Dynamical Systems: A Tutorial

Many complex systems, ranging from renewable resources [1] to very-large-scale robotic systems (VLRS) [2], can be described as multiscale dynamical systems comprising many interactive agents. In recent years, signi cant progress has been made in the formation control and stability analysis of teams of agents, such as robots, or autonomous vehicles. In these systems, the mutual goals of the agents are, for example, to maintain a desired con guration, such as a triangle or a star formation, or to perform a desired behavior, such as translating as a group (schooling) or maintaining the center of mass of the group (flocking) [2]-[7]. While this literature has successfully illustrated that the behavior of large networks of interacting agents can be conveniently described and controlled by density functions, it has yet to provide an approach for optimizing the agent density functions such that their mutual goals are optimized.

A Simple Stochastic Parameterization for Reduced Models of Multiscale Dynamics

Multiscale dynamics are frequently present in real-world processes, such as the atmosphere-ocean and climate science. Because of time scale separation between a small set of slowly evolving variables and much larger set of rapidly changing variables, direct numerical simulations of such systems are difficult to carry out due to many dynamical variables and the need for an extremely small time discretization step to resolve fast dynamics. One of the common remedies for that is to approximate a multiscale dynamical systems by a closed approximate model for slow variables alone, which reduces the total effective dimension of the phase space of dynamics, as well as allows for a longer time discretization step. Recently, we developed a new method for constructing a deterministic reduced model of multiscale dynamics where coupling terms were parameterized via the Fluctuation-Dissipation theorem. In this work we further improve this previously developed method for deterministic reduced models of multiscale dynamics by introducing a new method for parameterizing slow-fast interactions through additive stochastic noise in a systematic fashion. For the two-scale Lorenz 96 system with linear coupling, we demonstrate that the new method is able to recover additional features of multiscale dynamics in a stochastically forced reduced model, which the previously developed deterministic method could not reproduce.

Chaos, patterns, coherent structures, and turbulence: Reflections on nonlinear science.

The paradigms of nonlinear science were succinctly articulated over 25 years ago as deterministic chaos, pattern formation, coherent structures, and adaptation/evolution/learning. For chaos, the main unifying concept was universal routes to chaos in general nonlinear dynamical systems, built upon a framework of bifurcation theory. Pattern formation focused on spatially extended nonlinear systems, taking advantage of symmetry properties to develop highly quantitative amplitude equations of the Ginzburg-Landau type to describe early nonlinear phenomena in the vicinity of critical points. Solitons, mathematically precise localized nonlinear wave states, were generalized to a larger and less precise class of coherent structures such as, for example, concentrated regions of vorticity from laboratory wake flows to the Jovian Great Red Spot. The combination of these three ideas was hoped to provide the tools and concepts for the understanding and characterization of the strongly nonlinear problem of fluid turbulence. Although this early promise has been largely unfulfilled, steady progress has been made using the approaches of nonlinear science. I provide a series of examples of bifurcations and chaos, of one-dimensional and two-dimensional pattern formation, and of turbulence to illustrate both the progress and limitations of the nonlinear science approach. As experimental and computational methods continue to improve, the promise of nonlinear science to elucidate fluid turbulence continues to advance in a steady manner, indicative of the grand challenge nature of strongly nonlinear multi-scale dynamical systems.

Linear theory for filtering nonlinear multiscale systems with model error.

In this paper, we study filtering of multiscale dynamical systems with model error arising from limitations in resolving the smaller scale processes. In particular, the analysis assumes the availability of continuous-time noisy observations of all components of the slow variables. Mathematically, this paper presents new results on higher order asymptotic expansion of the first two moments of a conditional measure. In particular, we are interested in the application of filtering multiscale problems in which the conditional distribution is defined over the slow variables, given noisy observation of the slow variables alone. From the mathematical analysis, we learn that for a continuous time linear model with Gaussian noise, there exists a unique choice of parameters in a linear reduced model for the slow variables which gives the optimal filtering when only the slow variables are observed. Moreover, these parameters simultaneously give the optimal equilibrium statistical estimates of the underlying system, and as a consequence they can be estimated offline from the equilibrium statistics of the true signal. By examining a nonlinear test model, we show that the linear theory extends in this non-Gaussian, nonlinear configuration as long as we know the optimal stochastic parametrization and the correct observation model. However, when the stochastic parametrization model is inappropriate, parameters chosen for good filter performance may give poor equilibrium statistical estimates and vice versa; this finding is based on analytical and numerical results on our nonlinear test model and the two-layer Lorenz-96 model. Finally, even when the correct stochastic ansatz is given, it is imperative to estimate the parameters simultaneously and to account for the nonlinear feedback of the stochastic parameters into the reduced filter estimates. In numerical experiments on the two-layer Lorenz-96 model, we find that the parameters estimated online, as part of a filtering procedure, simultaneously produce accurate filtering and equilibrium statistical prediction. In contrast, an offline estimation technique based on a linear regression, which fits the parameters to a training dataset without using the filter, yields filter estimates which are worse than the observations or even divergent when the slow variables are not fully observed. This finding does not imply that all offline methods are inherently inferior to the online method for nonlinear estimation problems, it only suggests that an ideal estimation technique should estimate all parameters simultaneously whether it is online or offline.

A Computational Method to Extract Macroscopic Variables and Their Dynamics in Multiscale Systems

This paper introduces coordinate-independent methods for analyzing multiscale dynamical systems using numerical techniques based on the transfer operator and its adjoint. In particular, we present a method for testing whether an arbitrary dynamical system exhibits multiscale behavior and for estimating the time-scale separation. For systems with such behavior, we establish techniques for analyzing the fast dynamics in isolation, extracting slow variables for the system, and accurately simulating these slow variables at a large time step. We illustrate our method with numerical examples and show how the reduced slow dynamics faithfully represents statistical features of the full dynamics which are not coordinate dependent.

Distributed optimal control for multi-agent trajectory optimization

This paper presents a novel optimal control problem, referred to as distributed optimal control, that is applicable to multiscale dynamical systems comprised of numerous interacting agents. The system performance is represented by an integral cost function of the macroscopic state that is optimized subject to a hyperbolic partial differential equation known as the advection equation. The microscopic control laws are derived from the optimal macroscopic description using a potential function approach. The optimality conditions of the distributed optimal control problem are first derived analytically and, then, demonstrated numerically through a multi-agent trajectory optimization problem.

The role of additive and multiplicative noise in filtering complex dynamical systems

Covariance inflation is an ad hoc treatment that is widely used in practical real-time data assimilation algorithms to mitigate covariance underestimation owing to model errors, nonlinearity, or/and, in the context of ensemble filters, insufficient ensemble size. In this paper, we systematically derive an effective ‘statistical’ inflation for filtering multi-scale dynamical systems with moderate scale gap, , to the case of no scale gap with , in the presence of model errors through reduced dynamics from rigorous stochastic subgrid-scale parametrizations. We will demonstrate that for linear problems, an effective covariance inflation is achieved by a systematically derived additive noise in the forecast model, producing superior filtering skill. For nonlinear problems, we will study an analytically solvable stochastic test model, mimicking turbulent signals in regimes ranging from a turbulent energy transfer range to a dissipative range to a laminar regime. In this context, we will show that multiplicative noise naturally arises in addition to additive noise in a reduced stochastic forecast model. Subsequently, we will show that a ‘statistical’ inflation factor that involves mean correction in addition to covariance inflation is necessary to achieve accurate filtering in the presence of intermittent instability in both the turbulent energy transfer range and the dissipative range.

Test Models for Filtering with Superparameterization

Superparameterization is a fast numerical algorithm to mitigate implicit scale separation of dynamical systems with large-scale, slowly varying “mean” and smaller-scale, rapidly fluctuating “eddy” term. The main idea of superparameterization is to embed parallel highly resolved simulations of small-scale eddies on each grid cell of coarsely resolved large-scale dynamics. In this paper, we study the effect of model errors in using superparameterization for filtering multiscale turbulent dynamical systems. In particular, we use a simple test model, designed to mimic typical multiscale turbulent dynamics with small-scale intermittencies without local statistical equilibriation conditional to the large-scale mean dynamics, and simultaneously force the large-scale dynamics through eddy flux terms. In this paper, we consider the Fourier domain Kalman filter for filtering regularly spaced sparse observations of the large-scale mean variables. We find high filtering and statistical prediction skill with superparameterization (identical to the skill with perfect model), beyond conventional approaches such as the “bare truncation model” that ignores completely the eddy fluxes and the “equilibrium closure” model that crudely approximates the eddy fluxes with classical averaging theory. We show that this high filtering skill is robust even for very sparse observation networks and turbulent signals with a very steep, $-6$, spectrum. This is a counterexample to naive thinking that the small-scale processes are not so important in multiscale turbulent dynamics with steep energy spectrum. We find that the high filtering skill with superparameterization is robust for small enough scale gap, provided that the filter prior model satisfies the classical linear controllability condition. We will demonstrate a spectacular failure of filtering deterministically forced true signals with the exactly perfect model that does not satisfy controllability condition and a dramatic improvement when the controllability condition is restored with additional stochastic forcings. This result reconfirms and justifies the counterintuitive viewpoint that judicious model errors (or noises) can help filtering turbulent signals.

Suppression of chaos at slow variables by rapidly mixing fast dynamics through linear energy-preserving coupling

Chaotic multiscale dynamical systems are common in many areas of science, one of the examples being the interaction of the low-frequency dynamics in the atmosphere with the fast turbulent weather dynamics. One of the key questions about chaotic multiscale systems is how the fast dynamics affects chaos at the slow variables, and, therefore, impacts uncertainty and predictability of the slow dynamics. Here we demonstrate that the linear slow-fast coupling with the total energy conservation property promotes the suppression of chaos at the slow variables through the rapid mixing at the fast variables, both theoretically and through numerical simulations. A suitable mathematical framework is developed, connecting the slow dynamics on the tangent subspaces to the infinite-time linear response of the mean state to a constant external forcing at the fast variables. Additionally, it is shown that the uncoupled dynamics for the slow variables may remain chaotic while the complete multiscale system loses chaos and becomes completely predictable at the slow variables through increasing chaos and turbulence at the fast variables. This result contradicts the common sense intuition, where, naturally, one would think that coupling a slow weakly chaotic system with another much faster and much stronger mixing system would result in general increase of chaos at the slow variables.

Approximation of slow and fast dynamics in multiscale dynamical systems by the linearized Relaxation Redistribution Method

In this paper, we introduce a fictitious dynamics for describing the only fast relaxation of a stiff ordinary differential equation (ODE) system towards a stable low-dimensional invariant manifold in the phase-space ( slow invariant manifold – SIM). As a result, the demanding problem of constructing SIM of any dimensions is recast into the remarkably simpler task of solving a properly devised ODE system by stiff numerical schemes available in the literature. In the same spirit, a set of equations is elaborated for local construction of the fast subspace, and possible initialization procedures for the above equations are discussed. The implementation to a detailed mechanism for combustion of hydrogen and air has been carried out, while a model with the exact Chapman–Enskog solution of the invariance equation is utilized as a benchmark.

PARTICLE FILTERS IN A MULTISCALE ENVIRONMENT: WITH APPLICATION TO THE LORENZ-96 ATMOSPHERIC MODEL

The study of random dynamical systems involves understanding the evolution of state variables that contain uncertainties and that are usually hidden, or not directly observable. Therefore, state variables have to be estimated and updated based on system models using information from observational data, which themselves are noisy, in the sense that they contain uncertainties and disturbances due to imperfections in observational devices and disturbances in the environment within which data are being collected. The development of efficient data assimilation methods for integrating observational data in predicting the evolution of random state variables is thus an important aspect in the study of random dynamical systems. In this paper, we consider a particle filtering approach to nonlinear filtering in multiscale dynamical systems. Particle filtering methods [1–3] utilizes ensembles of particles to represent the conditional density of state variables using particle positions, distributed over a sample space. The distribution of an ensemble of particles is updated using observational data to obtain the best representation of the conditional density of the state variables of interest. On the other hand, homogenization theory [4, 5], allows us to estimate the coarse-grained (slow) dynamics of a multiscale system on a larger timescale without having to explicitly study the fast variable evolution on a small timescale. The results of filter convergence presented in [6] shows the convergence of the filter of the actual state variable to a homogenized solution to the original multiscale system, and thus we develop a particle filtering scheme for multiscale random dynamical systems that utilizes this convergence result. This particle filtering method is called the Homogenized Hybird Particle Filter, and it incorporates a multiscale computation scheme, the Heterogeneous Multiscale Method developed in [7], with the novel branching particle filter described in [8–10]. By incorporating a multiscale scheme based on homogenization of the original system, estimation of the coarse-grained dynamics using observational data is performed over a larger timescale, thus resulting in computational time and cost reduction in terms of the evolution of the state variables as well as functional evaluations for the filtering aspect. We describe the theory behind this combined scheme and its general algorithm, concluded with an application to the Lorenz-96 [11] atmospheric model that mimics midlatitude geophysical dynamics with microscopic convective processes.