Linear Stochastic Dynamics Research Articles

The subseasonal predictability of the western North Pacific subtropical high and the 2020 record-breaking event

The western North Pacific subtropical high (WNPSH), a prominent feature in the North Pacific during the boreal summer, exerts significant socioeconomic consequences by influencing hydrological extremes such as tropical cyclones, the Meiyu front, and summer heat waves over East Asia. Accurately forecasting the characteristics of the WNPSH over extended timescales is crucial, but subseasonal prediction in this specific context is still in its early stages due to the complex dynamics involved. In this study, we investigate the optimal predictable pattern of the WNPSH using linear stochastic dynamics. Our findings reveal that convection over the Philippine/South China Sea and Japan serves as key precursors, where a dipole vorticity pattern leads to maximum growth of the WNPSH on subseasonal timescales, providing a potential source of predictability. Additionally, we examine the role of optimal predictable patterns in the record-breaking 2020 WNPSH event, and we find that the cumulative effect of stochastic forcing helps explain the sustained features of this extreme case.

Multiagent Persistent Monitoring of Targets With Uncertain States

We address the problem of persistent monitoring, where a finite set of mobile agents has to persistently visit a finite set of targets. Each of these targets has an internal state that evolves with linear stochastic dynamics. The agents can observe these states, and the observation quality is a function of the distance between the agent and a given target. The goal is then to minimize the mean squared estimation error of these target states. We approach the problem from an infinite horizon perspective, where we prove that, under some natural assumptions, the covariance matrix of each target converges to a limit cycle. The goal, therefore, becomes to minimize the steady-state uncertainty. Assuming that the trajectory is parameterized, we provide tools for computing the steady-state cost gradient. We show that, in 1-D (one dimensional) environments with bounded control and nonoverlapping targets, when an optimal control exists it can be represented using a finite number of parameters. We also propose an efficient parameterization of the agent trajectories for multidimensional settings using Fourier curves. Simulation results show the efficacy of the proposed technique in 1-D, 2-D, and 3-D scenarios.

An improved framework for the dynamic likelihood filtering approach to data assimilation.

We propose improvements to the Dynamic Likelihood Filter (DLF), a Bayesian data assimilation filtering approach, specifically tailored to wave problems. The DLF approach was developed to address the common challenge in the application of data assimilation to hyperbolic problems in the geosciences and in engineering, where observation systems are sparse in space and time. When these observations have low uncertainties, as compared to model uncertainties, the DLF exploits the inherent nature of information and uncertainties to propagate along characteristics to produce estimates that are phase aware as well as amplitude aware, as would be the case in the traditional data assimilation approach. Along characteristics, the stochastic partial differential equations underlying the linear or nonlinear stochastic dynamics are differential equations. This study focuses on developing the explicit challenges of relating dynamics and uncertainties in the Eulerian and Lagrangian frames via dynamic Gaussian processes. It also implements the approach using the ensemble Kalman filter (EnKF) and compares the DLF approach to the conventional one with respect to wave amplitude and phase estimates in linear and nonlinear wave problems. Numerical comparisons show that the DLF/EnKF outperforms the EnKF estimates, when applied to linear and nonlinear wave problems. This advantage is particularly noticeable when sparse, low uncertainty observations are used.

A class of hybrid LQG mean field games with state-invariant switching and stopping strategies

A novel framework is presented that combines Mean Field Game (MFG) theory and Hybrid Optimal Control (HOC) theory to obtain a unique ɛ-Nash equilibrium for a non-cooperative game with switching and stopping times. We consider the case where there exists one major agent with a significant influence on the system together with a large number of minor agents constituting two subpopulations, each agent with individually asymptotically negligible effect on the whole system. Each agent has stochastic linear dynamics with quadratic costs, and the agents are coupled in their dynamics and costs by the average state of minor agents (i.e. the empirical mean field). It is shown that for a class of Hybrid LQG MFGs, the optimal switching and stopping times are state-invariant and only depend on the dynamical parameters of each agent. Accordingly, a hybrid systems formulation of the game is presented via the indexing by discrete events: (i) the switching of the major agent between alternative dynamics or (ii) the termination of the agents’ trajectories in one or both of the subpopulations of minor agents. Optimal switchings and stopping time strategies together with best response control actions for, respectively, the major agent and all minor agents are established with respect to their individual cost criteria by an application of Hybrid LQG MFG theory.

Error bounds for model reduction of feedback-controlled linear stochastic dynamics on Hilbert spaces

We analyze structure-preserving model order reduction methods for Ornstein–Uhlenbeck processes and linear S(P)DEs with multiplicative noise based on balanced truncation. For the first time, we include in this study the analysis of non-zero initial conditions. We moreover allow for feedback-controlled dynamics for solving stochastic optimal control problems with reduced-order models and prove novel error bounds for a class of linear quadratic regulator problems. We provide numerical evidence for the bounds and discuss the application of our approach to enhanced sampling methods from non-equilibrium statistical mechanics.

Augmented reliability analysis for estimating imprecise first excursion probabilities in stochastic linear dynamics

Imprecise probability allows quantifying the level of safety of a system taking into account the effect of both aleatory and epistemic uncertainty. The practical estimation of an imprecise probability is usually quite demanding from a numerical viewpoint, as it is necessary to propagate separately both types of uncertainty, leading in practical cases to a nested implementation in the so-called double loop approach. In view of this issue, this contribution presents an alternative approach that avoids the double loop by replacing the imprecise probability problem by an augmented, purely aleatory reliability analysis. Then, with the help of Bayes’ theorem, it is possible to recover an expression for the failure probability as an explicit function of the imprecise parameters from the augmented reliability problem, which ultimately allows calculating the imprecise probability. The implementation of the proposed framework is investigated within the context of imprecise first excursion probability estimation of uncertain linear structures subject to imprecisely defined stochastic quantities and crisp stochastic loads. The associated augmented reliability problem is solved within the context of Directional Importance Sampling, leading to an improved accuracy at reduced numerical costs. The application of the proposed approach is investigated by means of two examples. The results obtained indicate that the proposed approach can be highly efficient and accurate.

Large-scale dynamic system optimization using dual decomposition method with approximate dynamic programming

In this paper, multi-agent dynamic optimization with a coupling constraint is studied. The aim is to minimize a strongly convex social cost function, by considering a linear stochastic dynamics for each agent and also coupling constraints among the agents. In order to handle the coupling constraint and also, to avoid high computational cost imposed by a centralized method for large scale systems, the dual decomposition method is used to decompose the problem into multiple individual sub-problems, while the dual variable is adjusted by a coordinator. Nevertheless, since each sub-problem is not a linear–quadratic (LQ) optimal control problem, and hence its closed-form solution does not exist, approximate dynamic programming (ADP) is utilized to solve the sub-problems. The main contribution of the paper is to propose an algorithm by considering the interrelated iterations of dual variable adjustment and ADP, and to prove the convergence of the algorithm to the global optimal solution of the social cost function. Additionally, the implementation of the proposed algorithm using a neural network is presented. Also, the computational advantage of the proposed algorithm in comparison with other bench-marking methods is discussed in simulation results.

Characterization of Pathogen Airborne Inoculum Density by Information Theoretic Analysis of Spore Trap Time Series Data.

In a previous study, air sampling using vortex air samplers combined with species-specific amplification of pathogen DNA was carried out over two years in four or five locations in the Salinas Valley of California. The resulting time series data for the abundance of pathogen DNA trapped per day displayed complex dynamics with features of both deterministic (chaotic) and stochastic uncertainty. Methods of nonlinear time series analysis developed for the reconstruction of low dimensional attractors provided new insights into the complexity of pathogen abundance data. In particular, the analyses suggested that the length of time series data that it is practical or cost-effective to collect may limit the ability to definitively classify the uncertainty in the data. Over the two years of the study, five location/year combinations were classified as having stochastic linear dynamics and four were not. Calculation of entropy values for either the number of pathogen DNA copies or for a binary string indicating whether the pathogen abundance data were increasing revealed (1) some robust differences in the dynamics between seasons that were not obvious in the time series data themselves and (2) that the series were almost all at their theoretical maximum entropy value when considered from the simple perspective of whether instantaneous change along the sequence was positive.

Are Strategies Favoring Pattern Matching a Viable Way to Improve Complexity Estimation Based on Sample Entropy?

It has been suggested that a viable strategy to improve complexity estimation based on the assessment of pattern similarity is to increase the pattern matching rate without enlarging the series length. We tested this hypothesis over short simulations of nonlinear deterministic and linear stochastic dynamics affected by various noise amounts. Several transformations featuring a different ability to increase the pattern matching rate were tested and compared to the usual strategy adopted in sample entropy (SampEn) computation. The approaches were applied to evaluate the complexity of short-term cardiac and vascular controls from the beat-to-beat variability of heart period (HP) and systolic arterial pressure (SAP) in 12 Parkinson disease patients and 12 age- and gender-matched healthy subjects at supine resting and during head-up tilt. Over simulations, the strategies estimated a larger complexity over nonlinear deterministic signals and a greater regularity over linear stochastic series or deterministic dynamics importantly contaminated by noise. Over short HP and SAP series the techniques did not produce any practical advantage, with an unvaried ability to discriminate groups and experimental conditions compared to the traditional SampEn. Procedures designed to artificially increase the number of matches are of no methodological and practical value when applied to assess complexity indexes.

Multi-Agent Infinite Horizon Persistent Monitoring of Targets with Uncertain States in Multi-Dimensional Environments

Abstract This paper investigates the problem of persistent monitoring, where a finite set of mobile agents persistently visits a finite set of targets in a multi-dimensional environment. The agents must estimate the targets’ internal states and the goal is to minimize the mean squared estimation error over time. The internal states of the targets evolve with linear stochastic dynamics and thus the optimal estimator is a Kalman-Bucy Filter. We constrain the trajectories of the agents to be periodic and represented by a truncated Fourier series. Taking advantage of the periodic nature of this solution, we define the infinite horizon version of the problem and explore the property that the mean estimation squared error converges to a limit cycle. We present a technique to compute online the gradient of the steady state mean estimation error of the targets’ states with respect to the parameters defining the trajectories and use a gradient descent scheme to obtain locally optimal movement schedules. This scheme allows us to address the infinite horizon problem with only a small number of parameters to be optimized.

On stochastic maximum principle for risk-sensitive of fully coupled forward-backward stochastic control of mean-field type with application

In this paper, we are concerned with an optimal control problem where the system is driven by fully coupled forward-backward stochastic differential equation of mean-field type with risk-sensitive performance functional. We study the risk-neutral model for which an optimal solution exists as a preliminary step. This is an extension of the initial stochastic control problem in this type of risk-sensitive performance problem, where an admissible set of controls are convex. We establish necessary as well as sufficient optimality conditions for the risk-sensitive performance functional control problem. Finally, we illustrate our main result of this paper by giving two examples of risk-sensitive control problem under linear stochastic dynamics with exponential quadratic cost function, the second example will be a mean-variance portfolio with a recursive utility functional optimization problem involving optimal control. The explicit expression of the optimal portfolio selection strategy is obtained in the state feedback.

Inference for dynamics of continuous variables: the extended Plefka expansion with hidden nodes

We consider the problem of a subnetwork of observed nodes embedded into a larger bulk of unknown (i.e. hidden) nodes, where the aim is to infer these hidden states given information about the subnetwork dynamics. The biochemical networks underlying many cellular and metabolic processes are important realizations of such a scenario as typically one is interested in reconstructing the time evolution of unobserved chemical concentrations starting from the experimentally more accessible ones. We present an application to this problem of a novel dynamical mean field approximation, the extended Plefka expansion, which is based on a path integral description of the stochastic dynamics. As a paradigmatic model we study the stochastic linear dynamics of continuous degrees of freedom interacting via random Gaussian couplings. The resulting joint distribution is known to be Gaussian and this allows us to fully characterize the posterior statistics of the hidden nodes. In particular the equal-time hidden-to-hidden variance—conditioned on observations—gives the expected error at each node when the hidden time courses are predicted based on the observations. We assess the accuracy of the extended Plefka expansion in predicting these single node variances as well as error correlations over time, focussing on the role of the system size and the number of observed nodes.

Critical scaling in hidden state inference for linear Langevin dynamics

We consider the problem of inferring the dynamics of unknown (i.e. hidden) nodes from a set of observed trajectories and study analytically the average prediction error and the typical relaxation time of correlations between errors. We focus on a stochastic linear dynamics of continuous degrees of freedom interacting via random Gaussian couplings in the infinite network size limit. The expected error on the hidden time courses can be found as the equal-time hidden-to-hidden covariance of the probability distribution conditioned on observations. In the stationary regime, we analyze the phase diagram in the space of relevant parameters, namely the ratio between the numbers of observed and hidden nodes, the degree of symmetry of the interactions and the amplitudes of the hidden-to-hidden and hidden-to-observed couplings relative to the decay constant of the internal hidden dynamics. In particular, we identify critical regions in parameter space where the relaxation time and the inference error diverge, and determine the corresponding scaling behaviour.

Topology optimization for linear stationary stochastic dynamics: Applications to frame structures

This work develops some foundations of topology optimization for the robust design of structural systems subjected to general stationary stochastic dynamic loads. Three methods are explored to evaluate the dynamic response – the time domain, frequency domain, and state space methods – and the associated design variable sensitivities are derived analytically. The resulting stochastic dynamic topology optimization problem is solved using the gradient-based optimizer Method of Moving Asymptotes (MMA). Sensitivities are computed using the adjoint method and the popular Solid Isotropic Material with Penalization (SIMP) is used to achieve clear existence of structural members. The approach is used to design the lateral load systems of structures that minimize the variance of the system response to stationary stochastic ground motion excitation. Numerical results are presented to illustrate the differences between topologies optimized for stochastic ground motion and topologies optimized for equivalent static loading.

Pontryagin’s Risk-Sensitive Stochastic Maximum Principle for Backward Stochastic Differential Equations with Application

This paper studies the risk-sensitive optimal control problem for a backward stochastic system. More precisely, we set up a necessary stochastic maximum principle for a risk-sensitive optimal control of this kind of equations. The control domain is assumed to be convex and the generator coefficient of such system is allowed to be depend on the control variable. As a preliminary step, we study the risk-neutral problem for which an optimal solution exists. This is an extension of initial control system to this type of problem, where the set of admissible controls is convex. An example to carried out to illustrate our main result of risk-sensitive control problem under linear stochastic dynamics with exponential quadratic cost function.

Inferring hidden states in Langevin dynamics on large networks: Average case performance.

We present average performance results for dynamical inference problems in large networks, where a set of nodes is hidden while the time trajectories of the others are observed. Examples of this scenario can occur in signal transduction and gene regulation networks. We focus on the linear stochastic dynamics of continuous variables interacting via random Gaussian couplings of generic symmetry. We analyze the inference error, given by the variance of the posterior distribution over hidden paths, in the thermodynamic limit and as a function of the system parameters and the ratio α between the number of hidden and observed nodes. By applying Kalman filter recursions we find that the posterior dynamics is governed by an "effective" drift that incorporates the effect of the observations. We present two approaches for characterizing the posterior variance that allow us to tackle, respectively, equilibrium and nonequilibrium dynamics. The first appeals to Random Matrix Theory and reveals average spectral properties of the inference error and typical posterior relaxation times; the second is based on dynamical functionals and yields the inference error as the solution of an algebraic equation.

The global dynamical complexity of the human brain network.

How much information do large brain networks integrate as a whole over the sum of their parts? Can the dynamical complexity of such networks be globally quantified in an information-theoretic way and be meaningfully coupled to brain function? Recently, measures of dynamical complexity such as integrated information have been proposed. However, problems related to the normalization and Bell number of partitions associated to these measures make these approaches computationally infeasible for large-scale brain networks. Our goal in this work is to address this problem. Our formulation of network integrated information is based on the Kullback-Leibler divergence between the multivariate distribution on the set of network states versus the corresponding factorized distribution over its parts. We find that implementing the maximum information partition optimizes computations. These methods are well-suited for large networks with linear stochastic dynamics. We compute the integrated information for both, the system’s attractor states, as well as non-stationary dynamical states of the network. We then apply this formalism to brain networks to compute the integrated information for the human brain’s connectome. Compared to a randomly re-wired network, we find that the specific topology of the brain generates greater information complexity.

Stochastic linearization of turbulent dynamics of dispersive waves in equilibrium and non-equilibrium state

Characterizing dispersive wave turbulence in the long time dynamics is central to understanding of many natural phenomena, e.g., in atmosphere ocean dynamics, nonlinear optics, and plasma physics. Using the β-Fermi–Pasta–Ulam nonlinear system as a prototypical example, we show that in thermal equilibrium and non-equilibrium steady state the turbulent state even in the strongly nonlinear regime possesses an effective linear stochastic structure in renormalized normal variables. In this framework, we can well characterize the spatiotemporal dynamics, which are dominated by long-wavelength renormalized waves. We further demonstrate that the energy flux is nearly saturated by the long-wavelength renormalized waves in non-equilibrium steady state. The scenario of such effective linear stochastic dynamics can be extended to study turbulent states in other nonlinear wave systems.

Temporal logic control for stochastic linear systems using abstraction refinement of probabilistic games

We consider the problem of computing the set of initial states of a dynamical system such that there exists a control strategy to ensure that the trajectories satisfy a temporal logic specification with probability 1 (almost-surely). We focus on discrete-time, stochastic linear dynamics and specifications given as formulas of the Generalized Reactivity(1) fragment of Linear Temporal Logic over linear predicates in the states of the system. We propose a solution based on iterative abstraction-refinement, and turn-based 2-player probabilistic games. While the theoretical guarantee of our algorithm after any finite number of iterations is only a partial solution, we show that if our algorithm terminates, then the result is the set of all satisfying initial states. Moreover, for any (partial) solution our algorithm synthesizes witness control strategies to ensure almost-sure satisfaction of the temporal logic specification. While the proposed algorithm guarantees progress and soundness in every iteration, it is computationally demanding. We offer an alternative, more efficient solution for the reachability properties that decomposes the problem into a series of smaller problems of the same type. All algorithms are demonstrated on an illustrative case study.

A class of interference induced games: Asymptotic Nash equilibria and parameterized cooperative solutions

We consider a multi-agent system with linear stochastic individual dynamics, and individual linear quadratic ergodic cost functions. The agents partially observe their own states. Their cost functions and initial statistics are a priori independent but they are coupled through an interference term (the mean of all agent states), entering each of their individual measurement equations. While in general for a finite number of agents, the resulting optimal control law may be a non linear function of the available observations, we establish that for certain classes of cost and dynamic parameters, optimal separated control laws obtained by ignoring the interference coupling, are asymptotically optimal when the number of agents goes to infinity, thus forming for finite N, an ϵ-Nash equilibrium. More generally though, optimal separated control laws may not be asymptotically optimal, and can in fact result in unstable overall behavior. Thus we consider a class of parameterized decentralized control laws whereby the separated Kalman gain is treated as the arbitrary gain of a Luenberger like observer. System stability regions are characterized and the nature of optimal cooperative control policies within the considered class is explored. Numerical results and an application example for wireless communications are reported.