Stochastic Dynamical Systems Research Articles

Stochastic Processes with Expected Stopping Time

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical optimization criterion is the maximal expected total reward where the MDP stops after T steps, which can be computed by a simple dynamic programming algorithm. We consider a natural generalization of the problem where the stopping times can be chosen according to a probability distribution, such that the expected stopping time is T, to optimize the expected total reward. Quite surprisingly we establish inter-reducibility of the expected stopping-time problem for Markov chains with the Positivity problem (which is related to the well-known Skolem problem), for which establishing either decidability or undecidability would be a major breakthrough. Given the hardness of the exact problem, we consider the approximate version of the problem: we show that it can be solved in exponential time for Markov chains and in exponential space for MDPs.

Privacy-preserving anomaly detection in stochastic dynamical systems: Synthesis of optimal Gaussian Mechanisms

Privacy-preserving anomaly detection in stochastic dynamical systems: Synthesis of optimal Gaussian Mechanisms

Stochastic Nonlinear Model Updating in Structural Dynamics Using a Novel Likelihood Function within the Bayesian-MCMC Framework

The study presents a novel approach for stochastic nonlinear model updating in structural dynamics, employing a Bayesian framework integrated with Markov Chain Monte Carlo (MCMC) sampling for parameter estimation by using an approximated likelihood function. The proposed methodology is applied to both numerical and experimental cases. The paper commences by introducing Bayesian inference and its constituents: the likelihood function, prior distribution, and posterior distribution. The resonant decay method is employed to extract backbone curves, which capture the non-linear behaviour of the system. A mathematical model based on a single degree of freedom (SDOF) system is formulated, and backbone curves are obtained from time response data. Subsequently, MCMC sampling is employed to estimate the parameters using both numerical and experimental data. The obtained results demonstrate the convergence of the Markov chain, present parameter trace plots, and provide estimates of posterior distributions of updated parameters along with their uncertainties. Experimental validation is performed on a cantilever beam system equipped with permanent magnets and electromagnets. The proposed methodology demonstrates promising results in estimating parameters of stochastic non-linear dynamical systems. Through the use of the proposed likelihood functions using backbone curves, the probability distributions of both linear and non-linear parameters are simultaneously identified. Based on this view, the necessity to segregate stochastic linear and non-linear model updating is eliminated.

Unified framework for stochastic dynamic responses and system reliability analysis of long-span cable-stayed bridges under near-fault ground motions

Long-span cable-stayed bridges are complex systems with numerous components, and prone to failures of multiple components under near-fault ground motions with long-period velocity pulses. The evaluation of dynamic system reliability of cable-stayed bridges is an important challenging issue. In this work, a unified framework for simultaneously determining both the stochastic dynamic responses and system reliabilities of long-span cable-stayed bridges under near-fault stochastic ground motions is proposed based on direct probability integral method (DPIM) with adaptive strategy. Firstly, the 3-dimensional finite element model of typical cable-stayed bridge and the stochastic synthesis model of near-fault ground motions are established. Based on probability density integral equation, new formulas in input probability space of time-variant moments of response are derived. Then, DPIM with adaptive strategy is suggested to calculate the mean value and standard deviation of response via new formulas. Dynamic system reliability of cable-stayed bridge under near-fault stochastic ground motions is evaluated by establishing failure criteria of extreme value mappings of multiple performance functions. Finally, numerical results indicate that the long-period velocity pulse of near-fault motions imposes an important effect, resulting in large mean responses and failure of cables. Failure probability of cables is significantly increased with the occurrence of velocity pulses, which causes the decrease of system reliability of cable-stayed bridge. Thus, the randomness and velocity pulse effect of near-fault ground motions should be seriously considered for reliability-based design and safety assessment of cable-stayed bridges.

Optimized distributed formation control using identifier–critic–actor reinforcement learning for a class of stochastic nonlinear multi-agent systems

This article is to propose an adaptive reinforcement learning (RL)-based optimized distributed formation control for the unknown stochastic nonlinear single-integrator dynamic multi-agent system (MAS). For solving the issue of unknown dynamic, an adaptive identifier neural network (NN) is developed to determine the stochastic MAS under expectation sense. And then, for deriving the optimized formation control, the RL is putted into effect via constructing a pair of critic and actor NNs. With regard of the traditional RL optimal controls, their algorithm exists the inherent complexity, because their adaptive RL algorithm are derived from negative gradient of the square of Hamilton–Jacobi–Bellman (HJB) equation. As a result, these methods are difficultly extended to stochastic dynamical systems. However, since this adaptive RL laws are derived from a simple positive function rather than the square of HJB equation, it can make optimal control with simple algorithm. Therefore, this optimized formation scheme can be smoothly performed to the stochastic MAS. Finally, according to theorem proof and computer simulation, the optimized method can realize the required control objective.

Fokker–Planck equation and Feynman–Kac formula for multidimensional stochastic dynamical systems with Lévy noises and time-dependent coefficients

The aim of this paper is to establish a version of the Feynman–Kac formula for the time-dependent multidimensional nonlocal Fokker–Planck equation corresponding to a class of time-dependent stochastic differential equations driven by multiplicative symmetric (or asymmetric) α-stable Lévy noise. First the forward nonlocal Fokker–Planck equation is derived by the adjoint operator method, overcoming the challenges posed by time-dependent multidimensional nonlinear symmetric α-stable Lévy noise. Subsequently, the Feynman–Kac formula for the forward multidimensional time-dependent nonlocal Fokker–Planck equation is established by applying techniques for the backward nonlocal Fokker–Planck equations, which is associated with the backward stochastic differential equation driven by the multiplicative symmetric α-stable Lévy noise. Notably, in the case of asymmetric α-stable Lévy noise case, the presence of the characteristic function in the nonlocal operator adds complexity to the analysis. Using the Feynman–Kac formula, it is demonstrated that the solution of the forward nonlocal Fokker–Planck equation can be readily simulated through Monte Carlo approximation, especially in scenarios involving long-time simulation settings with large steps. These concepts are illustrated with intriguing examples, and the dynamic evolution of the probability density function corresponding to the stochastic SIS model and the stochastic model for the MeKS network (reflecting the interactions among the MecA complex, ComK and ComS) are investigated over an extended period.

Approximation identification for the stochastic time-delayed dynamical system

This paper addresses the challenges of analyzing stochastic dynamical systems with a single time-delay within a data-driven framework. The presence of time-delay introduces non-Markovian characteristics to the system, complicating the analysis of its dynamic behavior using traditional approaches. Drawing inspiration from the small delay approximation, we apply a sparse identification technique to approximate the non-Markovian process with a Markovian one. This innovative method circumvents limitations associated with the system's dimensionality and the complexity of delayed diffusion terms, offering a versatile tool for investigating the dynamics of stochastic time-delayed systems. Our approach begins by establishing a connection between the system's coefficients and simulated data using the Kramers-Moyal formula, which captures the essential statistical properties of the system. We then leverage sparse identification to extract a concise model of the stochastic dynamical system, effectively eliminating the time-delay from consideration. The practicality and efficacy of our method are substantiated through a series of illustrative examples that showcase its application and validate its performance. By introducing this method, we aim to provide a novel analytical framework for stochastic time-delayed systems, advancing the current capabilities for modeling and understanding such complex dynamics.

Identifying the bottleneck and its shifts for the nexus across water supply, power generation and environment systems: Hehuang, China

Identifying the bottleneck is the key to alerting and preventing unexpected failures of the nexus across water supply, power generation, and environment (WPE) systems. However, the bottleneck identification is challenged by the unclear concept of bottleneck as well as the difficulty of simulating complex emergent behaviors. Here, we develop a generic framework for identifying the bottleneck for the WPE nexus, which is defined as the subsystem that has the highest risk of leading to the failure of WPE nexus. First, the evolution of uncertainties of the state variables of WPE nexus is simulated by a generic stochastic dynamical system. Next, the propagation matrix, accumulation ratio (AR), and risk contribution ratio are defined as effective metrics and tools to identify the system bottleneck. Finally, the effects of human decisions and input stochasticity on the shifts of bottleneck are analyzed using sensitivity and scenario analysis. The WPE system in Hehuang Region, China, is selected as a case study. Results indicate that: (a) The effects of human decisions on ARs can be superimposed, suggesting a “superimposition method” for policy-making of Hehuang Region; (b) The subsystems can be classified as “endogenous” or “exogenous”, depending on the sources of accumulated uncertainties that are from local inputs, or inputs from the other subsystems; (c) There exist multiple bottlenecks at the same time that could shift from one subsystem to another, which complicates the prevention of the failures of WPE nexus. For the Hehuang Region, water and energy subsystems are more likely to be bottlenecks at the beginning stage, while society and environment subsystems are more likely to be bottlenecks at the steady stage. This study provides the first attempt to identify the bottleneck within the WPE nexus, which helps to provide realistic insights for preventing unexpected failures.

Long-time dynamics of a random higher-order Kirchhoff model with variable coefficient rotational inertia

This paper delves into the stochastic asymptotic behavior of a non-autonomous stochastic higher-order Kirchhoff equation with variable coefficient rotational inertia. The equation is solved using the Galerkin method, and a stochastic dynamical system is established on this basis. Uniform estimation demonstrates a family of Dk−absorbing sets in the stochastic dynamical system Φk, and the asymptotic compactness of Φk is proved via decomposition. Finally, the family of Dk−random attractors is acquired for the stochastic dynamical system Φk in Vm+k(Ω)×Vm+kb0(Ω). These results improve and extend those in recent literature (Lv et al., 2021). The findings promote the relevant conclusions of the non-autonomous stochastic higher-order Kirchhoff model and provide a theoretical basis for its subsequent application and research.

Marcus Stochastic Differential Equations: Representation of Probability Density

Marcus stochastic delay differential equations are often used to model stochastic dynamical systems with memory in science and engineering. It is challenging to study the existence, uniqueness, and probability density of Marcus stochastic delay differential equations, due to the fact that the delays cause very complicated correction terms. In this paper, we identify Marcus stochastic delay differential equations with some Marcus stochastic differential equations without delays but subject to extra constraints. This helps us to obtain the following two main results: (i) we establish a sufficient condition for the existence and uniqueness of the solution to the Marcus delay differential equations; and (ii) we establish a representation formula for the probability density of the Marcus stochastic delay differential equations. In the representation formula, the probability density for Marcus stochastic differential equations with memory is analytically expressed in terms of probability density for the corresponding Marcus stochastic differential equations without memory.

Optimization based data enrichment using stochastic dynamical system models

We develop a general framework for state estimation in systems modeled with noise-polluted continuous time dynamics and discrete time noisy measurements. Our approach is based on maximum likelihood estimation and employs the calculus of variations to derive optimality conditions for continuous time functions. We make no prior assumptions on the form of the mapping from measurements to state-estimate or on the distributions of the noise terms, making the framework more general than Kalman filtering/smoothing where this mapping is assumed to be linear and the noises Gaussian. The optimal solution that arises is interpreted as a continuous time spline, the structure and temporal dependency of which is determined by the system dynamics and the distributions of the process and measurement noise. Similar to Kalman smoothing, the optimal spline yields increased data accuracy at instants when measurements are taken, in addition to providing continuous time estimates outside the measurement instances. We demonstrate the utility and generality of our approach via illustrative examples that render both linear and nonlinear data filters depending on the particular system. Application of the proposed approach to a Monte Carlo simulation exhibits significant performance improvement in comparison to a common existing method.

Estimation of Multiple Delays in a Stochastic Dynamical System

Estimation of Multiple Delays in a Stochastic Dynamical System

Sufficient condition for reliable logic operations in an over damped bistable system driven by Gaussian white noise.

In this paper, we investigate an overdamped bistable system subject to Gaussian white noise and logical inputs. By solving and estimating the steady distribution of the corresponding Fokker-Planck equation and considering the two essential features of the reliable logic operations (RLOs)-initial value independence and sign invariance-we establish the sufficient condition for RLO occurrence and identify parametric resonance phenomena in the system. Our numerical simulations confirm the reliability and accuracy of the theoretical results. This work offers insights into enhancing the accuracy of stochastic systems, particularly in the realm of logical stochastic resonance, thereby contributing to advancements in understanding and controlling stochastic dynamical systems.

A Pseudoreversible Normalizing Flow for Stochastic Dynamical Systems with Various Initial Distributions

A Pseudoreversible Normalizing Flow for Stochastic Dynamical Systems with Various Initial Distributions

Decision-making based on Markov decision process in integrated artificial reasoning framework—Part I: Theory

This paper presents a decision-making framework based on an integrated artificial reasoning framework and Markov decision process (MDP). The integrated artificial reasoning framework provides a physics-based approach that converts system information into state transition models, and the analysis result will be represented by the transition probabilities that can be used with an MDP to find a traceable and explainable optimal pathway. A dynamic Bayesian network (DBN) is well suited for representing the structure of an MDP. The causality information among process variables (or among subsystems) is mathematically represented in a DBN by the conditional probabilities of the node’s states provided different probabilities of the parent node’s states. To define node states in a physically understandable manner, we used multilevel flow modeling (MFM). An MFM follows the fundamental energy and mass conservation laws and supports the selection of process variables that represent the system of interest so that causal relations among process variables are properly captured. An MFM-based DBN supports developing state transition models in an MDP to capture the effect of process variables of system having physical relations. The operators of the target system can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from component degradation or random failures. We analyzed a simplified exemplary system to illustrate an optimal operational policy using the suggested approach.

Generalized complex fractional moment for the probabilistic characteristic of random vectors

A definition of multi-variate complex quantities called Generalized Complex Fractional Moment (GCFM) based on the multi-dimensional Mellin transform is proposed in this paper, which is also related to the multi-dimensional Riesz fractional integral evaluated in zero. The equivalence property between GCFM, for both multi-dimensional probability density functions and multi-dimensional characteristic functions is established. Furthermore, a method for obtaining marginal probability distributions from GCFM is presented. The validity of the GCFM method is verified through an example involving α-stable random vectors. Additionally, another example using GCFM to reconstruct the non-stationary PDF of the stochastic dynamic system highlights the prospect of applying the GCFM method in engineering.

PC-Kriging-powered parallelizing Bayesian updating for stochastic vehicle-track dynamical system with contact force measurements and Gaussian process discrepancy model

High-precision vehicle-track coupled dynamical models play a vital role in assessing the running safety of the vehicle, tracking fatigue behavior, and establishing a digital twin for high-speed railways. This study established a high-fidelity vehicle-track coupled dynamical model as a forward solver, which was then updated in a Bayesian inference framework based on the field tests of rail irregularities and wheel/rail normal contact forces. The prediction errors corresponding to different time steps were represented through a Gaussian process (GP) model characterized by a covariance function with the Gaussian kernel to incorporate the correlation between different time steps. To cope with the considerable expense of the likelihood function calculations with large-scale loop operations and the computational burden due to the large batch of repetitive evaluations of the high-fidelity forward model involved in the stochastic sampling from the posterior probability distribution, a metamodel-powered parallelizing stochastic sampling scheme was adopted. The posterior distribution could be formulated by replacing the expensive explicit model evaluation involved in the likelihood function with a Polynomial-Chaos-Kriging (PC-Kriging) predictor providing a surrogate mapping between the wheel/rail normal contact forces and physical parameters. Finally, a parallelizing computing scheme running across multiple CPU cores on high-performance computers was realized to sample from the vectorized formula of the emulator-powered posterior distribution. Field test data from a high-speed railway in China was adopted to demonstrate that the Bayesian inference scheme can quantify the uncertainties of the core physical parameters of the high-fidelity vehicle-track coupled dynamical system by efficiently trading off accuracy and computational efficiency.

Computational Methods for Stationary and Nonstationary Fokker–Planck–Kolmogorov Equations with Random Coefficients

In this paper, methodological approaches for distribution functions of nonlinear stochastic differential equations (SDEs) with uncertain parameters are elaborated. The stationary and nonstationary Fokker–Planck–Kolmogorov (FPK) equations associated with the considered random equations are investigated. A semi-analytical approach based on the extended exponential closure handling the parameters uncertainty effects is presented. A new numerical method coupling the polynomial chaos with the differential quadrature method (DQM) is elaborated for the stationary case. A compact matrix formulation is established allowing considering a large number of generalized polynomial chaos. For nonstationary random FPK equations, the polynomial chaos is coupled with the quadrature discretization. A symmetric discretization of the main Fokker–Planck operator using a tensor product of the Hilbert state space and the Hilbert probabilistic space has resulted. The obtained eigenvalues and random eigenfunctions are used for a spectral decomposition. The time solution is obtained by the spectral expansion as well as by the random Euler stochastic method. The accuracy, effectiveness and advantages of the developed procedure in analyzing the stationary and nonstationary probabilistic solutions of nonlinear stochastic dynamical systems with uncertain parameters excited by a Gaussian white noise are demonstrated.

Using reservoir computing to solve FPK equations for stochastic dynamical systems under Gaussian or Non-Gaussian excitation

This paper presents a new approach that uses the Reservoir Computing Algorithm to solve Fokker-Planck-Kolmogorov (FPK) equation excited by both Gaussian white noise and non-Gaussian noise. Unlike typical numerical methods, this methodology does not necessitate spatial reconstruction or numerical supplementation. The novelty of this paper lies in the modifications made to the conventional Reservoir Computing algorithm. We altered the approach for calculating values of the input weight matrix and incorporated autoregressive techniques in the reservoir layer. In addition, we applied data normalization to the training data before training the algorithm to avoid a zero solution. The efficacy of this approach was verified through multiple arithmetic examples, showcasing its practicality and efficiency in solving FPK equations. Moreover, the Reservoir Computing-FPK algorithm is capable of solving high-dimensional and fractional-order FPK equations with a smaller training set than earlier algorithms. Finally, we analyzed how values of the input weight matrix and regularization parameter affected the performance of the algorithm. The findings suggest that the careful selection of hyperparameters can greatly improve the performance of the Reservoir Computing algorithm.

An Exact Theory of Causal Emergence for Linear Stochastic Iteration Systems.

After coarse-graining a complex system, the dynamics of its macro-state may exhibit more pronounced causal effects than those of its micro-state. This phenomenon, known as causal emergence, is quantified by the indicator of effective information. However, two challenges confront this theory: the absence of well-developed frameworks in continuous stochastic dynamical systems and the reliance on coarse-graining methodologies. In this study, we introduce an exact theoretic framework for causal emergence within linear stochastic iteration systems featuring continuous state spaces and Gaussian noise. Building upon this foundation, we derive an analytical expression for effective information across general dynamics and identify optimal linear coarse-graining strategies that maximize the degree of causal emergence when the dimension averaged uncertainty eliminated by coarse-graining has an upper bound. Our investigation reveals that the maximal causal emergence and the optimal coarse-graining methods are primarily determined by the principal eigenvalues and eigenvectors of the dynamic system's parameter matrix, with the latter not being unique. To validate our propositions, we apply our analytical models to three simplified physical systems, comparing the outcomes with numerical simulations, and consistently achieve congruent results.