Stochastic State Research Articles

Model-free distributed state estimation with local measurements.

In this paper, the state estimation problem of physical plants with unknown system dynamic is revisited from the perspective of limited output information measurement, which corresponds to those with characteristics of high-dimensional, wide-area coverage and scatter. Given this fact, a network of sensors are used to carry out the measurement with each one accessing only partial outputs of the targeted systems and a novel model-free state estimation approach, named distributed stochastic variational inference state estimation, is proposed. The key idea of this method is to compensate for the impacts of local output measurements by adding nearest-neighbor rule-based information interaction among estimators to complete the state estimation. It finds from the numerical experiments that the proposed method has clear advantages in both estimation accuracy and speed, and it also provides guidance on how to improve the efficiency of state estimation under local measurements.

Second Order Extended Ensemble Filter for Non-linear Filtering

Whenever the state of a system must be estimated from noisy information, a state estimator is employed to fuse the data with the model to produce an accurate estimate of the state. When the system dynamics and observation models are linear, the Kalman Filter which is optimal, is used. However, in most applications of interest the system dynamics and observations equations are not- linear and suitable extensions of the Kalman Filter have been developed; for example, the Extended Kalman Filter(EKF). The Extended Kalman Filter is based on linearization by the Taylor series expansion about the mean of the state. This filtering process is however computationally expensive especially in high dimensional data. The cause for this is the high cost of integrating the equation of evolution of covariances. Due to this complexity in integration, new methods were sought known as the particle filters. It replaces linearization of non-linearities with Monte Carlo methods. The particle filter formed a basis for Ensemble Kalman Filter (EnKF) an extension of Kalman filter to non-linear filtering. The EnKF reduced the computational cost but its innovation process does not capture information sufficiently hence there is need to improve its performance. This study has developed a new filter, Second order Extended Ensemble Filter (SoEEF).We derived it from stochastic state models by expansion of expected values to the second order by use of Taylor series together with Monte Carlo method and Matlab. We used Lorenz 63 system of ordinary equations and differential Matlab to test the performance of the new filter. Then we compared its performance with four other filters like Bootstrap Particle Filter (BPF), First order Kalman Bucy Filter (FoEKBF),Second order Kalman Bucy Filter (SoKBF) and First order Extended Ensemble Filter (FoEEF). SoEEKF performs much better than the other four filters.

Chance-constrained stochastic optimal control of epidemic models: A fourth moment method-based reformulation

This work proposes a methodology for the reformulation of chance-constrained stochastic optimal control problems that ensures reliable uncertainty management of epidemic outbreaks. Specifically, the chance constraints are reformulated in terms of the first four moments of the stochastic state variables through the so-called fourth moment method for reliability. Moreover, a spectral technique is employed to obtain surrogate models of the stochastic state variables, which enables the efficient computation of the required statistics. The practical implementation of the proposed approach is demonstrated via the optimal control of two different stochastic mathematical models of the COVID-19 transmission. The numerical experiments confirm that, unlike those reformulations based on the Chebyshev–Cantelli’s inequality, the proposed method does not exhibit the undesired outcomes that are typically observed when a higher precision is required for the risk level associated to the given chance constraints.

Automatic activity-travel sequence generator using language, grammar, and machine theory

ABSTRACT Activity schedule results from a complex decision-making process characterized by several interrelated decisions. Different facets of an activity schedule such as activity type, timing, duration, etc. influence each other and this makes modeling activity schedules a complex task. This complexity has compelled researchers to explore different approaches for modeling activity schedules, among which two predominant approaches can be identified: the utility-maximization theory based econometric approach and the computational process modeling approach. Despite their advantages and a few successful practical applications, challenges still remain leaving avenues for exploration of new approaches. This paper contributes in this direction by reviewing the relationship between language, grammar, and machines in the context of sequence analysis for activity sequence generation. Following that, the paper presents a stochastic Finite State Machine that can generate activity sequences to match the frequency distribution of sequences from a given data set. Our results show that the proposed algorithm can not only generate activity sequences with a distribution similar to that of original data but can also efficiently generate new patterns not in the original data.

Tunable Stochastic State Switching in 2D MoS2 Nanomechanical Resonators with Nonlinear Mode Coupling and Internal Resonance.

Coupled nanomechanical resonators have unveiled fascinating physical phenomena, including phonon-cavity coupling, coupled energy decay pathway, avoided crossing, and internal resonance. Despite these discoveries, the mechanisms and control techniques of nonlinear mode coupling phenomena with internal resonances require further exploration. Here, we report on the observation of stochastic switching between the two resonance states with coupled 1:1 internal resonance, for resonant two-dimensional (2D) molybdenum disulfide (MoS2) nanoelectromechanical systems (NEMS), which is directly driven to the critical coupling regime without parametric pumping. We further demonstrate that the probability of state switching is linearly tunable from ∼0% to ∼100% by varying the driving voltage. Furthermore, we gradually increase the white noise amplitude and show that the probability of obtaining the higher-energy state decreases, and the stochastic switching phenomenon eventually disappears. The results provide insights into the dynamics of coupled NEMS resonators and open up new possibilities for sensing and stochastic computing.

A Spin–Orbit Torque Knob to Tailor Stochasticity for Physically Secured Applications

AbstractThe deterministic switching of spin–orbit torque (SOT) memory has long been the primary challenge for implementation. Conversely, stochastic switching has recently sparked significant interest for physically secured applications. This study investigates the effects of an external field (Hx) applied to break the SOT symmetry for deterministic switching in a Pt/Co bilayer with perpendicular anisotropy. This external field serves as a tuning mechanism to adjust the stochasticity at the intermediate state during SOT switching. It is observed that moderate Hx could lead to a substantial variation in electrical output during the repeating write/read operations. In contrast, a large Hx enables precise control over the intermediate states of the Pt/Co bilayer, effectively functioning as a memristor. The stochasticity derives from small Hx resulting from the dynamics of dendritic domain wall propagation during SOT switching. Herein, the oblique domain spiking from the initially reversed domain is notably influenced by the transverse component of SOT. Increasing Hx suppresses this oblique dendritic domain growth, allowing for controllable domain propagation and reducing stochasticity. By utilizing Hx as an external knob to tailor the stochasticity at the intermediate state, the high stochastic state can be employed as an ideal entropy source for an SOT‐based random number generator.

Stochastic Initial States Randomization Method for Robust Knowledge Transfer in Multi-Agent Reinforcement Learning

Reinforcement learning, which are also studied in the field of defense, face the problem of sample efficiency, which requires a large amount of data to train. Transfer learning has been introduced to address this problem, but its effectiveness is sometimes marginal because the model does not effectively leverage prior knowledge. In this study, we propose a stochastic initial state randomization(SISR) method to enable robust knowledge transfer that promote generalized and sufficient knowledge transfer. We developed a simulation environment involving a cooperative robot transportation task. Experimental results show that successful tasks are achieved when SISR is applied, while tasks fail when SISR is not applied. We also analyzed how the amount of state information collected by the agents changes with the application of SISR.

A stepwise scheme for the optimization implementation of lazy wave dynamic cables

The dynamic cables are usually designed to present a complex layout to improve service life and safety in commercial scale floating applications. It is urgent to establish consistent design philosophy and analysis methodologies to provide cost-efficient design requirement. A stepwise scheme for the optimization implementation of lazy wave dynamic cables is proposed and performed with a mass-damping equivalence model. A complicated multi-objective optimal procedure is implemented progressively under hydrostatic state, regular wave state and stochastic sea state. It is validated through a lazy wave dynamic cable (LWDC) from a cylindrical FPSO system under extreme conditions. Stepwise upgraded calibrations based on the coupling of multi-objective optimization modular NSGA-II and lumped mass FEM are applied to assess every sub-objective in defined steps. It is shown that the result is far beyond expectation than initial design after the application of the progressive solution system.

Hybrid fault diagnostics and prognostics system development for motor bearings health monitoring

The article focuses on developing a specialized prognostics system for motor bearings to ensure the safety and reliability of motors and generators. Timely maintenance improves both equipment health and economic viability. The primary objective is to predict early-stage faults, anticipate degradation, and analyze the Remaining Useful Life (RUL) without interrupting the system. The versatile system is developed to continuously monitors deviation profiles, calculates RUL, and optimizes the industrial performance. To address the persistent challenge of imprecise model, integrating sensor data and understanding the physical context, the proposed framework combines data-driven and model-driven learning approaches. The hybrid diagnostics and prognostics system employ Convolutional-LSTM (CNN-LSTM) for feature extraction and fault diagnostics. Particle Filtering (PF) is integrated for stochastic degradation state prediction and RUL estimation. The framework prioritizes learning degradation trends from sensor data without direct physical integration. To validate the effectiveness of the proposed scheme, experiments were performed using the popular PROGNOSTIA bearings dataset, confirming its superior performance in RUL prediction. The developed prognostics system will find broad applicability in process monitoring across various industries, advancing the field and optimizing maintenance practices.

Fundamental properties of ideal and resistive infernal modes in tokamaks

Infernal modes are unstable in regions of weak magnetic shear and significant pressure gradients. These modes comprise a broad class of instabilities, encompassing interchange modes and kink modes, with both short and long length scales. Toroidal effects and fully electromagnetic fields are of crucial importance for their description. The role of resistive diffusion and compressibility are also critical. In order to investigate this awkward problem while still enabling fundamental physics interpretation, a new resistive MHD eigensolver has been developed. An outcome of this study is the identification of an unstable spectrum of resistive infernal modes in regions of the plasma with weak average curvature, and in regions where the average curvature is destabilising. These fast growing modes may be collectively important for our understanding of global reconnection events, stochastic magnetic fields states, and neighbouring supercritical bifurcations.

Memory‐based event‐triggered control for networked control system under cyber‐attacks

AbstractThis article focuses on the problem of stability for a class of linear networked control systems (NCSs) subjected to network communication delays and random deception attacks. A new memory event‐triggered mechanism (METM) is proposed to reduce the unnecessary transmitted data through the communication channel and then enhance the network resources. In this context, a new memory stochastic state feedback controller is proposed to stabilize the closed‐loop networked control system. A new randomly occurring deception attacks model is employed to deal with the security problem of NCSs. Sufficient stability conditions are derived based on a suitable Lyapunov‐Krasovskii functional (LKF). The designed methodology is proposed in terms of linear matrix inequality to synthesize both event‐triggered parameters and controller gains, and to reduce the conservatism of the system some integral lemma are exploited to bind the time derivative of the LKF. Finally, two numerical examples are presented to illustrate the effectiveness of the proposed method which provides a maximal upper bound value of the network‐induced delay and less transmitted packet regarding the maximal value delay obtained in other works, so less conservatism results are obtained, compared to previous ones in the literature.

Reinforcement learning with predefined and inferred reward machines in stochastic games

This paper focuses on Multi-Agent Reinforcement Learning (MARL) in non-cooperative stochastic games, particularly addressing the challenge of task completion characterized by non-Markovian reward functions. We employ Reward Machines (RMs) to incorporate high-level task knowledge. Firstly, we introduce Q-learning with Reward Machines for Stochastic Games (QRM-SG), where RMs are predefined and available to agents. QRM-SG learns each agent’s best-response policy at Nash equilibrium by defining the Q-function in augmented state space that integrates the stochastic game and RM states. The Lemke-Howson method is utilized to compute the best-response policies for the stage game defined by the current Q-functions at each time step. Subsequently, we explore a more challenging scenario where RMs are unavailable and propose Multi-Agent Reinforcement learning with Concurrent High-level knowledge inference (MARCH). MARCH uses automata learning to learn RMs iteratively and combines this process with QRM-SG for learning the best-response policies. The RL episodes where the obtained rewards are inconsistent with the rewards from the current RMs trigger the inference of new RMs. We prove QRM-SG and MARCH converge to the best-response policies under certain conditions. Two scenarios are conducted to demonstrate the superior performance of QRM-SG and MARCH compared to baseline methods.

Term structure of equity risk premia in rough terrain: 150 years of the French stock market

We implement a state-space modeling to capture jointly the one-year and infinite horizons equity risk premia (ERPs) over a secular period in France. Expected stock returns are represented by mixing traditional expectation processes, expected variances are from GARCHX models and risk prices are stochastic state variables estimated using the Kalman filter method. Represented by the spread between long-and short-term ERPs, the term structure strongly varies over time exhibiting a dominant downward sloping. Both expected variances and risk prices are highly at play in determining the ERPs term structure, the effects of restraints on borrowing and of international stock market contagion completing the explanation. Overall, our modeling provides rather similar results using US data. We finally show that the French ERPs term structure varies with the economic cycle, the cost of capital and the liquidity preference.

Irreversibility of stochastic state transitions in Langevin systems with odd elasticity.

Active microscopic objects, such as an enzyme molecule, are modeled by the Langevin system with the odd elasticity, in which energy injection from the substrate to the enzyme is described by the antisymmetric part of the elastic matrix. By applying the Onsager-Machlup integral and large deviation theory to the Langevin system with odd elasticity, we can calculate the cumulant generating function of the irreversibility of the state transition. For an N-component system, we obtain a formal expression of the cumulant generating function and demonstrate that the oddness λ, which quantifies the antisymmetric part of the elastic matrix, leads to higher-order cumulants that do not appear in a passive elastic system. To demonstrate the effect of the oddness under the concrete parameter, we analyze the simplest two-component system and obtain the optimal transition path and cumulant generating function. The cumulants obtained from expansion of the cumulant generating function increase monotonically with the oddness. This implies that the oddness causes the uncertainty of stochastic state transitions.

Recognition of local fiber orientation state in prepreg platelet molded composites via deep learning

This paper presents a novel deep learning approach for reconstruction of local through-the-thickness fiber orientation distribution (FOD) in discontinuous long-fiber, prepreg-platelet molded composites (PPMC). The thermal-residual strains on composite surfaces are used as inputs to train a fully convolutional neural network, named U-Net, to predict spatially varying, local FOD. High fidelity synthetic data was generated via computational simulation of PPMC with stochastic material orientation state and was used for training the deep learning model. The proposed U-Net model allowed for rapid recognition of PPMC morphology by solving the inverse structural mechanics problem of determining the average fiber orientation through the composite thickness based on the provided surface strain measurements. Upon training and validation, the U-Net deep learning model was deployed to rapidly predict complex distributions of the local through-the-thickness FOD in PPMC.

Long-Term Safe Reinforcement Learning with Binary Feedback

Safety is an indispensable requirement for applying reinforcement learning (RL) to real problems. Although there has been a surge of safe RL algorithms proposed in recent years, most existing work typically 1) relies on receiving numeric safety feedback; 2) does not guarantee safety during the learning process; 3) limits the problem to a priori known, deterministic transition dynamics; and/or 4) assume the existence of a known safe policy for any states. Addressing the issues mentioned above, we thus propose Long-term Binary-feedback Safe RL (LoBiSaRL), a safe RL algorithm for constrained Markov decision processes (CMDPs) with binary safety feedback and an unknown, stochastic state transition function. LoBiSaRL optimizes a policy to maximize rewards while guaranteeing long-term safety that an agent executes only safe state-action pairs throughout each episode with high probability. Specifically, LoBiSaRL models the binary safety function via a generalized linear model (GLM) and conservatively takes only a safe action at every time step while inferring its effect on future safety under proper assumptions. Our theoretical results show that LoBiSaRL guarantees the long-term safety constraint, with high probability. Finally, our empirical results demonstrate that our algorithm is safer than existing methods without significantly compromising performance in terms of reward.

Critic-Guided Decision Transformer for Offline Reinforcement Learning

Recent advancements in offline reinforcement learning (RL) have underscored the capabilities of Return-Conditioned Supervised Learning (RCSL), a paradigm that learns the action distribution based on target returns for each state in a supervised manner. However, prevailing RCSL methods largely focus on deterministic trajectory modeling, disregarding stochastic state transitions and the diversity of future trajectory distributions. A fundamental challenge arises from the inconsistency between the sampled returns within individual trajectories and the expected returns across multiple trajectories. Fortunately, value-based methods offer a solution by leveraging a value function to approximate the expected returns, thereby addressing the inconsistency effectively. Building upon these insights, we propose a novel approach, termed the Critic-Guided Decision Transformer (CGDT), which combines the predictability of long-term returns from value-based methods with the trajectory modeling capability of the Decision Transformer. By incorporating a learned value function, known as the critic, CGDT ensures a direct alignment between the specified target returns and the expected returns of actions. This integration bridges the gap between the deterministic nature of RCSL and the probabilistic characteristics of value-based methods. Empirical evaluations on stochastic environments and D4RL benchmark datasets demonstrate the superiority of CGDT over traditional RCSL methods. These results highlight the potential of CGDT to advance the state of the art in offline RL and extend the applicability of RCSL to a wide range of RL tasks.

Modulation Instability and Convergence of the Random-Phase Approximation for Stochastic Sea States

The nonlinear Schrödinger equation is widely used as an approximate model for the evolution in time of the water wave envelope. In the context of simulating ocean waves, initial conditions are typically generated from a measured power spectrum using the random-phase approximation, and periodized on an interval of length L. It is known that most realistic ocean waves power spectra do not exhibit modulation instability, but the most severe ones do; it is thus a natural question to ask whether the periodized random-phase approximation has the correct stability properties. In this work, we specify a random-phase approximation scaling, so that, in the limit of L→∞,\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L\\rightarrow \\infty ,$$\\end{document} the stability properties of the periodized problem are identical to those of the continuous power spectrum on the infinite line. Moreover, it is seen through concrete examples that using a too short computational domain can completely suppress the modulation instability.

On Doob h-transformations for finite-time quantum state reduction

The paper develops a finite-time quantum state reduction framework via the use of Lévy random bridges (LRBs) that can be understood as Doob h-transformations on Lévy processes. Building upon the non-anticipative semimartingale representation of LRBs, we propose a family of energy-driven stochastic Schrödinger equations that go beyond the purely-continuous Brownian motion setup, and enter the scope of quantum systems involving discontinuities and heavy-tails. In doing so, we allow collapse dynamics to be governed intrinsically by the Markovian statistics of LRBs. The framework can host progressively convoluted stochastic state reduction dynamics in a tractable way and encapsulates jump-diffusion Schrödinger evolutions by use of the Lévy-Itô decomposition. Our construct motivates the usage of more general Markov processes and Doob h-transformations in producing consistent wave function collapse dynamics in finite-time.

Analytical results in calculating the entropy of recurrence microstates

Since the development of recurrence plots (RP) and recurrence quantification analysis (RQA), there has been a growing interest in many areas in studying physical systems using recursion techniques. In particular, as part of the RQAs, we observed the development of the concept of recurrence microstates, defined as small blocks obtained from a recurrence graph. It can be shown that some other RQAs can be calculated as a function of recurrence microstates, and the probabilities of occurrences of these microstates can define an information entropy, the so-called entropy of recurrence microstates. It was also observed that recurrence microstates and recurrence entropy can distinguish between correlated and uncorrelated stochastic and deterministic states, due to their symmetry properties. In this paper we propose analytical expressions for calculating the entropy of recurrence microstates, avoiding the need to sample a large set of recurrence microstates. The results can be particularly important in quantifying small amounts of data, where significant sampling of microstates may not be possible. In this paper we propose analytical expressions to compute the entropy of recurrence microstates avoiding the need to sample a large set of recurrence microstates. We show that our results are accurate for cases where the probability distribution function is known. For other situations, the results can be calculated approximately. Another important fact is that the approximate results can be generalized to any size of microstate, making them a powerful tool for calculating the entropy of recurrence. Our approximate methods allow us to know what these properties are and how to exploit this quantifier in the best possible way, with minimal memory usage. Finally, we show that our analytical results are in remarkable agreement with numerical simulations.