Stochastic State Research Articles

Spectral fatigue analysis of ship structures based on a stochastic crack growth state model

Fatigue life estimation in ship structures is a very complex problem, because of the stochasticity in the wave loading experienced by the hull and the inherent variability in the material and geometric parameters. On an effort to quantify such aleatoric uncertainties, this work integrates in a novel setting the spectral approach to loading description into a probabilistic fatigue crack growth model. The proposed framework effectively models both the randomness in the experienced sea states and the corresponding random sequence of the sea states that are encountered by the ship and contribute to the fatigue accumulation. The spectral approach eventually allows for the construction of potential crack growth path instances from a single realization of an effective random stress range that is applied for a corresponding realization of number of cycles. The obtained bundle of stochastic crack paths is further used in order to perform reliability estimations in the entire life cycle. Initially the proposed method is presented formally and algorithmically and its application is demonstrated through a numerical case study of a cracked plate where we are interested in estimating its remaining useful life along with the time distribution of the probability of failure.

Stabilization in Distribution of Hybrid Systems by Intermittent Noise

For many stochastic hybrid systems in the real world, it is inappropriate to study if their solutions will converge to an equilibrium state (say, 0 by default) but more appropriate to discuss if the probability distributions of the solutions will converge to a stationary distribution. The former is known as the asymptotic stability of the equilibrium state while the latter the stability in distribution. This paper aims to determine whether or not a stochastic state feedback control can make a given nonlinear hybrid differential equation, which is not stable in distribution, to become stable in distribution. We will refer to this problem as stabilisation in distribution by noise or stochastic stabilisation in distribution. Although the stabilisation by noise in the sense of almost surely exponential stability of the equilibrium state has been well studied, there is little known on the stabilisation in distribution by noise. This paper initiates the study in this direction.

State constrained stochastic optimal control for continuous and hybrid dynamical systems using DFBSDE

We develop a computationally efficient learning-based forward–backward stochastic differential equations (FBSDE) controller for both continuous and hybrid dynamical (HD) systems subject to stochastic noise and state constraints. Solutions to stochastic optimal control (SOC) problems satisfy the Hamilton–Jacobi–Bellman (HJB) equation. Using current FBSDE-based solutions, the optimal control can be obtained from the HJB equations using deep neural networks (e.g., long short-term memory (LSTM) networks). To ensure the learned controller respects the constraint boundaries, we enforce the state constraints using a soft penalty function. In addition to previous works, we adapt the deep FBSDE (DFBSDE) control framework to handle HD systems consisting of continuous dynamics and a deterministic discrete state change. We demonstrate our proposed algorithm in simulation on a continuous nonlinear system (cart–pole) and a hybrid nonlinear system (five-link biped).

ODE-RSSM: Learning Stochastic Recurrent State Space Model from Irregularly Sampled Data

ODE-RSSM: Learning Stochastic Recurrent State Space Model from Irregularly Sampled Data

Differential flatness based design of robust controllers using polynomial chaos for linear systems

The focus of this paper is the design of open-loop control profiles for linear differentially flat systems whose model parameters are uncertain and are represented probabilistically. To account for the time-invariant model parameter uncertainties, polynomial chaos is used to derive a surrogate model which permits easy evaluation of the mean and variance of the uncertain states. A chance constrained optimisation problem is posed to minimise the terminal error of the stochastic states for a prescribed risk of not satisfying the terminal state bounds. To permit posing a convex optimisation problem, the cost function which is the residual energy is approximated by a hypercube circumscribing the hypersphere which bounds the terminal residual error. This relaxation permits posing a convex optimisation problem to arrive at the robust control profile. The proposed technique is illustrated on two examples including the benchmark spring-mass-dashpot system undergoing a rest-to-rest maneuver and a UAV undergoing a translation in a prescribed time. Parametric studies are conducted where the impact of varying the order of the polynomial chaos expansion and the degree of the parameterised control profiles are used to conclude that a 2–3 degree polynomial chaos expansion is sufficient to capture the time evolution of the mean and variance of the states which are required for the chance constrained optimisation problems. As expected, the design results in improved performance as the number of free variables used to parameterise the control profiles increases.

Quantum statistical effect induced through conditioned postprocessing procedures with unitary t -designs

We propose a few-body quantum phenomenon, which manifests itself through stochastic state preparations and measurements followed by a conditioned post-processing procedure. We show two experimental protocols to implement these phenomena with existing quantum computers, and examine their feasibility by using simulations. Our simulation results suggest that the experimental demonstration is feasible if we repeat the state preparations and measurements about thirty thousand times to three-qubit systems.

Stabilization of Triangular Nonlinear Systems With Multiplicative Stochastic State Sensing Noise

We present new state-feedback control designs for lower-triangular/upper-triangular nonlinear systems with multiplicative stochastic sensor uncertainty. For lower-triangular nonlinear systems with small sensor noise, we develop a novel control design where the control gains are suitably constructed to simultaneously dominate the nonlinear functions and sensor noise of sufficiently small multiplicative gain. For upper-triangular nonlinear systems, we propose a new low-gain domination design, the advantage of which is that it can effectively deal with the sensor noise with arbitrarily large intensities. These two designs can both ensure that closed-loop system has an almost surely unique global solution, the origin of the closed-loop system is mean-square stable, and the states can be regulated to zero almost surely. Finally, two simulation examples are given to illustrate the designs.

State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions.

Recurrent neural networks are a widely used class of neural architectures. They have, however, two shortcomings. First, they are often treated as black-box models and as such it is difficult to understand what exactly they learn as well as how they arrive at a particular prediction. Second, they tend to work poorly on sequences requiring long-term memorization, despite having this capacity in principle. We aim to address both shortcomings with a class of recurrent networks that use a stochastic state transition mechanism between cell applications. This mechanism, which we term state-regularization, makes RNNs transition between a finite set of learnable states. We evaluate state-regularized RNNs on (1) regular languages for the purpose of automata extraction; (2) non-regular languages such as balanced parentheses and palindromes where external memory is required; and (3) real-word sequence learning tasks for sentiment analysis, visual object recognition and text categorisation. We show that state-regularization (a) simplifies the extraction of finite state automata that display an RNN's state transition dynamic; (b) forces RNNs to operate more like automata with external memory and less like finite state machines, which potentiality leads to a more structural memory; (c) leads to better interpretability and explainability of RNNs by leveraging the probabilistic finite state transition mechanism over time steps.

A statistical moment-based spectral approach to the chance-constrained stochastic optimal control of epidemic models

This paper presents a spectral approach to the uncertainty management in epidemic models through the formulation of chance-constrained stochastic optimal control problems. Specifically, a statistical moment-based polynomial expansion is used to calculate surrogate models of the stochastic state variables of the problem that allow for the efficient computation of their main statistics as well as their marginal and joint probability density functions at each time instant, which enable the uncertainty management in the epidemic model. Moreover, the surrogate models are employed to perform the corresponding sensitivity and risk analyses. The proposed methodology provides the designers of the optimal control policies with the capability to increase the predictability of the outcomes by adding suitable chance constraints to the epidemic model and formulating a proper cost functional. The chance-constrained optimal control of a COVID-19 epidemic model is considered in order to illustrate the practical application of the proposed methodology.

Valuation of Mortgages by Using Lévy Models

In a mortgage valuation model, the early termination (i.e., prepayment and default) hazard rates and the recovery rate can be specified as multivariate affine functions that include the correlated stochastic state variables. For good capturing of the distributions for state variables, we specify that the state variables follow Lévy models. Accordingly, the early termination hazard rates and the recovery rate also follow Lévy models. Three popular Lévy models, the normal, Variance Gamma (VG), and Negative Inverse Normal (NIG), were used to obtain the closed-form pricing formula for a mortgage. We also conduct numerical applications. Our results reveal the following findings: first, VG model is better than the normal and NIG models in fitting the actual distributions of the interest rate and the housing return rate. Thus, mortgage valuation using a VG model should be better than that using the other two models. Second, the mortgage value estimated by the normal model is the lowest among the three Lévy models. Third, a prepayment hazard rate with a deterministic value (e.g., using the PSA prepayment model) could result in an unreasonable mortgage duration. Fourth, the mortgage convexity calculated by the normal model is higher than that in the other two Lévy models. Our general pricing formula for a mortgage as described in this study can help market participants accurately value mortgages and effectively manage their risks.

Intelligent control system for droplet volume in inkjet printing based on stochastic state transition soft actor–critic DRL algorithm

Inkjet printing is a low-cost, high efficiency technology for organic light emitting diode (OLED) manufacturing, and it is essential to accurately control the volume of droplets for high-resolution OLED manufacturing. A major challenge in volume control is to address the problem of droplet volume deviation caused by uncertain factors in the production process. In contrast, existing studies focus on the design of fixed waveforms to eject stable droplets, which cannot be used for volume control when the droplet volume deviates due to perturbations. In this paper, a soft actor–critic deep reinforcement learning (DRL) algorithm based on stochastic state transition is proposed for intelligent control of the droplet volume, which can adjust waveform parameters in time to control the volume when the observed droplet volume changes during the production. Firstly, based on the stochastic state transition method, the offline strategy samples are created from the discrete state dataset collected by the visual observation system, and then the control strategy is learned from the offline strategy samples via the soft actor–critic DRL algorithm. Then, according to the droplet volume deviation calculated by the visual observation system, the waveform parameters are adjusted by the control strategy to ensure the droplet volume accuracy. Finally, droplet volume control experiments were performed on an OLED inkjet printing equipment developed by our group. The experimental results show that the proposed method can control the droplet volume within ±5% deviation of the target volume with disturbances in the industrial manufacturing environment.

Hybrid event-triggered synchronization control of delayed chaotic neural networks against communication delay and random data loss

This paper investigates the network-based synchronization control of delayed chaotic neural networks. A new hybrid event-triggered communication scheme, which consists of fixed and dynamic triggering conditions, is presented to handle the random loss of triggered sampled-data. Under this scheme, a quantitative relation between the allowable data loss probability and the fixed triggering interval can be revealed. The synchronous error system is modeled by a multi-constrained system affected by nonlinear terms, piecewise input delay, stochastic update input and reset states. By using the boundary information and interrelation of both communication delay and input delay, a binary quadratic convex lemma and a tailored discontinuous augmented Lyapunov-Krasovskii functional approach are proposed to exploit delay-dependent synchronization criteria with less conservatism. Simulation results are provided for checking the merits of the obtained criteria.

Event-triggered remote state estimation over a collision channel with incomplete information

This paper investigates the stochastic event-triggered remote state estimation problem over a collision channel. The remote estimator does not have the knowledge of the measurement in the case of collision and the origin of the data packet in the case of successful transmission, resulting in the non-Gaussianity of the estimation process in the two cases. By using a commonly used Gaussian approximation method, the posterior distribution of the system state is proved to be a mixture of two Gaussians and a mixture of three Gaussians in the cases of collision and successful transmission, respectively. Then an approximate minimum mean squared error estimator with adaptive weights is proposed, and the weights convexly combine the estimates for the possible transmission situations. Moreover, the proposed estimator is also shown to be conventional forms under three extreme situations. Finally, numerical results illustrate the effectiveness of the proposed estimator based on the incomplete information.

Stochastic methods for slip prediction in a sheared granular system.

We consider a sheared granular system experiencing intermittent dynamics of stick-slip type via discrete element simulations. The considered setup consists of a two-dimensional system of soft frictional particles sandwiched between solid walls, one of which is exposed to a shearing force. The slip events are detected using stochastic state space models applied to various measures describing the system. The amplitudes of the events spread over more than four decades and present two distinctive peaks, one for the microslips and the other for the slips. We show that the measures describing the forces between the particles provide earlier detection of an upcoming slip event than the measures based solely on the wall movement. By comparing the detection times obtained from the considered measures, we observe that a typical slip event starts with a local change in the force network. However, some local changes do not spread globally over the force network. For the changes that become global, we find that their size strongly influences the further behavior of the system. If the size of a global change is large enough, then it triggers a slip event; if it is not, then a much weaker microslip follows. Quantification of the changes in the force network is made possible by formulating clear and precise measures describing their static and dynamic properties.

A comparison of numerical approaches for statistical inference with stochastic models

Due to our limited knowledge about complex environmental systems, our predictions of their behavior under different scenarios or decision alternatives are subject to considerable uncertainty. As this uncertainty can often be relevant for societal decisions, the consideration, quantification and communication of it is very important. Due to internal stochasticity, often poorly known influence factors, and only partly known mechanisms, in many cases, a stochastic model is needed to get an adequate description of uncertainty. As this implies the need to infer constant parameters, as well as the time-course of stochastic model states, a very high-dimensional inference problem for model calibration has to be solved. This is very challenging from a methodological and a numerical perspective. To illustrate aspects of this problem and show options to successfully tackle it, we compare three numerical approaches: Hamiltonian Monte Carlo, Particle Markov Chain Monte Carlo, and Conditional Ornstein-Uhlenbeck Sampling. As a case study, we select the analysis of hydrological data with a stochastic hydrological model. We conclude that the performance of the investigated techniques is comparable for the analyzed system, and that also generality and practical considerations may be taken into account to guide the choice of which technique is more appropriate for a particular application.

Event-triggered adaptive optimal tracking control for nonlinear stochastic systems with dynamic state constraints

This paper investigates the issue of event-triggered adaptive optimal tracking control for uncertain nonlinear systems with stochastic disturbances and dynamic state constraints. To handle the dynamic state constraints, a novel unified tangent-type nonlinear mapping function is proposed. A neural networks (NNs)-based identifier is designed to cope with the stochastic disturbances. By utilizing adaptive dynamic programming (ADP) of identifier-actor-critic architecture and event triggering mechanism, the adaptive optimized event-triggered control (ETC) approach for the nonlinear stochastic system is first proposed. It is proven that the designed optimized ETC approach guarantees the robustness of the stochastic systems and the semi-globally uniformly ultimately bounded in the mean square of the NNs adaptive estimation error, and the Zeno behavior can be avoided. Simulations are offered to illustrate the effectiveness of the proposed control approach.

Transport in reservoir computing

Reservoir computing systems are essentially dynamical systems influenced by an exogenous input. Such systems are extensively used in biologically inspired information processing, and are the state-of-the art techniques for several machine learning tasks. If the statistics of the response or output of the system depends discontinuously on the distribution of the inputs, a fundamental challenge arises in applications where inherent changes in the input stochastic source or noise are expected. This problem can be experimentally demonstrated by showing that altering input statistics can drastically affect the statistics of the response. We solve this instability problem by providing sufficient conditions under which both the marginals and the joint distributions of the response depend continuously on that of the input. To prove our results, we establish the existence of an invariant measure and show that its dependence on the input process is continuous when the processes are endowed with the Wasserstein distance. The main tool in these developments is the characterization of those invariant measures as fixed points of naturally defined Foias operators that appear in this context and which are examined extensively in the paper. These fixed points are obtained by imposing a newly introduced stochastic state contractivity on the driven system that is readily verifiable in examples. Stochastic state contractivity can be satisfied by systems that are not state-contractive, which is a need typically evoked to guarantee the echo state property in reservoir computing. As a result, it may actually be satisfied even if the echo state property is missing.

Output‐feedback stochastic model predictive control of chance‐constrained nonlinear systems

AbstractThis study covers the output‐feedback model predictive control (MPC) of nonlinear systems subjected to stochastic disturbances and state chance constraints. The stochastic optimal control problem is solved in a stochastic dynamic programming fashion, and the output‐feedback control is performed with the extended Kalman filter. The information state is summarized as a dynamic Gaussian belief model. Thus, the stochastic Bellman equation is transformed into a deterministic equation using this model. The resulting constrained Bellman equation is solved using the proposed constrained, approximate dynamic programming algorithm. The algorithm is proved to have a Q‐superlinear local convergence rate. Numerical experiments show that the proposed algorithm can attain good control performance and reasonable chance‐constraint satisfaction and is computationally efficient owing to its dynamic programming structure.

Stochastic event-triggered remote state estimation over Gaussian channels without knowing triggering decisions: A Bayesian inference approach

This paper investigates the stochastic event-triggered remote state estimation problem over a Gaussian communication channel. The triggering decisions of the sensor determine the transmissions of measurements, and are unknown to the remote estimator due to the interference of channel noises. Based on a commonly-accepted Gaussian assumption, an approximate minimum mean squared error estimator with adaptive weights is derived by a Bayesian inference approach. The approximate estimator convexly combines the estimates for both transmission and no transmission cases, and the weights are adaptively updated according to the received data. Further, the proposed estimator behaves like the Kalman filtering with intermittent observations under two extreme situations. Finally, the a posteriori distribution of the estimation process is analyzed when the remote estimator knows the triggering decisions. Numerical results demonstrate that the performance of our estimator is comparable to those that know the triggering decisions, and also better than the detection-based estimator.

T-distributed stochastic neighbor embedding echo state network with state matrix dimensionality reduction for time series prediction

Echo state network (ESN), a novel type of recurrent neural network, possesses high nonlinear mapping capability, which is particularly appropriate for time series prediction. However, the huge reservoir may lead to ill-conditioned solutions in the output weight matrix, reducing the generalization ability and prediction performance of the network. To address this issue, a t-distributed stochastic neighbor embedding ESN (TESN) is proposed in this paper to replace the initial large-scale reservoir state matrix with a low-dimensional manifold. By maintaining the local neighbor relationship of the data in the original high-dimensional space, the ill-conditioned dilemma of the output weight matrix is successfully solved. Moreover, the proposed TESN has a strong ability to preserve the global features of the data, which effectively improves the prediction performance of the network. The superiority of the TESN model is demonstrated through two benchmark prediction tasks and a practical application.