Continuous-time Stochastic Systems Research Articles

Stabilisation of hybrid stochastic systems with Lévy noise by discrete-time feedback control

This paper is devoted to deal with the stabilisation problems of continuous-time hybrid stochastic systems (often described by stochastic differential equations with Markovian switching) with Lévy noise by feedback controls based on discrete-time state observations. There are so far few results on the stabilisation of continuous-time hybrid stochastic systems with Lévy noise based on discrete-time state observations, despite the stabilisation of stochastic systems by discrete-time feedback controls has been investigated in general. Precisely, it is discussed the stabilisation in the sense of -stability, asymptotic stability, mean-square exponential stability and almost sure exponential stability for such systems. It is also presented an upper bound on the duration τ between two consecutive state observations by means of the Lyapunov function. Two examples are given to illustrate the obtained results.

Compositional abstraction-based synthesis for continuous-time stochastic hybrid systems

In this paper, we propose a compositional framework for the construction of discrete-time finite abstractions, also known as finite Markov decision processes, from continuous-time stochastic hybrid systems by quantifying the distance between their outputs in a probabilistic setting. The proposed scheme is based on the notion of stochastic simulation functions, which is used to relate continuous-time stochastic systems with their discrete-time counterparts. Accordingly, one can employ discrete-time abstract systems as substitutions of the continuous-time ones in the controller design process with guaranteed error bounds on their output trajectories. To this end, we first derive sufficient small-gain type conditions for the compositional quantification of the probabilistic distance between the interconnection of original continuous-time stochastic hybrid systems and their discrete-time (finite or infinite) abstractions. We then construct finite abstractions together with their corresponding stochastic simulation functions for a particular class of nonlinear stochastic hybrid systems having some stability property. We illustrate the effectiveness of the proposed results by applying our approaches to the temperature regulation in a circular building and constructing compositionally a discrete-time abstraction from its original continuous-time dynamics in a network containing 1000 rooms. We employ the constructed discrete-time abstractions as substitutes to compositionally synthesize policies regulating the temperature of each room for a bounded time horizon.

Pareto optimal control of the mean-field stochastic systems by adaptive dynamic programming algorithm

The Pareto game for the model-free continuous-time stochastic system is studied through approximate/adaptive dynamic programming (ADP) in this paper. Firstly, the model-based online iterative algorithm is proposed, and it is proved that the control iterative sequence converges to the Pareto efficient solution, but the algorithm requires complete system parameters. Then, we derive the model-free iterative equation and develop the ADP algorithm to calculate the equation by collecting updated states and input information online. From the derivation of the ADP algorithm, the model-free iterative equation and the model-based iterative equation have the same solution, which means that the ADP algorithm can approximate the Pareto optimal solution. Next, the convergence analysis shows that the Pareto optimal strategy is uniquely determined by the ADP algorithm. Finally, two simulation examples confirm the feasibility of the ADP algorithm.

Compositional Construction of Finite MDPs for Continuous-Time Stochastic Systems: A Dissipativity Approach

Abstract This paper provides a compositional scheme based on dissipativity approaches for constructing finite abstractions of continuous-time continuous-space stochastic control systems. The proposed framework enjoys the structure of the interconnection topology and employs a notion of stochastic storage functions, that describe joint dissipativity-type properties of subsystems and their abstractions. By utilizing those stochastic storage functions, one can establish a relation between continuous-time continuous-space stochastic systems and their finite counterparts while quantifying probabilistic distances between their output trajectories. Consequently, one can employ the finite system as a suitable substitution of the continuous-time one in the controller design process with a guaranteed error bound. In this respect, we first leverage dissipativity-type compositional conditions for the compositional quantification of the distance between the interconnection of continuous-time continuous-space stochastic systems and that of their discrete-time (finite or infinite) abstractions. We then consider a specific class of stochastic affine systems and construct their finite abstractions together with their corresponding stochastic storage functions. We illustrate the effectiveness of the proposed techniques by applying them to a physical case study.

Compositional Construction of Control Barrier Functions for Networks of Continuous-Time Stochastic Systems

Abstract In this paper, we propose a compositional framework for the construction of control barrier functions for networks of continuous-time stochastic control systems. The proposed scheme is based on a notion of so-called pseudo-barrier functions computed for subsystems, using which one can synthesize state-feedback controllers for interconnected systems enforcing safety specifications over a finite-time horizon. Particularly, we first leverage sufficient small-gain type conditions to compositionally construct control barrier functions for interconnected systems based on the corresponding pseudo-barrier functions computed for subsystems. Then, using the constructed control barrier functions, we quantify upper bounds on exit probabilities - the probability that an interconnected system reaches certain unsafe regions - in a finite-time horizon. We employ a systematic technique based on the sum-of-squares optimization program to search for pseudo-barrier functions of subsystems while synthesizing safety controllers. We demonstrate our proposed results by applying them to a temperature regulation in a network of 1000 rooms.

On Positive Realness for Stochastic Hybrid Singular Systems

This paper deals with the problem of passivity analysis and passivity-based synthesis for continuous-time stochastic hybrid singular systems (SHSSs). First, a set of linear matrix inequalities for the stochastic admissibility and passivity, which is also known as the positive real lemma, is obtained for continuous-time SHSSs. The proposed condition successfully holds a necessary and sufficient condition for the positive realness of SHSSs, whereas the existing papers in the literature have been handled only sufficient conditions. Next, the passivity-based control synthesis problem is also considered based on the new positive real lemma. The passivity-based stabilization criterion for the closed-loop system with its mode-dependent state-feedback control is expressed in terms of matrix inequality. Thus, by introducing an additional slack matrix to the non-convex conditions, the feasible conditions for the control gain are obtained in terms of linear matrix inequalities. Finally, a numerical example is provided to show the effectiveness of the result.

Conditions of Almost Sure Boundedness and Practical Asymptotic Stability of Continuous-Time Stochastic Systems

This paper investigates the boundedness conditions of solutions of stochastic differential equations in the almost sure sense. Boundedness is one of the most fundamental properties in a lot of control problems. In general, it is hard to investigate almost sure boundedness of solutions of stochastic differential equations, unlike deterministic systems. However, for a class of systems, the almost sure boundedness can be investigated. This paper deals with conditions for the almost sure boundedness of stochastic systems, which is based on boundary properties of one-dimensional diffusion processes. Moreover, based on the boundedness, we show the characterization of a kind of practical asymptotic stability of one-dimensional stochastic systems in the almost sure sense. The presented results are validated through a numerical example.

Статистические алгоритмы фильтрации для систем со случайно изменяющейся структурой

New algorithms for solving the optimal filtering problem for continuous-timesystems with a random structure are proposed. This problem is to estimate the currentsystem state vector from observations. The mathematical model of the dynamic systemincludes nonlinear stochastic differential equations, the right side of which defines the systemstructure (regime mode). The right side of these stochastic differential equations may bechanged at random time moments. The structure switching process is the Markov or conditionalMarkov random process with a finite set of states (structure numbers). The state vectorof such system consists of two components: the real vector (continuous part) and the integerstructure number (discrete part). The switch condition for the structure number may bedifferent: the achievement of a given surface by the continuous part of the state vector orthe distribution of a random time period between structure switchings. Each ordered pairof structure numbers can correspond to its own switch law. Algorithms for the estimation of the current state vector for systems with a randomstructure are particle filters, they are based on the statistical modeling method (MonteCarlo method). This work continues the authors’ research in the field of statistical methodsand algorithms for the continuous-time stochastic systems analysis and filtering.

Asymptotic stabilization of continuous-time periodic stochastic systems by feedback control based on periodic discrete-time observations

<p style='text-indent:20px;'>In 2013, Mao initiated the study of stabilization of continuous-time hybrid stochastic differential equations (SDEs) by feedback control based on discrete-time state observations. In recent years, this study has been further developed while using a constant observation interval. However, time-varying observation frequencies have not been discussed for this study. Particularly for non-autonomous periodic systems, it's more sensible to consider the time-varying property and observe the system at periodic time-varying frequencies, in terms of control efficiency. This paper introduces a periodic observation interval sequence, and investigates how to stabilize a periodic SDE by feedback control based on periodic observations, in the sense that, the controlled system achieves <inline-formula><tex-math id="M1">\begin{document}$ L^p $\end{document}</tex-math></inline-formula>-stability for <inline-formula><tex-math id="M2">\begin{document}$ p&gt;1 $\end{document}</tex-math></inline-formula>, almost sure asymptotic stability and <inline-formula><tex-math id="M3">\begin{document}$ p $\end{document}</tex-math></inline-formula>th moment asymptotic stability for <inline-formula><tex-math id="M4">\begin{document}$ p \ge 2 $\end{document}</tex-math></inline-formula>. This paper uses the Lyapunov method and inequalities to derive the theory. We also verify the existence of the observation interval sequence and explain how to calculate it. Finally, an illustrative example is given after a useful corollary. By considering the time-varying property of the system, we reduce the observation frequency dramatically and hence reduce the observational cost for control.

Gradient Flow Algorithms for Density Propagation in Stochastic Systems

We develop a new computational framework to solve the partial differential equations (PDEs) governing the flow of the joint probability density functions (PDFs) in continuous-time stochastic nonlinear systems. The need for computing the transient joint PDFs subject to prior dynamics arises in uncertainty propagation, nonlinear filtering, and stochastic control. Our methodology breaks away from the traditional approach of spatial discretization or function approximation—both of which, in general, suffer from the “curse-of-dimensionality.” In the proposed framework, we discretize time but not the state space. We solve infinite dimensional proximal recursions in the manifold of joint PDFs, which in the small time-step limit, is theoretically equivalent to solving the underlying transport PDEs. The resulting computation has the geometric interpretation of gradient flow of certain free energy functional with respect to the Wasserstein metric arising from the theory of optimal mass transport. We show that dualization along with an entropic regularization, leads to a cone-preserving fixed point recursion that is proved to be contractive in Thompson metric. A block co-ordinate iteration scheme is proposed to solve the resulting nonlinear recursions with guaranteed convergence. This approach enables remarkably fast computation for nonparametric transient joint PDF propagation. Numerical examples and various extensions are provided to illustrate the scope and efficacy of the proposed approach.

Optimal Continuous-Discrete Linear Filter and Moment Equations for Nonlinear Diffusions

In this article, we consider the estimation problem of continuous-time stochastic systems with discrete measurements, having linear drift and nonlinear diffusion term. We build the infinite-dimensional linear system equivalent to this class of systems by means of a Carleman linearization approach. Based on this embedding, we investigate the properties of the moment equations of the original system, and we show that it is possible to write the optimal linear filter, for which a finite-dimensional approximation can be implemented. We validate the approach by showing that the resulting algorithm may outperform widely used continuous-discrete filters without increasing the computational burden.

Event-Triggered Stabilization for Continuous-Time Stochastic Systems

This article addresses the event-triggered stabilization for continuous-time stochastic systems. Due to stochastic effects, the system state is hard to predict and dominate, and the system behavior would vary with every trial (i.e., sample path) even for the same initial condition. This gives rise to substantial challenges, especially, in determining the execution/sampling times and assessing the closed-loop performance, which urges us to develop powerful and sophisticated tools/methods for the analysis and design of stochastic event-triggered control. In this article, basic theorems, particularly a stochastic convergence theorem, are first proposed for stochastic event-triggered controlled systems. Then, a framework of event-triggered stabilization is established for the stochastic systems without applying the well-known Lyapunov theorems. Specifically, we present conditions under which event-triggered stabilization is feasible for the systems. Accordingly, static and dynamic event-triggering mechanisms are proposed with enforcing a fixed positive lower bound for the interexecution times. While avoiding infinitely fast execution/sampling, both asymptotic stabilization and exponential stabilization are achieved for the systems by the proposed stochastic convergence theorem. The involved analysis is helpful to form a pattern for stochastic event-triggered controlled systems. As the direct application of the established framework, the constructive design of event-triggered controller is realized, respectively, for two representative classes of stochastic systems.

Probability Analysis of a Perturbed Epidemic System with Relapse and Cure

This paper is devoted to a continuous-time stochastic differential system which is derived by incorporating white noise to a deterministic [Formula: see text] epidemic model with mass action incidence, cure and relapse. We focus on the impact of a relapse on the asymptotic properties of the stochastic system. We show that the relapse encourages the persistence of the disease in the population and we determine the threshold of the relapse rate, above the threshold the disease prevails in the population. Furthermore, we show that there exists a unique density function of solutions which converges in [Formula: see text], under certain conditions of the parameters to an invariant density.

Two-Stage Algorithm for Estimation of Nonlinear Functions of State Vector in Linear Gaussian Continuous Dynamical Systems

This paper focuses on the optimal minimum mean square error estimation of a nonlinear function of state (NFS) in linear Gaussian continuous-time stochastic systems. The NFS represents a multivariate function of state variables which carries useful information of a target system for control. The main idea of the proposed optimal estimation algorithm includes two stages: the optimal Kalman estimate of a state vector computed at the first stage is nonlinearly transformed at the second stage based on the NFS and the minimum mean square error (MMSE) criterion. Some challenging theoretical aspects of analytic calculation of the optimal MMSE estimate are solved by usage of the multivariate Gaussian integrals for the special NFS such as the Euclidean norm, maximum and absolute value. The polynomial functions are studied in detail. In this case the polynomial MMSE estimator has a simple closed form and it is easy to implement in practice. We derive effective matrix formulas for the true mean square error of the optimal and suboptimal quadratic estimators. The obtained results we demonstrate on theoretical and practical examples with different types of NFS. Comparison analysis of the optimal and suboptimal nonlinear estimators is presented. The subsequent application of the proposed estimators demonstrates their effectiveness.

Periodic Event-Triggered Output-Feedback Stabilization for Stochastic Systems.

This article is devoted to explore the periodic event-triggered stabilization for continuous-time stochastic systems, and to develop analysis tools/methods for stochastic periodic event-triggered control. Notably, without real-time monitoring of system behavior as in the scenario with continuous event evaluation, it is critical to delicately estimate and govern the execution/sampling error to achieve the desired system performance. This, under stochastic effects, would be substantially challenging, since the behavior of the stochastic system is hard to predict and is disparate between trials even with the same initial conditions. In this article, a framework of global stabilization via periodic event-triggered output-feedback is established: 1) a criterion condition based on the ISS-Lyapunov function is presented for the feasibility of the desired event-triggered stabilization; 2) both the asymptotic stabilization and exponential stabilization are achieved for the systems, with delicately specifying the periodic event-triggering mechanism; and 3) the involved analysis, without applying the well-known Lyapunov theorems, can serve as a pattern from estimating sampling and execution errors to assess the closed-loop stability for stochastic periodic event-triggered control. Moreover, based on the established framework, we contribute the stabilizing controller design via periodic event-triggered output-feedback for a class of stochastic nonlinear systems.

Preview control for a class of linear stochastic systems with multiplicative noise

ABSTRACT In this paper, the preview control problem for a class of linear continuous time stochastic systems with multiplicative noise is studied based on the augmented error system method. First, a deterministic assistant system is introduced, and the original system is translated to the assistant system. Then, the integrator is employed to ensure the output of the closed-loop system tracking the reference signal accurately. Second, the augmented error system, which includes integrator vector, control vector and reference signal, is constructed based on the system after translation. As a result, the tracking problem is transformed into the optimal control problem of the augmented error system, and the optimal control input is obtained by the dynamic programming method. This control input is regarded as the preview controller of the original system. For a linear stochastic system with multiplicative noise, the difficulty being unable to construct an augmented error system by the derivation method is solved in this paper. And, the existence and uniqueness solution of the Riccati equation corresponding to the stochastic augmented error system is discussed. The numerical simulations show that the preview controller designed in this paper is very effective.

Preview control for continuous-time singular stochastic systems

In this paper, the problem of preview control for continuous-time singular stochastic systems is studied by the augmented error system approach. First, a static output feedback is introduced to make the closed-loop system impulse-free. And through the second restricted equivalent transformation, the continuous-time singular stochastic system is transformed into a normal stochastic system and some algebra equations. Then, an assistant system is designed to overcome the difficulty of trying to make derivation to a stochastic system like in deterministic system condition. And the stochastic system after the transformation is translated to the assistant system. In order to make the output of the singular system track the reference signal as accurately as possible without static error, an integrator is introduced. Based on the system after the translation and the error equation, an augmented error system is constructed with the state vector, the error vector and the reference signal. Finally, the tracking problem for the singular stochastic system is transformed into the optimal regulating problem for the augmented error system. By the dynamic programming method, the optimal preview controller for the original singular stochastic system is obtained. The existence and uniqueness solution of the Riccati equation is discussed and the numerical analysis shows the effectiveness for the preview controller designed in this paper.

Stabilization of Stochastic Nonlinear Delay Systems With Exogenous Disturbances and the Event-Triggered Feedback Control

This note is devoted to study the stabilization problem of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. By introducing the notation of input-to-state practical stability and an event-triggered strategy, we establish the input-to-state practically exponential mean-square stability of the suggested system. Moreover, we investigate the stabilization result by designing the feedback gain matrix and the event-triggered feedback controller, which is expressed in terms of linear matrix inequalities. Also, the lower bounds of interexecution times by the proposed event-triggered control method are obtained. Finally, an example is given to show the effectiveness of the proposed method. Compared with a large number of results for discrete-time stochastic systems, only a few results have appeared on the event-triggered control for continuous-time stochastic systems. In particular, there have been no published papers on the event-triggered control for continuous-time stochastic delay systems. This note is a first try to fill the gap on the topic.

Chaos as a symmetry-breaking phenomenon

Chaos is an ubiquitous and fundamental phenomenon with a wide range of features pointing to a similar phenomenology. Although apparently distinct, it is natural to ask if all these features emerge from a unifying principle. Recently, it was realized that all continuous-time stochastic dynamical systems (DSs) — the most relevant in physics because natural DSs are always subject to noise and time is continuous — possess a topological supersymmetry (TS). It was then suggested that its spontaneous breakdown could be interpreted as the stochastic generalization of deterministic chaos. This conclusion stems from the fact that such phenomenon encompasses features that are traditionally associated with chaotic dynamics such as non-integrability, positive topological entropy, sensitivity to initial conditions and the Poincaré–Bendixson theorem. Here, we strengthen and complete this picture by showing that the remaining hallmarks of deterministic chaos — topological transitivity/mixing and dense periodic orbits — while being consistent with the TS breaking, do not actually admit a stochastic generalization. This is a major limitation, since all physical systems are always noisy to some extent. We, therefore, conclude that spontaneous TS breaking can be considered as the most general definition of continuous-time dynamical chaos. Contrary to the common perception and semantics of the word “chaos,” this phenomenon should then be truly interpreted as the low-symmetry, or ordered phase of the DSs that manifest it. We also argue that the concept of chaos in low-dimensional, discrete-time DSs that do not obey the Poincaré–Bendixson theorem, is related to the explicit TS breaking.

NIRK-based Cholesky-factorized square-root accurate continuous-discrete unscented Kalman filters for state estimation in nonlinear continuous-time stochastic models with discrete measurements

This paper further advances the idea of accurate Gaussian filtering towards efficient unscented-type Kalman methods for estimating continuous-time nonlinear stochastic systems with discrete measurements. It implies that the differential equations evolving sigma points utilized in computations of the predicted mean and covariance in time-propagations of the Gaussian distribution are solved accurately, i.e. with negligible error. The latter allows the total error of the unscented Kalman filtering technique to be reduced significantly and gives rise to the novel accurate continuous-discrete unscented Kalman filtering algorithm. At the same time, this algorithm is rather vulnerable to round-off and numerical integration errors committed in each state estimation run because of the need for the Cholesky decomposition of covariance matrices involved. Such a factorization will always fail when the covariance's positivity is lost. This positivity lost issue is commonly resolved with square-root filtering implementations, which propagate not the full covariance matrix but its square root (Cholesky factor) instead. Unfortunately, negative weights encountered in applications of the accurate continuous-discrete unscented Kalman filter to high-dimensional stochastic systems preclude from designing conventional square-root methods. In this paper, we address this problem with low-rank Cholesky factor update procedures or with hyperbolic QR transforms used for yielding J-orthogonal square roots. Our novel square-root algorithms are justified theoretically and examined numerically in an air traffic control scenario.