Continuous-time Stochastic Systems Research Articles

Robust Recursive Least-Squares Fixed-Point Smoother and Filter using Covariance Information in Linear Continuous-Time Stochastic Systems with Uncertainties

This study develops robust recursive least-squares (RLS) fixed-point smoothing and filtering algorithms for signals in linear continuous-time stochastic systems with uncertainties. The algorithms use covariance information, such as the cross-covariance function of the signal with the observed value and the autocovariance function of the degraded signal. A finite Fourier cosine series expansion approximates these functions. Additive white Gaussian noise is present in the observation of the degraded signal. A numerical simulation compares the estimation accuracy of the proposed robust RLS filter with the robust RLS Wiener filter, showing similar mean square values (MSVs) of the filtering errors. The MSVs of the proposed robust RLS fixed-point smoother are also compared to those of the proposed robust RLS filter.

Active fault diagnosis for stochastic systems subject to non-convex input constraints

In this paper, the problem of active fault diagnosis (AFD) is investigated for a class of continuous-time stochastic systems subject to non-convex input constraints. The input is designed to minimize the probability of misdiagnosis. Since the misdiagnosis probability cannot be expressed in the closed form, its Chernoff upper bound likely tighter than its widely used Bhattacharyya upper bound is exploited as an alternative. The closed-form expression of the Chernoff distance under Gaussian distributions is correctly deduced. For calculation simplification and diagnosis performance enhancement purposes, a novel input design criterion is constructed for AFD based on the Chernoff distance. The optimization problem for AFD is formulated as a bilevel optimization, where the outer and inner layers optimize the Chernoff distance parameter and the input respectively. A convex relaxation is proposed to relax non-convex input constraints into convex ones, and the optimal solution to the relaxed inner input design problem is proven to be also optimal for the original inner problem whose objective function and input constraints are both non-convex under certain conditions, which is referred to as lossless relaxation. The relaxed inner input design problem under the constructed design criterion can be solved to global optimums, and the relaxed outer optimization is solved by a grid search method. Given the optimal input, faults are diagnosed based on the minimum misdiagnosis probability decision rule. The effectiveness of the proposed AFD approach is demonstrated by utilizing a planetary landing example.

Data-Driven Stochastic Optimal Control With Safety Constraints Using Linear Transfer Operators

We provide a data-driven framework for optimal control of a continuous-time control-affine stochastic dynamical system. The proposed framework relies on the linear operator theory involving Perron-Frobenius (P-F) and Koopman operators. Our first result involving the P-F operator provides a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the linear operators. We provide rigorous error bounds and convergence rate for the data-driven approximation of the linear operators and comment on the convergence rate of the optimal control w.r.t. data size. Our second results demonstrate using the Koopman operator, dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving the P-F operator and the Koopman-based formulation using the HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of the P-F and Koopman operators. Finally, we present examples to demonstrate the efficacy of the developed framework and verify the convergence rates for the operator and optimal control numerically.

Design of unbiased reduced order filter for stochastic systems with sampled measurements

The problem of synthesizing reduced order filter, that forms an unbiased estimate of linear functional from an unknown phase vector of a continuous time stochastic system with sampled measurements and additive noises, is considered in this work. The H 2 norm of the weighted transfer function from input noises to estimation error is used as a performance criterion. The method for reduced order filter design is based on the expression of the explicit dependence of performance criterion on unknown parameters in the canonical basis and the application of the non-convex optimization technique. Numerical simulations are performed to illustrate the proposed approach.

Distributed order estimation for continuous-time stochastic systems

In this paper, we investigate the distributed estimation problem of continuous-time stochastic dynamic systems over sensor networks when both the system order and parameters are unknown. We propose a local information criterion (LIC) based on the L0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L_0$$\\end{document} penalty term. By minimizing LIC at the diffusion time instant and utilizing the continuous-time diffusion least squares algorithm, we obtain a distributed estimation algorithm to simultaneously estimate the unknown order and the parameters of the system. By dealing with the effect of the system noises and the coupling relationship between estimation of system orders and parameters, we establish the almost sure convergence results of the proposed distributed estimation algorithm. Furthermore, we give a simulation example to verify the effectiveness of the distributed algorithm in estimating the system order and parameters.

Generalized Invariance Principles for Stochastic Dynamical Systems and Their Applications

Investigating long-term behaviors of stochastic dynamical systems often requires to establish criteria that are able to describe delicate dynamics of the considered systems. In this article, we develop generalized invariance principles for continuous-time stochastic dynamical systems. Particularly, in a sense of probability one and by the developed semimartingale convergence theorem, we not only establish a local invariance principle, but also provide a generalized global invariance principle that allows the sign of the diffusion operator to be positive in some bounded region. We further provide an estimation for the time when a trajectory, initiating outside a particular bounded set, eventually enters it. Finally, we use several representative examples, including stochastic oscillating dynamics, to illustrate the practical usefulness of our analytical criteria in deciphering the stabilization or/and the synchronization dynamics of stochastic systems.

Closed-Loop Continuous-Time Subspace Identification with Prior Information

This paper presents a closed-loop continuous-time subspace identification method using prior information. Based on a rational inner function, a generalized orthonormal basis can be constructed, and the transformed noises have ergodicity features. The continuous-time stochastic system is converted into a discrete-time stochastic system by using generalized orthogonal basis functions. As is known to all, incorporating prior information into identification strategies can increase the precision of the identified model. To enhance the precision of the identification method, prior information is integrated through the use of constrained least squares, and principal component analysis is adopted to achieve the reliable estimate of the system. Moreover, the identification of open-loop models is the primary intent of the continuous-time system identification approaches. For closed-loop systems, the open-loop subspace identification methods may produce biased results. Principal component analysis, which reliably estimates closed-loop systems, provides a solution to this problem. The restricted least-squares method with an equality constraint is used to incorporate prior information into the impulse response following the principal component analysis. The input–output algebraic equation yielded an optimal multi-step-ahead predictor, and the equality constraints describe the prior information. The effectiveness of the proposed method is provided by numerical simulations.

Multiple adaptive fuzzy Nussbaum-type functions design for stochastic nonlinear systems with fixed-time performance

Nussbaum design has been successfully exploited to deal with uncertain systems with a single unknown control coefficient. However, it is challenging for the traditional Nussbaum method to efficiently handle the feedback control design with multiple unknown time-varying control coefficients because the multiple Nussbaum-type gains might mutually cancel out, which makes the Lyapunov-based stability analysis infeasible. To tackle this issue, a novel type of Nussbaum function is designed for the tracking problem of a class of stochastic strict-feedback nonlinear systems. In the presented Nussbaum design, the multiple unknown control coefficients are separately compensated, where the effect of mutual cancellation no longer exists and the Lyapunov stability analysis can be directly applied. Moreover, the proposed Nussbaum design is combined with the fixed-time stability theory, which ensures the settling time is bounded and independent of the initial condition. Within the adaptive fuzzy backstepping framework, it is shown that all signals in the closed-loop system remain bounded through the theoretical discussions. The simulation experiment is conducted to verify the effectiveness of the presented design for the continuous-time stochastic nonlinear dynamical system.

Observer-based SMC design for stochastic systems with Levy noise

Abstract This work addresses the problem of sliding mode control (SMC) design for a continuous-time non-linear stochastic system with Levy-type noise. A state observer model is constructed to estimate the unavailable state information. Furthermore, Levy-type noise is considered to analyse small perturbations and to characterize the appearance of large samples that will occur in the system. Lyapunov stability and SMC theory are used to provide some sufficient conditions that ensure the stochastic stability of the error system and reachability of the predefined sliding surface. Finally, an example is given to demonstrate the feasibility of the proposed approach.

An improved iterative algorithm for solving optimal tracking control problems of stochastic systems

This paper proposes an improved iterative algorithm based on a Newton iterative algorithm, which is used to research the stochastic linear quadratic optimal tracking (SLQT) control for stochastic continuous-time systems. Firstly, a Newton iterative algorithm for solving stochastic algebraic Riccati equations (SAREs) is derived based on the Frechet derivative. Secondly, an improved iterative algorithm in an incremental form for indirectly solving SAREs is presented by combining the numerical iterative method with the incremental Newton iterative (INI) algorithm and introducing a tuning parameter. Under a suitable initial condition, the convergence of the improved iterative algorithm is talked over. Finally, numerical simulation experiments are carried out to compare our proposed improved iterative algorithm with some known works. The results show that the improved iterative algorithm has a faster convergence rate.

Distributed RHC Stabilization for Continuous-time Stochastic Large-scale Systems With Imposed Constraints

Distributed RHC Stabilization for Continuous-time Stochastic Large-scale Systems With Imposed Constraints

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

This article studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed, which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness.

Direct Adaptive Control for Stochastic Systems with Risk-Sensitive Indices

We propose a direct adaptive control law based on the adaptive dynamic programming (ADP) algorithm for continuous-time stochastic linear systems with partially unknown system dynamics and infinite horizon quadratic risk-sensitive indices. A control design methodology is employed to iteratively solve the generalized algebraic Riccati equation by using the online information of the state and input, and to directly learn the optimal control law. We prove the convergence of the online ADP algorithm and show that the direct adaptive control law approximates the optimal control law as time goes on. Finally, a numerical simulation example is presented to demonstrate the effectiveness of our algorithm.

Estimation of Infinitesimal Generators for Unknown Stochastic Hybrid Systems via Sampling: A Formal Approach

In this letter, we develop a data-driven framework with formal confidence bounds for the estimation of infinitesimal generators for continuous-time stochastic hybrid systems with unknown dynamics. The proposed approximation scheme employs both time discretization and sampling from the solution process, and estimates the infinitesimal generator of the solution process via a set of data collected from trajectories of systems. We assume some mild continuity assumptions on the dynamics of the system and quantify the closeness between the infinitesimal generator and its approximation while ensuring an a-priori guaranteed confidence bound. To provide a reasonable closeness precision, we discuss significant roles of both time discretization and number of data in our approximation scheme. In particular, for a fixed number of data, variance of the estimation converges to infinity when the time discretization goes to zero. The proposed approximation framework guides us how to jointly select a suitable data size and a time discretization parameter to cope with this counter-intuitive behavior. We demonstrate the effectiveness of our proposed results by applying them to a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nonlinear</i> jet engine compressor with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unknown</i> dynamics.

An Identification Method for Stochastic Continuous-time Disturbances in Adaptive Optics Systems*

This paper presents a novel identification method for stochastic continuous-time systems applied to Adaptive Optics. We consider a discrete-time sampled-data model of a linear combination of continuous-time second-order systems for modelling disturbances. The Maximum Likelihood framework is used in time and frequency domain to develop an estimation algorithm with sampled-data. We propose an estimation algorithm where we write the likelihood function in the frequency domain in terms of the discrete-time output spectrum (Whittle's log-likelihood function). An approximation for the discrete-time spectrum is used in order to reduce the computational load. A comparative analysis of the proposed method and some available methods is illustrated via numerical simulations.

Practical stability of continuous-time stochastic nonlinear system via event-triggered feedback control

This paper develops new practical stability criteria for continuous-time stochastic nonlinear system with uncertainties and external disturbances. Two cases of the system are considered: the system with state-dependent disturbance and the system with state-independent disturbance. Based on the event-triggered mechanism and Lyapunov function, we establish the input-to-state practical exponential stability in mean square for each case of the system. The obtained results improve some previous works in the literature. Finally, several examples are given to show the effectiveness and practicability of the main results.

Incentive feedback Stackelberg strategy for the discrete-time stochastic systems

The paper is devoted to designing an incentive Stackelberg strategy for the discrete-time stochastic systems. Most of the existing works are about the deterministic systems and the continuous-time stochastic systems. Although many methods have been used to help the leader to attain his team-optimal value, due to the involvement of the conditional expectation, the design of the incentive Stackelberg strategy for the discrete-time stochastic systems is more difficult and complicated. In this paper, this problem is studied for both the finite horizon case and the infinite horizon situation. Also, the mean-square stability is guaranteed by the follower. In addition, for the infinite horizon case, the specific algorithm procedure is put forward to obtain the incentive feedback Stackelberg strategy conveniently. At last, two examples are given to verify the effectiveness of the proposed algorithm procedure.

Automated verification and synthesis of stochastic hybrid systems: A survey

Stochastic hybrid systems have received significant attentions as a relevant modeling framework describing many systems, from engineering to the life sciences: they enable the study of numerous applications, including transportation networks, biological systems and chemical reaction networks, smart energy and power grids, and beyond. Automated verification and policy synthesis for stochastic hybrid systems can be inherently challenging: this is due to the heterogeneity of their dynamics (presence of continuous and discrete components), the presence of uncertainty, and in some applications the large dimension of state and input sets. Over the past few years, a few hundred articles have investigated these models, and developed diverse and powerful approaches to mitigate difficulties encountered in the analysis and synthesis of such complex stochastic systems. In this survey, we overview the most recent results in the literature and discuss different approaches, including (in)finite abstractions, verification and synthesis for temporal logic specifications, stochastic similarity relations, (control) barrier certificates, compositional techniques, and a selection of results on continuous-time stochastic systems; we finally survey recently developed software tools that implement the discussed approaches. Throughout the manuscript we discuss a few open topics to be considered as potential future research directions: we hope that this survey will guide younger researchers through a comprehensive understanding of the various challenges, tools, and solutions in this enticing and rich scientific area.

Suboptimal Filtering Over Sensor Networks With Random Communication

The problem of filter design is considered for continuous-time linear stochastic systems using distributed sensors. Each sensor unit, represented by a node in an undirected and connected graph, collects some information about the state and communicates its own estimate with the neighbors. It is stipulated that this communication between sensor nodes connected by an edge is time-sampled randomly and for each edge, the sampling process is an independent Poisson counter. Our proposed filtering algorithm for each sensor node is a stochastic hybrid system: It comprises a continuous-time differential equation, and at random time instants when communication takes place, each sensor node updates its state estimate based on the information received by its neighbors. In this setting, we compute the expectation of the error covariance matrix for each unit which is governed by a matrix differential equation. To study the asymptotic behavior of these covariance matrices, we show that if the gain matrices are appropriately chosen and the mean sampling rate is large enough, then the error covariances practically converge to a constant matrix.

Compositional construction of control barrier functions for continuous-time stochastic hybrid systems

In this work, we propose a compositional framework for the construction of control barrier functions for networks of continuous-time stochastic hybrid systems enforcing complex logic specifications expressed by finite-state automata. The proposed scheme is based on a notion of so-called pseudo-barrier functions computed for subsystems, by employing which one can synthesize hybrid controllers for interconnected systems enforcing complex specifications over a finite-time horizon. Particularly, we first leverage sufficient small-gain type conditions to compositionally construct control barrier functions for interconnected systems based on the corresponding pseudo-barrier functions computed for subsystems. Then, using the constructed control barrier functions, we provide probabilistic guarantees on the satisfaction of given complex specifications in a bounded time horizon. In this respect, we decompose the given complex specification to simpler reachability tasks based on automata representing the complements of original finite-state automata. We then provide systematic approaches to solve those simpler reachability tasks by computing corresponding pseudo-barrier functions. Two different systematic techniques are provided based on (i) the sum-of-squares (SOS) optimization program and (ii) counter-example guided inductive synthesis (CEGIS) to search for pseudo-barrier functions of subsystems while synthesizing local controllers. We demonstrate the effectiveness of our proposed results by applying them to a fully-interconnected Kuramoto network of 100 nonlinear oscillators with Markovian switching signals.