Continuous-time Linear Stochastic Systems Research Articles

Robust Recursive Least-Squares Finite Impulse Response Filter in Linear Continuous-Time Stochastic Systems with Uncertainties

The current research designs an original robust recursive least-squares (RLS) finite impulse response (FIR) filter for linear continuous-time systems with uncertainties in both the system and observation matrices. These uncertainties in the state-space model generate the degraded signal and observed value. The robust RLS FIR filter does not account for the norm-bounded uncertainties in the system and observation matrices. This study uses an observable companion form to represent the degraded signal state-space model. The system and observation matrices are estimated based on the author's previous computational methods. The robust RLS FIR filtering problem aims to minimize the mean-square errors in FIR filtering for the system state. The robust FIR filtering estimate is formulated as an integral transformation of the degraded observations using an impulse response function. Section 3 obtains the integral equation satisfied by the optimal impulse response function. Theorem 1 presents the robust RLS FIR filtering algorithms for the signal and the system state. This integral equation derives the robust RLS-FIR filtering algorithms. Numerical simulation examples show the validity of the proposed robust RLS FIR filter.

DUALITY: DETECTABILITY VERSUS STABILIZABILITY IN THE STOCHASTIC CONTEXT

The aim of this paper is to extend to the stochastic framework the well known duality relation between the detectability property and the stabilizability property of a finite dimensional, linear time invariant, deterministic system. We consider continuous time linear stochastic systems having the state space representation described by a system of Itˆo differential equations with periodic coefficients possibly subject to some stochastic perturbations modeled by a standard homogeneous Markov process with a finite number of states. It will be seen that in the case when the given system is affected by jump Markov perturba tions a state space representation of the dual triple may be rigorously defined if and only if the transition probability matrix of the Markov process is double stochastic (in the sense that the sums of all elements from its each row and each column are equal 1).

Robust Recursive Least-Squares Fixed-Point Smoother and Filter using Covariance Information in Linear Continuous-Time Stochastic Systems with Uncertainties

This study develops robust recursive least-squares (RLS) fixed-point smoothing and filtering algorithms for signals in linear continuous-time stochastic systems with uncertainties. The algorithms use covariance information, such as the cross-covariance function of the signal with the observed value and the autocovariance function of the degraded signal. A finite Fourier cosine series expansion approximates these functions. Additive white Gaussian noise is present in the observation of the degraded signal. A numerical simulation compares the estimation accuracy of the proposed robust RLS filter with the robust RLS Wiener filter, showing similar mean square values (MSVs) of the filtering errors. The MSVs of the proposed robust RLS fixed-point smoother are also compared to those of the proposed robust RLS filter.

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Martingale Additive Noise

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Martingale Additive Noise

Convergence Analysis of Ensemble Filters for Linear Stochastic Systems with Poisson-Sampled Observations

For continuous-time linear stochastic dynamical systems driven by Wiener processes, we consider the problem of designing ensemble filters when the observation process is randomly time-sampled. We propose a continuous-discrete McKean–Vlasov type diffusion process with additive Gaussian noise in observation model, which is used to describe the evolution of the individual particles in the ensemble. These particles are coupled through the empirical covariance and require less computations for implementation than the optimal ones based on solving Riccati differential equations. Using appropriate analysis tools, we show that the empirical mean and the sample covariance of the ensemble filter converges to the mean and covariance of the optimal filter if the mean sampling rate of the observation process satisfies certain bounds and as the number of particles tends to infinity.

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Multiplicative Noise

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Multiplicative Noise

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

This article studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed, which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness.

Direct Adaptive Control for Stochastic Systems with Risk-Sensitive Indices

We propose a direct adaptive control law based on the adaptive dynamic programming (ADP) algorithm for continuous-time stochastic linear systems with partially unknown system dynamics and infinite horizon quadratic risk-sensitive indices. A control design methodology is employed to iteratively solve the generalized algebraic Riccati equation by using the online information of the state and input, and to directly learn the optimal control law. We prove the convergence of the online ADP algorithm and show that the direct adaptive control law approximates the optimal control law as time goes on. Finally, a numerical simulation example is presented to demonstrate the effectiveness of our algorithm.

Exact Detectability of Discrete-Time and Continuous-Time Linear Stochastic Systems: A Unified Approach

This article is devoted to the problem of detectability of a large class of linear stochastic systems with time varying coefficients simultaneously affected by state multiplicative white noise perturbations and Markovian switching. The main contribution of this article is to propose a Popov–Belevich–Hautus-type test, which is equivalent to the detectability of the considered stochastic systems in the sense that all unstable modes produce some nonzero output. The proposed setting unifies in some sense the discrete-time and continuous-time ones, dissimilarly to the vast majority of existing works that study discrete and continuous time separately.

Suboptimal Filtering Over Sensor Networks With Random Communication

The problem of filter design is considered for continuous-time linear stochastic systems using distributed sensors. Each sensor unit, represented by a node in an undirected and connected graph, collects some information about the state and communicates its own estimate with the neighbors. It is stipulated that this communication between sensor nodes connected by an edge is time-sampled randomly and for each edge, the sampling process is an independent Poisson counter. Our proposed filtering algorithm for each sensor node is a stochastic hybrid system: It comprises a continuous-time differential equation, and at random time instants when communication takes place, each sensor node updates its state estimate based on the information received by its neighbors. In this setting, we compute the expectation of the error covariance matrix for each unit which is governed by a matrix differential equation. To study the asymptotic behavior of these covariance matrices, we show that if the gain matrices are appropriately chosen and the mean sampling rate is large enough, then the error covariances practically converge to a constant matrix.

Instabilizability Conditions for Continuous-Time Stochastic Systems Under Control Input Constraints

In this letter, we investigate constrained control of continuous-time linear stochastic systems. We show that for certain system parameter settings, constrained control policies can never achieve stabilization. Specifically, we explore a class of control policies that are constrained to have a bounded average second moment for Ito-type stochastic differential equations with additive and multiplicative noise. We prove that in certain settings of the system parameters and the bounding constant of the control constraint, divergence of the second moment of the system state is inevitable regardless of the initial state value and regardless of how the control policy is designed.

Bayesian Algorithms Learn to Stabilize Unknown Continuous-Time Systems

Linear dynamical systems are canonical models for learning-based control of plants with uncertain dynamics. The setting consists of a stochastic differential equation that captures the state evolution of the plant understudy, while the true dynamics matrices are unknown and need to be learned from the observed data of state trajectory. An important problem is to ensure that the system is stabilized and destabilizing control actions due to model uncertainties are precluded as soon as possible. A reliable stabilization procedure for this purpose that can learn from unstable data to stabilize the system in finite time, is not currently available. In this work, we establish sufficient learning-accuracy bounds for guaranteeing stabilization and propose a novel Bayesian algorithm that stabilizes unknown continuous-time stochastic linear systems. The presented algorithm is flexible and exposes effective stabilization performance after a remarkably short time period of interacting with the system.

Constrained regulation problem for continuous-time stochastic systems under state and control constraints

Constrained regulation problem (CRP) for continuous-time stochastic systems is investigated in this article. New existence conditions of linear feedback control law for continuous-time stochastic systems under constraints are proposed. The computation method for solving constrained regulation problem of stochastic systems considered in this article is also presented. Continuous-time stochastic linear systems and stochastic nonlinear systems are focused on, respectively. First, the condition of polyhedral invariance for stochastic systems is established by using the theory of positive invariant set and the principle of comparison. Second, the asymptotic stability conditions in the sense of expectation for two types of stochastic systems are established. Finally, finding the linear feedback controller model and corresponding algorithm of constrained regulation problem for two types of stochastic systems are also proposed by using the obtained condition. The presented model of the stochastic constrained regulation problem in this article is formulated as a linear programming problem, which can be easily implemented from a computational point of view. Our approach establishes a connection between the stochastic constrained regulation problem and positively invariant set theory, as well as provides the possibility of using optimization methodology to find the solution of stochastic constrained regulation problem, which differs from other methods. Numerical examples illustrate the proposed method.

Modified spectral method for optimal estimation in linear continuous-time stochastic systems

The spectral method to solve estimation problems for linear continuous-time stochastic systems with polynomial measurements is presented. It is based on both the spectral form of mathematical description (the representation of deterministic functions and random processes by orthogonal series) and the particle filter. The main goal of this work is to implement the continuous-time particle filter without a time discretization. The proposed spectral method provides the possibility to solve estimation problems such as filtering, smoothing and prediction.

Preview control for a class of linear stochastic systems with multiplicative noise

ABSTRACT In this paper, the preview control problem for a class of linear continuous time stochastic systems with multiplicative noise is studied based on the augmented error system method. First, a deterministic assistant system is introduced, and the original system is translated to the assistant system. Then, the integrator is employed to ensure the output of the closed-loop system tracking the reference signal accurately. Second, the augmented error system, which includes integrator vector, control vector and reference signal, is constructed based on the system after translation. As a result, the tracking problem is transformed into the optimal control problem of the augmented error system, and the optimal control input is obtained by the dynamic programming method. This control input is regarded as the preview controller of the original system. For a linear stochastic system with multiplicative noise, the difficulty being unable to construct an augmented error system by the derivation method is solved in this paper. And, the existence and uniqueness solution of the Riccati equation corresponding to the stochastic augmented error system is discussed. The numerical simulations show that the preview controller designed in this paper is very effective.

Predictor-based control of stochastic systems with nonlinear diffusions and input delay

We consider the problem of state-feedback control of linear continuous-time stochastic systems with nonlinear diffusion terms affected by time-varying input delays or, equivalently, by time-varying state measurement delays. We propose a finite-dimensional control law based on closed-loop predictors and derive sufficient delay bounds for the exponential stability in mean and exponential mean square stability with prescribed rate.

Static output feedback H2/H∞ control with spectrum constraints for stochastic systems subject to multiplicative noises

ABSTRACTThis paper investigates the problem of static output feedback control with spectrum constraints for linear continuous-time stochastic systems with state dependent noises. By using the spectrum technique, the notion of stability for continuous-time stochastic systems is proposed. Subsequently, the design method of the static output feedback controller is presented by solving a set of linear matrix inequalities. Under stability, a sufficient condition for the existence of stochastic control is derived, which transforms the problem of control into a convex optimization problem. Finally, a numerical example is given to show the effectiveness of the obtained results.

Applications of the preview control method to the optimal problem for linear continuous-time stochastic systems with time-delay

This paper discusses the optimal control problem for a class of linear continuous-time stochastic systems with time-delay based on the preview control method. In the design procedure, the time-delay terms are formally eliminated and the augmented error system method is used. The optimal controller of the time-delay stochastic system is obtained based on dynamic programming principle. In the previous approaches, the preview control method is applied to continuous-time deterministic system. Consequently, compared with the existing results, the main contributions of this paper are: (1) the conclusion of the non-singular transformation is improved and (2) the conclusion in deterministic system is extended by using the methods of designing an assistant system and introducing an integrator when building the stochastic augmented error system. The optimal tracking of the stochastic system with time-delay is realized by solving the optimal regulation problem of the stochastic augmented error system. The simulation shows the effectiveness of the controller designed in this paper.

RLS Fixed-Lag Smoother Using Covariance Information Based on Innovation Approach in Linear Continuous Stochastic Systems

This paper newly designs the RLS (recursive least-squares) fixed-lag smoother and filter, based on the innovation theory, in linear continuous-time stochastic systems. It is assumed that the signal is observed with additive white noise and the signal is uncorrelated with the observation noise. It is a characteristic that the estimators use the covariance information of the signal, in the form of the semi-degenerate kernel, and the observation noise. With respect to the RLS fixed-lag smoother, the algorithm for the estimation error variance function is developed to guarantee the stability of the fixed-lag smoother. The proposed estimators have the recursive property in calculating the fixed-lag smoothing and filtering estimates. Also, this paper proposes the Chandrasekhar-type RLS Wiener filter in linear wide-sense stationary stochastic system. Unlike the usual filter including the Riccati-type equations, the Chandrasekhar-type filter does not contain the Riccati-type differential equations and has an advantage of eliminating the possibility of the covariance matrix becoming nonnegative.

Stability Criterion of Linear Stochastic Systems Subject to MixedH2/PassivityPerformance

TheH2control scheme and passivity theory are applied to investigate the stability criterion of continuous-time linear stochastic system subject to mixed performance. Based on the stochastic differential equation, the stochastic behaviors can be described as multiplicative noise terms. For the considered system, theH2control scheme is applied to deal with the problem on minimizing output energy. And the asymptotical stability of the system can be guaranteed under desired initial conditions. Besides, the passivity theory is employed to constrain the effect of external disturbance on the system. Moreover, the Itô formula and Lyapunov function are used to derive the sufficient conditions which are converted into linear matrix inequality (LMI) form for applying convex optimization algorithm. Via solving the sufficient conditions, the state feedback controller can be established such that the asymptotical stability and mixed performance of the system are achieved in the mean square. Finally, the synchronous generator system is used to verify the effectiveness and applicability of the proposed design method.