Continuous-time Linear Stochastic Systems Research Articles

Robust Recursive Least-Squares Fixed-Point Smoother and Filter using Covariance Information in Linear Continuous-Time Stochastic Systems with Uncertainties

This study develops robust recursive least-squares (RLS) fixed-point smoothing and filtering algorithms for signals in linear continuous-time stochastic systems with uncertainties. The algorithms use covariance information, such as the cross-covariance function of the signal with the observed value and the autocovariance function of the degraded signal. A finite Fourier cosine series expansion approximates these functions. Additive white Gaussian noise is present in the observation of the degraded signal. A numerical simulation compares the estimation accuracy of the proposed robust RLS filter with the robust RLS Wiener filter, showing similar mean square values (MSVs) of the filtering errors. The MSVs of the proposed robust RLS fixed-point smoother are also compared to those of the proposed robust RLS filter.

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Martingale Additive Noise

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Martingale Additive Noise

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Multiplicative Noise

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Multiplicative Noise

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

This article studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed, which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness.

Direct Adaptive Control for Stochastic Systems with Risk-Sensitive Indices

We propose a direct adaptive control law based on the adaptive dynamic programming (ADP) algorithm for continuous-time stochastic linear systems with partially unknown system dynamics and infinite horizon quadratic risk-sensitive indices. A control design methodology is employed to iteratively solve the generalized algebraic Riccati equation by using the online information of the state and input, and to directly learn the optimal control law. We prove the convergence of the online ADP algorithm and show that the direct adaptive control law approximates the optimal control law as time goes on. Finally, a numerical simulation example is presented to demonstrate the effectiveness of our algorithm.

Exact Detectability of Discrete-Time and Continuous-Time Linear Stochastic Systems: A Unified Approach

This article is devoted to the problem of detectability of a large class of linear stochastic systems with time varying coefficients simultaneously affected by state multiplicative white noise perturbations and Markovian switching. The main contribution of this article is to propose a Popov–Belevich–Hautus-type test, which is equivalent to the detectability of the considered stochastic systems in the sense that all unstable modes produce some nonzero output. The proposed setting unifies in some sense the discrete-time and continuous-time ones, dissimilarly to the vast majority of existing works that study discrete and continuous time separately.

Suboptimal Filtering Over Sensor Networks With Random Communication

The problem of filter design is considered for continuous-time linear stochastic systems using distributed sensors. Each sensor unit, represented by a node in an undirected and connected graph, collects some information about the state and communicates its own estimate with the neighbors. It is stipulated that this communication between sensor nodes connected by an edge is time-sampled randomly and for each edge, the sampling process is an independent Poisson counter. Our proposed filtering algorithm for each sensor node is a stochastic hybrid system: It comprises a continuous-time differential equation, and at random time instants when communication takes place, each sensor node updates its state estimate based on the information received by its neighbors. In this setting, we compute the expectation of the error covariance matrix for each unit which is governed by a matrix differential equation. To study the asymptotic behavior of these covariance matrices, we show that if the gain matrices are appropriately chosen and the mean sampling rate is large enough, then the error covariances practically converge to a constant matrix.

Instabilizability Conditions for Continuous-Time Stochastic Systems Under Control Input Constraints

In this letter, we investigate constrained control of continuous-time linear stochastic systems. We show that for certain system parameter settings, constrained control policies can never achieve stabilization. Specifically, we explore a class of control policies that are constrained to have a bounded average second moment for Ito-type stochastic differential equations with additive and multiplicative noise. We prove that in certain settings of the system parameters and the bounding constant of the control constraint, divergence of the second moment of the system state is inevitable regardless of the initial state value and regardless of how the control policy is designed.

Bayesian Algorithms Learn to Stabilize Unknown Continuous-Time Systems

Linear dynamical systems are canonical models for learning-based control of plants with uncertain dynamics. The setting consists of a stochastic differential equation that captures the state evolution of the plant understudy, while the true dynamics matrices are unknown and need to be learned from the observed data of state trajectory. An important problem is to ensure that the system is stabilized and destabilizing control actions due to model uncertainties are precluded as soon as possible. A reliable stabilization procedure for this purpose that can learn from unstable data to stabilize the system in finite time, is not currently available. In this work, we establish sufficient learning-accuracy bounds for guaranteeing stabilization and propose a novel Bayesian algorithm that stabilizes unknown continuous-time stochastic linear systems. The presented algorithm is flexible and exposes effective stabilization performance after a remarkably short time period of interacting with the system.

Constrained regulation problem for continuous-time stochastic systems under state and control constraints

Constrained regulation problem (CRP) for continuous-time stochastic systems is investigated in this article. New existence conditions of linear feedback control law for continuous-time stochastic systems under constraints are proposed. The computation method for solving constrained regulation problem of stochastic systems considered in this article is also presented. Continuous-time stochastic linear systems and stochastic nonlinear systems are focused on, respectively. First, the condition of polyhedral invariance for stochastic systems is established by using the theory of positive invariant set and the principle of comparison. Second, the asymptotic stability conditions in the sense of expectation for two types of stochastic systems are established. Finally, finding the linear feedback controller model and corresponding algorithm of constrained regulation problem for two types of stochastic systems are also proposed by using the obtained condition. The presented model of the stochastic constrained regulation problem in this article is formulated as a linear programming problem, which can be easily implemented from a computational point of view. Our approach establishes a connection between the stochastic constrained regulation problem and positively invariant set theory, as well as provides the possibility of using optimization methodology to find the solution of stochastic constrained regulation problem, which differs from other methods. Numerical examples illustrate the proposed method.

Modified spectral method for optimal estimation in linear continuous-time stochastic systems

The spectral method to solve estimation problems for linear continuous-time stochastic systems with polynomial measurements is presented. It is based on both the spectral form of mathematical description (the representation of deterministic functions and random processes by orthogonal series) and the particle filter. The main goal of this work is to implement the continuous-time particle filter without a time discretization. The proposed spectral method provides the possibility to solve estimation problems such as filtering, smoothing and prediction.

Preview control for a class of linear stochastic systems with multiplicative noise

ABSTRACT In this paper, the preview control problem for a class of linear continuous time stochastic systems with multiplicative noise is studied based on the augmented error system method. First, a deterministic assistant system is introduced, and the original system is translated to the assistant system. Then, the integrator is employed to ensure the output of the closed-loop system tracking the reference signal accurately. Second, the augmented error system, which includes integrator vector, control vector and reference signal, is constructed based on the system after translation. As a result, the tracking problem is transformed into the optimal control problem of the augmented error system, and the optimal control input is obtained by the dynamic programming method. This control input is regarded as the preview controller of the original system. For a linear stochastic system with multiplicative noise, the difficulty being unable to construct an augmented error system by the derivation method is solved in this paper. And, the existence and uniqueness solution of the Riccati equation corresponding to the stochastic augmented error system is discussed. The numerical simulations show that the preview controller designed in this paper is very effective.

Predictor-based control of stochastic systems with nonlinear diffusions and input delay

We consider the problem of state-feedback control of linear continuous-time stochastic systems with nonlinear diffusion terms affected by time-varying input delays or, equivalently, by time-varying state measurement delays. We propose a finite-dimensional control law based on closed-loop predictors and derive sufficient delay bounds for the exponential stability in mean and exponential mean square stability with prescribed rate.

Static output feedback H2/H∞ control with spectrum constraints for stochastic systems subject to multiplicative noises

ABSTRACTThis paper investigates the problem of static output feedback control with spectrum constraints for linear continuous-time stochastic systems with state dependent noises. By using the spectrum technique, the notion of stability for continuous-time stochastic systems is proposed. Subsequently, the design method of the static output feedback controller is presented by solving a set of linear matrix inequalities. Under stability, a sufficient condition for the existence of stochastic control is derived, which transforms the problem of control into a convex optimization problem. Finally, a numerical example is given to show the effectiveness of the obtained results.

Applications of the preview control method to the optimal problem for linear continuous-time stochastic systems with time-delay

This paper discusses the optimal control problem for a class of linear continuous-time stochastic systems with time-delay based on the preview control method. In the design procedure, the time-delay terms are formally eliminated and the augmented error system method is used. The optimal controller of the time-delay stochastic system is obtained based on dynamic programming principle. In the previous approaches, the preview control method is applied to continuous-time deterministic system. Consequently, compared with the existing results, the main contributions of this paper are: (1) the conclusion of the non-singular transformation is improved and (2) the conclusion in deterministic system is extended by using the methods of designing an assistant system and introducing an integrator when building the stochastic augmented error system. The optimal tracking of the stochastic system with time-delay is realized by solving the optimal regulation problem of the stochastic augmented error system. The simulation shows the effectiveness of the controller designed in this paper.

RLS Fixed-Lag Smoother Using Covariance Information Based on Innovation Approach in Linear Continuous Stochastic Systems

This paper newly designs the RLS (recursive least-squares) fixed-lag smoother and filter, based on the innovation theory, in linear continuous-time stochastic systems. It is assumed that the signal is observed with additive white noise and the signal is uncorrelated with the observation noise. It is a characteristic that the estimators use the covariance information of the signal, in the form of the semi-degenerate kernel, and the observation noise. With respect to the RLS fixed-lag smoother, the algorithm for the estimation error variance function is developed to guarantee the stability of the fixed-lag smoother. The proposed estimators have the recursive property in calculating the fixed-lag smoothing and filtering estimates. Also, this paper proposes the Chandrasekhar-type RLS Wiener filter in linear wide-sense stationary stochastic system. Unlike the usual filter including the Riccati-type equations, the Chandrasekhar-type filter does not contain the Riccati-type differential equations and has an advantage of eliminating the possibility of the covariance matrix becoming nonnegative.

Stability Criterion of Linear Stochastic Systems Subject to MixedH2/PassivityPerformance

TheH2control scheme and passivity theory are applied to investigate the stability criterion of continuous-time linear stochastic system subject to mixed performance. Based on the stochastic differential equation, the stochastic behaviors can be described as multiplicative noise terms. For the considered system, theH2control scheme is applied to deal with the problem on minimizing output energy. And the asymptotical stability of the system can be guaranteed under desired initial conditions. Besides, the passivity theory is employed to constrain the effect of external disturbance on the system. Moreover, the Itô formula and Lyapunov function are used to derive the sufficient conditions which are converted into linear matrix inequality (LMI) form for applying convex optimization algorithm. Via solving the sufficient conditions, the state feedback controller can be established such that the asymptotical stability and mixed performance of the system are achieved in the mean square. Finally, the synchronous generator system is used to verify the effectiveness and applicability of the proposed design method.

A new filtering approach for continuous-time linear systems with delayed measurements

This paper introduces a new filter for linear continuous-time stochastic systems with delayed measurements. The approach is inspired by an observer designed for deterministic systems. The obtained solution is suboptimal and does not use distributed integration terms with advantages in terms of computational load. The relationship between the delay bound and the variance of the estimation error is formally characterized and confirmed by a numerical example.

Design of RLS-FIR filter using covariance information in linear continuous-time stochastic systems

This paper addresses a new design method of recursive least-squares (RLS) and finite impulse response (FIR) filter, using covariance information, in linear continuous-time stochastic systems. The signal process is observed with additive white noise. It is assumed that the white observation noise is independent of the signal process. The auto-covariance function of the signal is expressed in the semi-degenerate kernel form. The RLS-FIR filter uses the following information:1.The auto-covariance function of the signal expressed in the semi-degenerate kernel form.2.The variance of the white observation noise process.3.The observed values.

Robust L∞-induced deconvolution filtering for linear stochastic systems and its application to fault reconstruction

The problem of stationary robust L∞-induced deconvolution filtering for the uncertain continuous-time linear stochastic systems is addressed. The state space model of the system contains state- and input-dependent noise and deterministic parameter uncertainties residing in a given polytope. In the presence of input-dependent noise, we extend the derived Lemma in [22] characterizing the induced L∞ norm by linear matrix inequalities (LMIs), according to which we solve the deconvolution problem in the quadratic framework. By decoupling product terms between the Lyapunov matrix and system matrices, an improved version of the proposed L∞-induced norm bound Lemma for continuous-time stochastic systems is obtained, which allows us to realize exploit parameter-dependent stability idea in the deconvolution filter design. The theories presented are utilized for sensor fault reconstruction in uncertain linear stochastic systems. The effectiveness and advantages of the proposed design methods are shown via two numerical examples.