Discrete-time Stochastic Control Systems Research Articles

H∞ Control of Discrete-Time Stochastic State-multiplicative Systems Constrained in State by Equality Constraints

Design conditions for the existence of the H∞ state feedback control for discrete-time stochastic systems with state-multiplicative noise stabilizing the closed-loop in such a way that the state variables satisfy equality constraints in the mean are presented in the paper. Using an enhanced form of the bounded real lemma for discrete-time stochastic systems with state-multiplicative noise, the LMI-based procedure is provided for computation of the gain matrix of state control law and the influence of equality constraints is explained. The approach is illustrated on example demonstrating the validity of the proposed method.

Algorithms to Solve StochasticH2/H∞Control with State-Dependent Noise

This paper is concerned with the algorithms which solveH2/H∞control problems of stochastic systems with state-dependent noise. Firstly, the algorithms for the finite and infinite horizonH2/H∞control of discrete-time stochastic systems are reviewed and studied. Secondly, two algorithms are proposed for the finite and infinite horizonH2/H∞control of continuous-time stochastic systems, respectively. Finally, several numerical examples are presented to show the effectiveness of the algorithms.

Linear-Quadratic Control of Discrete-Time Stochastic Systems with Indefinite Weight Matrices and Mean-Field Terms

In this paper, the linear-quadratic optimal control problem is considered for discrete-time stochastic systems with indefinite weight matrices in the cost function and mean-field terms in both the cost function and system dynamics. A set of generalized difference Riccati equations (GDREs) is introduced in terms of algebraic equality constraints and matrix pseudo-inverse. It is shown that the solvability of the GDRE is not only sufficient but also necessary for the well-posedness of the indefinite mean-field linear-quadratic optimal control problem and the existence of optimal feedback as well as open-loop controls.

Linear Unbiased Optimal Filter for Discrete-Time Systems with One-Step Random Delays and Inconsecutive Packet Dropouts

This paper is concerned with the linear unbiased minimum variance estimation problem for discrete-time stochastic linear control systems with one-step random delay and inconsecutive packet dropout. A new model is developed to describe the phenomena of the one-step delay and inconsecutive packet dropout by employing a Bernoulli distributed stochastic variable. Based on the model, a recursive linear unbiased optimal filter in the linear minimum variance sense is designed by the method of completing the square. The solution to the linear filter is given by three equations including a Riccati equation, a Lyapunov equation and a simple difference equation. A sufficient condition for the existence of the steady-state filter is given. A simulation shows the effectiveness of the proposed algorithm. DOI: http://dx.doi.org/10.11591/telkomnika.v11i11.3523

Discrete-time stochastic control systems: A continuous Lyapunov function implies robustness to strictly causal perturbations

Discrete-time stochastic systems employing possibly discontinuous state-feedback control laws are addressed. Allowing discontinuous feedbacks is fundamental for stochastic systems regulated, for instance, by optimization-based control laws. We introduce generalized random solutions for discontinuous stochastic systems to guarantee the existence of solutions and to generate enough solutions to get an accurate picture of robustness with respect to strictly causal perturbations. Under basic regularity conditions, the existence of a continuous stochastic Lyapunov function is sufficient to establish that asymptotic stability in probability for the closed-loop system is robust to sufficiently small, state-dependent, strictly causal, worst-case perturbations. Robustness of a weaker stochastic stability property called recurrence is also shown in a global sense in the case of state-dependent perturbations, and in a semiglobal practical sense in the case of persistent perturbations. An example shows that a continuous stochastic Lyapunov function is not sufficient for robustness to arbitrarily small worst-case disturbances that are not strictly causal. Our positive results are also illustrated by examples.

A new iteration to coupled discrete-time generalized Riccati equations

We consider a set of discrete-time generalized Riccati equations that arise in quadratic optimal control of discrete-time stochastic systems subjected to both state-dependent noise and Markovian jumps. The iterative method to compute the maximal and stabilizing solution of wide class of discrete-time nonlinear equations is derived by Dragan et al. (Int J Control 83(4):837–847, 2010). Here we introduce a new approach to compute the maximal solution of a system of discrete-time generalized Riccati equations. Convergence properties of the new iteration are analyzed. Numerical experiments have been executed and numerical effectiveness estimated.

Nonfragile Gain-Scheduled Control for Discrete-Time Stochastic Systems with Randomly Occurring Sensor Saturations

This paper is devoted to tackling the control problem for a class of discrete-time stochastic systems with randomly occurring sensor saturations. The considered sensor saturation phenomenon is assumed to occur in a random way based on the time-varying Bernoulli distribution with measurable probability in real time. The aim of the paper is to design a nonfragile gain-scheduled controller with probability-dependent gains which can be achieved by solving a convex optimization problem via semidefinite programming method. Subsequently, a new kind of probability-dependent Lyapunov functional is proposed in order to derive the controller with less conservatism. Finally, an illustrative example will demonstrate the effectiveness of our designed procedures.

Mean square BIBO stability of discrete-time stochastic control systems with delays and nonlinear perturbations

The problem of mean square bounded-input bounded-output(BIBO) stability is investigated for a class of discrete-time stochastic control systems with time delays and non-linear perturbations. In this paper, a special point δ in the time delay's variation interval is introduced, and the variation interval is divided into two subintervals. Then, by defining a special Lyapunov-Krasovskii functional and checking its variation in the two subintervals, respectively, some novel delay-dependent stability criteria for the discrete-time stochastic control systems are derived. These conditions are expressed in the forms of linear matrix inequalities(LMIs), whose feasibility can be easily checked by using Matlab LMI Toolbox. Meanwhile, this paper provides a new method for studying discrete-time stochastic mean square BIBO stability. Finally, a numerical example is given to illustrate the validity of the main results.

New Results on Robust Stability and Stabilization of Linear Discrete-Time Stochastic Systems with Convex Polytopic Uncertainties

This paper addresses the robust stability for a class of linear discrete-time stochastic systems with convex polytopic uncertainties. The system to be considered is subject to both interval time-varying delays and convex polytopic type uncertainties. Based on the augmented parameter-dependent Lyapunov-Krasovskii functional, new delay-dependent conditions for the robust stability are established in terms of linear matrix inequalities. An application to robust stabilization of linear discrete-time stochastic control systems is given. Numerical examples are included to illustrate the effectiveness of our results.

BIBO Stabilization of Discrete-Time Stochastic Control Systems with Mixed Delays and Nonlinear Perturbations

The problem of bounded-input bounded-output (BIBO) stabilization in mean square for a class of discrete-time stochastic control systems with mixed time-varying delays and nonlinear perturbations is investigated. Some novel delay-dependent stability conditions for the previously mentioned system are established by constructing a novel Lyapunov-Krasovskii function. These conditions are expressed in the forms of linear matrix inequalities (LMIs), whose feasibility can be easily checked by using MATLAB LMI Toolbox. Finally, a numerical example is given to illustrate the validity of the obtained results.

First Passage Problems for Nonstationary Discrete-Time Stochastic Control Systems

This paper concerns first passage problems for nonstationary nonlinear discrete-time stochastic control systems. The state and control spaces are general Borel spaces, the transition probabilities are nonstationary, the costs/rewards are time-varying and may be unbounded. The optimal control problem is to minimize the expected discounted costs incurred until the first passage time to some target set, in which the discount factors are allowed to be both time- and state-dependent. Under reasonably mild conditions we establish the so-called first passage optimality equations, and prove that the optimal cost/reward functions satisfy the optimality equations. Furthermore, from the optimality equations we show the existence of optimal Markov policies.

A delay fractioning approach to robust sliding mode control for discrete-time stochastic systems with randomly occurring non-linearities

A delay fractioning approach to robust sliding mode control for discrete-time stochastic systems with randomly occurring non-linearities

Control of Discrete-Time Stochastic Systems with State Equality Constraints

AbstractDesign conditions for existence of state feedback control stabilizing discrete-time stochastic systems with multiplicative noise where closed-loop state variables satisfy equality constraints in the mean are presented in the paper. Using classical feedback control principle the LMI-based procedure is provided for computation of the gain matrix of state control laws, and influence of equality constraints is explained. The approach is successfully illustrated on example demonstrating validity of the proposed method using an equality constraint tying together all state variables of the system.

Filtering for Networked Stochastic Time-Delay Systems With Sector Nonlinearity

This paper is concerned with the filtering problem for a class of discrete-time stochastic nonlinear networked control systems with network-induced incomplete measurements. The incomplete measurements include both the multiple random communication delays and random packet losses, which are modeled by a unified stochastic expression in terms of a set of indicator functions that is dependent on certain stochastic variable. The nonlinear functions are assumed to satisfy the sector nonlinearities. The purpose of the addressed filtering problem is to design a linear filter such that the filtering-error dynamics is exponentially mean-square stable. By using the linear-matrix-inequality (LMI) method and delay-dependent technique, sufficient conditions are derived which are dependent on the occurrence probability of both the random communication delays and missing measurement. The filter gain is then characterized by the solution to a set of LMIs. A simulation example is exploited to demonstrate the effectiveness of the proposed design procedures.

Optimal Sensor Querying: General Markovian and LQG Models With Controlled Observations

This paper is motivated by networked control systems deployed in a large-scale sensor network where data collection from all sensors is prohibitive. We model it as a class of discrete-time stochastic control systems for which the observations available to the controller are not fixed, but there are a number of options to choose from, and each choice has a cost associated with it. The observation costs are added to the running cost of the optimization criterion and the resulting optimal control problem is investigated. Since only part of the observations are available at each time step, the controller has to balance the system performance with the penalty of the requested information (query). We first formulate the problem for a general partially observed Markov decision process model and then specialize to the stochastic linear quadratic Gaussian problem. We focus primarily on the ergodic control problem and analyze this in detail.

Infinite horizon stochastic [formula omitted] control for discrete-time systems with state and disturbance dependent noise

This paper studies the infinite horizon mixed H 2 / H ∞ control for discrete-time stochastic systems with state and disturbance dependent noise. We shall first establish a version of stochastic bounded real lemma (SBRL) which by itself has theoretical importance. Based on the SBRL, it is shown that under the condition of exact observability, the existence of a static state feedback H 2 / H ∞ controller is equivalent to the solvability of four coupled matrix-valued equations. A suboptimal H 2 / H ∞ controller design is given based on a convex optimization approach and an iterative algorithm is proposed to solve the four coupled matrix-valued equations.

Constrained control of discrete-time stochastic systems

The purpose of the paper is to present a class of algorithms using to solve the optimization problem concerning with H∞feedback control for linear discrete-time stochastic systems with stochastic parameter uncertainties, as well as a method for the optimization problem reducing this to a standard formulation used two convex inequalities, which can be solved by linear matrix inequality (LMI) methodology. Some generalized considerations for the algorithm procedures are given and the problem of LMI set-up, using the technique based on Schur complement, for calculating the terminal weight matrix P is outlined. Finally, obtained results are adapted by that way to design constrained H∞feedback control for system which state variable satisfy equality constraints in the mean. The procedure results the constant feedback for the linear controller defined in terms of matrix inequalities and a matrix equality.

Adaptive control of stochastic systems with unknown disturbance distribution: discounted criteria

We consider a class of discrete-time stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to the equation x t +1=F(x t , a t , ξ t ), t=0, 1, ..., where the ξ t are i.i.d. random vectors whose common distribution is unknown. Assuming observability of {ξ t }, we use the empirical estimator of its distribution to construct adaptive policies which are asymptotically discounted cost optimal .

H∞-type control for discrete-time stochastic systems

In this paper we consider discrete-time, linear stochastic systems with random state and input matrices which are subjected to stochastic disturbances and controlled by dynamic output feedback. The aim is to develop an H∞-type theory for such systems. For this class of systems a stochastic bounded real lemma is derived which provides the basis for a linear matrix inequality (LMI) approach similar to, but more general than the one presented in Reference 1 for stochastic differential systems. Necessary and sufficient conditions are derived for the existence of a stabilizing controller which reduces the norm of the closed-loop perturbation operator to a level below a given threshold γ. These conditions take the form of coupled nonlinear matrix inequalities. In the absence of the stochastic terms they get reduced to the linear matrix inequalities of deterministic H∞-theory for discrete time systems. Copyright © 1999 John Wiley & Sons, Ltd.

Robust estimation without positive real condition

The strictly positive real (SPR) condition on the noise model is necessary for a discrete-time linear stochastic control system with unmodeled dynamics, even so for a time-invariant ARMAX system, in the past robust analysis of parameter estimation. However, this condition is hardly satisfied for a high-order and/or multidimensional system with correlated noise. The main work in this paper is to show that for robust parameter estimation and adaptive tracking, as well as closed-loop system stabilization, the SPR condition is replaced by a stable matrix polynomial. The main method is to design a "two-step" recursive least squares algorithm with or without a weighted factor and with a fixed lag regressive vector and to define an adaptive control with bounded external excitation and with randomly varying truncation.