Control Of Stochastic Systems Research Articles

Temporal Difference-Based Policy Iteration for Optimal Control of Stochastic Systems

In this paper, a unified policy iteration approach is presented for the optimal control problem of stochastic system with discounted average cost and continuous state space. The approach consists o...

On designing network‐based H ∞ controllers for stochastic systems

SUMMARYThis paper is concerned with network‐based H ∞ stabilization for stochastic systems, where network‐induced delays, packet dropouts, and packet disorders are taken into account simultaneously. The packet disorders arising from both the sampler‐to‐controller channel and the controller‐to‐actuator channel are considered by introducing a logic controller and a logic zero‐order hold. The network‐induced delays and packet dropouts are modeled as a constant delay plus a non‐differentiable time‐varying delay in the input. By employing Lyapunov–Krasovskii functional approach, we establish results that parallel well‐known bounded real Lemmas. More specifically, these results provide conditions to bound the H ∞ level of the system, which means the worst case energy of the output of the system when subjected to a unitary norm deterministic disturbance signal. On the basis of these results, suitable network‐based H ∞ controllers are designed by using cone complementary linearization method. An air vehicle system is finally taken as an example to show the effectiveness of the proposed method. Copyright © 2013 John Wiley & Sons, Ltd.

Hybrid Impulsive Control of Stochastic Systems with Multiplicative Noise Under Markovian Switching

A problem of state output feedback stabilization of discrete-time stochastic systems with multiplicative noise under Markovian switching is considered. Under some appropriate assumptions, the stability of this system under pure impulsive control is given. Further under hybrid impulsive control, the output feedback stabilization problem is investigated. The hybrid control action is formulated as a combination of the regular control along with an impulsive control action. The jump Markovian switching is modeled by a discrete-time Markov chain. The control input is simultaneously applied to both the stochastic and the deterministic terms. Sufficient conditions based on stochastic semi-definite programming and linear matrix inequalities (LMIs) for both stochastic stability and stabilization are obtained. Such a nonconvex problem is solved using the existing optimization algorithms and the nonconvex CVX package. The robustness of the stability and stabilization concepts against all admissible uncertainties are also investigated. The parameter uncertainties we consider here are norm bounded. Two examples are given to demonstrate the obtained results.

Discrete-time stochastic control systems: A continuous Lyapunov function implies robustness to strictly causal perturbations

Discrete-time stochastic systems employing possibly discontinuous state-feedback control laws are addressed. Allowing discontinuous feedbacks is fundamental for stochastic systems regulated, for instance, by optimization-based control laws. We introduce generalized random solutions for discontinuous stochastic systems to guarantee the existence of solutions and to generate enough solutions to get an accurate picture of robustness with respect to strictly causal perturbations. Under basic regularity conditions, the existence of a continuous stochastic Lyapunov function is sufficient to establish that asymptotic stability in probability for the closed-loop system is robust to sufficiently small, state-dependent, strictly causal, worst-case perturbations. Robustness of a weaker stochastic stability property called recurrence is also shown in a global sense in the case of state-dependent perturbations, and in a semiglobal practical sense in the case of persistent perturbations. An example shows that a continuous stochastic Lyapunov function is not sufficient for robustness to arbitrarily small worst-case disturbances that are not strictly causal. Our positive results are also illustrated by examples.

Mean-Square Exponential Input-to-State Stability of Numerical Solutions for Stochastic Control Systems

This paper deals with the mean-square exponential input-to-state stability (exp-ISS) of numerical solutions for stochastic control systems (SCSs). Firstly, it is shown that a finite-time strong convergence condition holds for the stochastic θ-method on SCSs. Then, we can see that the mean-square exp-ISS of an SCS holds if and only if that of the stochastic θ-method (for sufficiently small step sizes) is preserved under the finite-time strong convergence condition. Secondly, for a class of SCSs with a one-sided Lipschitz drift, it is proved that two implicit Euler methods (for any step sizes) can inherit the mean-square exp-ISS property of the SCSs. Finally, numerical examples confirm the correctness of the theorems presented in this study.

Stochastic H 2/H ∞ control with random coefficients

This paper is concerned with the mixed H2/H∞ control for stochastic systems with random coefficients, which is actually a control combining the H2 optimization with the H∞ robust performance as the name of H2/H∞ reveals. Based on the classical theory of linear-quadratic (LQ, for short) optimal control, the sufficient and necessary conditions for the existence and uniqueness of the solution to the indefinite backward stochastic Riccati equation (BSRE, for short) associated with H∞ robustness are derived. Then the sufficient and necessary conditions for the existence of the H2/H∞ control are given utilizing a pair of coupled stochastic Riccati equations.

Robust resilient H ∞ control for stochastic systems with Markovian jump parameters under partially known transition probabilities

SUMMARYIn this paper, the robust resilient H ∞ control problem for a class of stochastic systems with partially known transition probabilities is investigated. The system under consideration finds extensive applications because of the uncertainties involved in the system matrices and the general assumption upon the transition probabilities. Attention is focused upon the design of a robust resilient H ∞ state feedback controller, which guarantees the stochastic stability of the closed‐loop system and a prescribed H ∞ disturbance attenuation level for all admissible uncertainties. Sufficient criteria ensuring the stochastic stability and stabilization of the underlying systems are presented via LMIs formulation. A numerical example is provided to demonstrate the effectiveness of all the results. Copyright © 2013 John Wiley & Sons, Ltd.

Correction to “Packetized Predictive Control of Stochastic Systems Over Bit-Rate Limited Channels With Packet Loss”

The authors correct the results in Section V of the above-named article (ibid., vol. 56, no. 12, pp. 2854-2868, Dec. 2011).

EXISTENCE-AND-UNIQUENESS AND MEAN-SQUARE BOUNDEDNESS OF THE SOLUTION TO STOCHASTIC CONTROL SYSTEMS

This paper mainly deals with the stochastic control system, the existence and uniqueness of solutions and the behavior of solutions are investigated. Firstly, we obtain sufficient conditions which guarantee the existence and uniqueness of solutions to the stochastic control system. And then, boundedness of the solution to the system is achieved under mean-square linear growth condition.

A maximum principle for fully coupled forward–backward stochastic control systems with terminal state constraints

We study a stochastic optimal control problem where the controlled system is described by a fully coupled forward–backward stochastic differential equation (FBSDE), while the forward state is constrained in a convex set at the terminal time. By introducing an equivalent backward control problem, we use terminal variation approach to obtain a stochastic maximum principle. Applications to the utility optimization problem in the financial market and state constrained stochastic linear quadratic control models are investigated.

Second-order Taylor expansion for backward doubly stochastic control system

We consider the second-order Taylor expansion for backward doubly stochastic control system. The results are obtained under no restriction on the convexity of control domain. Moreover, the control variable is allowed in the drift coefficient and the diffusion coefficient.

Input-to-state stability of Euler-Maruyama method for stochastic delay control systems

This paper develops the mean-square exponential input-to-state stability (exp-ISS) of the Euler-Maruyama (EM) method for stochastic delay control systems (SDCSs). The definition of mean-square exp-ISS of numerical methods is established. The conditions of the exact and EM method for an SDCS with the property of mean-square exp-ISS are obtained without involving control Lyapunov functions or functional. Under the global Lipschitz coefficients and mean-square continuous measurable inputs, it is proved that the mean-square exp-ISS of an SDCS holds if and only if that of the EM method is preserved for a sufficiently small step size. The proposed results are evaluated by using numerical experiments to show their effectiveness.

On stochastic optimality for a linear controller with attenuating disturbances

For a linear stochastic control system with quadratic objective functional, we introduce various generalizations of the notions of optimality on average and stochastic optimality on an infinite time interval that take into account possible degeneration of the parameter of the disturbing process with time (attenuation of the disturbances) or the presence of a discount function in the objective functional. This lets us improve upon the quality estimate for a well known optimal control in this problem from the point of view of both asymptotic behavior of the functional's expectation and its asymptotic probabilistic properties. In particular, in the considered case we have found an improvement for the well known logarithmic upper bound on the optimal control for a family of defect processes.

Probabilistic DHP adaptive critic for nonlinear stochastic control systems

Following the recently developed algorithms for fully probabilistic control design for general dynamic stochastic systems (Herzallah & Káarnáy, 2011; Kárný, 1996), this paper presents the solution to the probabilistic dual heuristic programming (DHP) adaptive critic method (Herzallah & Káarnáy, 2011) and randomized control algorithm for stochastic nonlinear dynamical systems. The purpose of the randomized control input design is to make the joint probability density function of the closed loop system as close as possible to a predetermined ideal joint probability density function. This paper completes the previous work (Herzallah & Káarnáy, 2011; Kárný, 1996) by formulating and solving the fully probabilistic control design problem on the more general case of nonlinear stochastic discrete time systems. A simulated example is used to demonstrate the use of the algorithm and encouraging results have been obtained.

Two levels of Bayesian model averaging for optimal control of stochastic systems

Bayesian model averaging provides the best possible estimate of a model, given the data. This article uses that approach twice: once to get a distribution of plausible models of the world, and again to find a distribution of plausible control functions. The resulting ensemble gives control instructions different from simply taking the single best-fitting model and using it to find a single lowest-error control function for that single model. The only drawback is, of course, the need for more computer time: this article demonstrates that the required computer time is feasible. The test problem here is from flood control and risk management.

Dynamic robust stabilization of stochastic differential control systems

The stabilization of stochastic differential control systems by means of dynamic feedback laws is provided. We extend the well-known Artstein–Sontag theorem to derive the necessary and sufficient conditions for the dynamic robust stabilization of stochastic differential systems. An explicit formula for feedback law exhibiting dynamic robust stability in probability is established, and a numerical example is also given to illustrate our results.

BSDEs with general filtration driven by Lévy processes, and an application in stochastic controllability

In this paper, we introduce a weak version of the strong solution (the adapted solution used in Pardoux and Peng (1990) [2]), i.e., the transposition solution, to the backward stochastic differential equation (BSDE) with general filtration and random jumps, and study the corresponding well-posedness. The main tools that we employ are the Riesz representation theorem and the Banach fixed point theorem, without using the martingale representation theorem. As an application, we give a definition of controllability to the stochastic linear control system in the sense of the transposition solution and provide a Kalman-type rank condition to guarantee this property.

Approximate Controllability of Sobolev Type Nonlocal Fractional Stochastic Dynamic Systems in Hilbert Spaces

We study a class of fractional stochastic dynamic control systems of Sobolev type in Hilbert spaces. We use fixed point technique, fractional calculus, stochastic analysis, and methods adopted directly from deterministic control problems for the main results. A new set of sufficient conditions for approximate controllability is formulated and proved. An example is also given to provide the obtained theory.

Fault diagnosis and fault‐tolerant control for non‐Gaussian non‐linear stochastic systems using a rational square‐root approximation model

The purpose of the fault detection and diagnosis of stochastic distribution control systems is to use the measured input and the system output probability density functions (PDFs) to obtain the fault information of the system. In this paper, the rational square-root B-spline model is used to represent the dynamics between the output PDF and the input. This is then followed by the novel design of a non-linear neural network observer-based fault diagnosis (FD) algorithm so as to diagnose the fault in the dynamic part of such systems. Convergency analysis is performed for the error dynamic system raised from the fault detection and diagnosis phase using the Lyapunov stability theorem. Finally, based on the FD information, a new fault-tolerant control based on proportional integral tracking control scheme is designed to make the post-fault PDF still track the given distribution. A simulated example is given to illustrate the efficiency of the proposed algorithms.

On Exponential Stability of Composite Stochastic Control Systems

New results for exponential stability in probability of a composite stochastic control system are established. The main results of this paper enable us to derive sufficient conditions for exponential stability in <svg style="vertical-align:-0.1638pt;width:7.3000002px;" id="M1" height="7.9499998" version="1.1" viewBox="0 0 7.3000002 7.9499998" width="7.3000002" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"> <g transform="matrix(.017,-0,0,-.017,.062,7.675)"><path id="x1D45F" d="M393 379q-9 -16 -28 -29q-15 -10 -23 -2q-19 19 -36 19q-21 0 -52 -38q-57 -72 -82 -126l-40 -197q-23 -3 -75 -18l-7 7q49 196 74 335q7 43 -2 43q-7 0 -30 -14.5t-47 -37.5l-16 23q37 42 82 73t67 31q41 0 15 -113l-11 -50h4q41 71 85 117t77 46q29 0 45 -26&#xA;q13 -21 0 -43z" /></g> </svg>-th mean and almost sure exponential stability in probability of composite stochastic control system. Two numerical examples are given to illustrate the results.