Stochastic Algebraic Riccati Equations Research Articles

Stochastic Algebraic Riccati Equations Are Almost as Easy as Deterministic Ones Theoretically

Stochastic Algebraic Riccati Equations Are Almost as Easy as Deterministic Ones Theoretically

An improved iterative algorithm for solving optimal tracking control problems of stochastic systems

This paper proposes an improved iterative algorithm based on a Newton iterative algorithm, which is used to research the stochastic linear quadratic optimal tracking (SLQT) control for stochastic continuous-time systems. Firstly, a Newton iterative algorithm for solving stochastic algebraic Riccati equations (SAREs) is derived based on the Frechet derivative. Secondly, an improved iterative algorithm in an incremental form for indirectly solving SAREs is presented by combining the numerical iterative method with the incremental Newton iterative (INI) algorithm and introducing a tuning parameter. Under a suitable initial condition, the convergence of the improved iterative algorithm is talked over. Finally, numerical simulation experiments are carried out to compare our proposed improved iterative algorithm with some known works. The results show that the improved iterative algorithm has a faster convergence rate.

Incremental Newton’s iterative algorithm for optimal control of Itô stochastic systems

In this paper, a novel incremental Newton’s iterative algorithm for investigating the optimal control problem of Itô stochastic systems is presented. Newton’s method is employed under the Fréchet derivative framework to iteratively solve a stochastic algebraic Riccati equation. Under moderate conditions, the convergence and even quadratic convergence of the proposed incremental Newton’s iterative algorithm are discussed, respectively. In addition, the Newton’s method is extended to the one with linear search. In the end, numerical results are given to demonstrate the effectiveness and superiority of the proposed algorithms.

Full Information H2 Control of Borel-Measurable Markov Jump Systems with Multiplicative Noises

This paper addresses an H2 optimal control problem for a class of discrete-time stochastic systems with Markov jump parameter and multiplicative noises. The involved Markov jump parameter is a uniform ergodic Markov chain taking values in a Borel-measurable set. In the presence of exogenous white noise disturbance, Gramian characterization is derived for the H2 norm, which quantifies the stationary variance of output response for the considered systems. Moreover, under the condition that full information of the system state is accessible to measurement, an H2 dynamic optimal control problem is shown to be solved by a zero-order stabilizing feedback controller, which can be represented in terms of the stabilizing solution to a set of coupled stochastic algebraic Riccati equations. Finally, an iterative algorithm is provided to get the approximate solution of the obtained Riccati equations, and a numerical example illustrates the effectiveness of the proposed algorithm.

On the Linear Quadratic Optimal Control for Systems Described by Singularly Perturbed Itô Differential Equations with Two Fast Time Scales

In this paper a stochastic optimal control problem described by a quadratic performance criterion and a linear controlled system modeled by a system of singularly perturbed Itô differential equations with two fast time scales is considered. The asymptotic structure of the stabilizing solution (satisfying a prescribed sign condition) to the corresponding stochastic algebraic Riccati equation is derived. Furthermore, a near optimal control whose gain matrices do not depend upon small parameters is discussed.

Incentive Stackelberg Games for Stochastic Linear Systems With H∞ Constraint.

In this paper, incentive Stackelberg games with one leader and multiple followers are investigated for a class of stochastic linear systems with external disturbance. Unlike the existing ordinary Stackelberg games, the leader is required to design an incentive Stackelberg strategy set that can lead to the leader's team-optimal solution and the follower's Nash equilibrium, and attenuate the external disturbance in the system simultaneously. It is shown that the incentive Stackelberg strategy set is obtained by solving a set of the cross-coupled stochastic Riccati differential equations in a finite-horizon case and a set of the cross-coupled stochastic algebraic Riccati equations in an infinite-horizon case. Numerical examples are solved to demonstrate the effectiveness of the proposed incentive Stackelberg strategy set.

A study of robust controller and observer design method for a rolling object and beam system

This paper deals with a robust control problem for a ball and beam system with an uncertainty. At first, we clarify the effect of a measurement error in the beam rotational angle and its differential value. It is expressed as additional terms in the state and input matrix of the state equation, which is called as a structured uncertainty. For this system, a robust tracking controller and a robust state observer are applied. Both components are designed based on a robust control, which is called guaranteed cost control. This is a kind of cost minimization method for designing problem of a linear feedback system. The controller is obtained as the solution of the matrix algebraic Riccati like equation, which is referred to as stochastic algebraic Riccati equation. The nominal system performance is degraded due to the influence of the uncertainty, but it is able to design a system with robustness for disturbance. Next, we consider the design problem of a state observer to estimate all state variables form the output signal easy to measure, like rotation angle of the beam. For the design problem of the state observer, we consider dual problem of the original system. Consequently, our proposed system estimates the whole state vector, and using estimated state values, it controls the ball to the desired position. Both components have robustness for the uncertainty. Numerical result shows the validity of our proposed method.

Optimal control for a singularly perturbed linear stochastic system with multiplicative white noise perturbations and Markovian jumping

SummaryIn this study, we investigate the optimal control of a class of singularly perturbed linear stochastic systems with Markovian jumping parameters. After establishing an asymptotic structure for the stabilizing solution of the coupled stochastic algebraic Riccati equations, a parameter‐independent composite controller is derived. Furthermore, the cost degradation in a reduced‐order controller is discussed. Thus, the exactness of the proposed approximate control is discussed for the first time. As an additional important contribution, a numerical algorithm for solving the coupled stochastic algebraic Riccati equations is proposed, and the feature of the resulting higher‐order controller is shown. Finally, a simple example is presented to demonstrate the validity of the proposed method. Copyright © 2016 John Wiley & Sons, Ltd.

Discrete-time mean-field Stochastic linear–quadratic optimal control problems, II: Infinite horizon case

This paper first presents results on the equivalence of several notions of L2-stability for linear mean-field stochastic difference equations with random initial value. Then, it is shown that the optimal control of a mean-field linear–quadratic optimal control with an infinite time horizon uniquely exists, and the optimal control can be expressed as a linear state feedback involving the state and its mean, via the minimal nonnegative definite solution of two coupled algebraic Riccati equations. As a byproduct, the open-loop L2-stabilizability is proved to be equivalent to the closed-loop L2-stabilizability. Moreover, the minimal nonnegative definite solution, the maximal solution, the stabilizing solution of the algebraic Riccati equations and their relations are carefully investigated. Specifically, it is shown that the maximal solution is employed to construct the optimal control and value function to another infinite time horizon mean-field linear–quadratic optimal control. In addition, the maximal solution being the stabilizing solution, is completely characterized by properties of the coefficients of the controlled system. This enriches the existing theory about stochastic algebraic Riccati equations. Finally, the notion of exact detectability is introduced with its equivalent characterization of stochastic versions of the Popov–Belevitch–Hautus criteria. It is then shown that the minimal nonnegative definite solution is the stabilizing solution if and only if the uncontrolled system is exactly detectable.

Decentralized <inline-formula> <tex-math notation="TeX">$H_{2}$</tex-math></inline-formula> Control for Multi-Channel Stochastic Systems

This technical note investigates a decentralized H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> control problem for multi-channel linear time-invariant stochastic systems governed by Itô's differential equation. After using a stochastic algebraic Riccati equation (SARE) to establish the necessary conditions for the existence of the optimal strategy set that minimizes an H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm, we show that the necessary conditions can also be expressed using a linear matrix inequality (LMI). The equivalence between the solvability of the SARE and the feasibility of the LMI is proved for the first time using a Karush-Kuhn-Tucker (KKT) condition. Furthermore, the static output feedback solution was also studied. A numerical example is given to demonstrate the usefulness of the obtained features.

Perturbation analysis of the stochastic algebraic Riccati equation

Abstract In this paper we study a general class of stochastic algebraic Riccati equations (SARE) arising from the indefinite linear quadratic control and stochastic H ∞ problems. Using the Brouwer fixed point theorem, we provide sufficient conditions for the existence of a stabilizing solution of the perturbed SARE. We obtain a theoretical perturbation bound for measuring accurately the relative error in the exact solution of the SARE. Moreover, we slightly modify the condition theory developed by Rice and provide explicit expressions of the condition number with respect to the stabilizing solution of the SARE. A numerical example is applied to illustrate the sharpness of the perturbation bound and its correspondence with the condition number. MSC: Primary 15A24; 65F35; secondary 47H10, 47H14.

Infinite time horizon nonzero-sum linear quadratic stochastic differential games with state and control-dependent noise

This paper discusses the infinite time horizon nonzero-sum linear quadratic (LQ) differential games of stochastic systems governed by Ito’s equation with state and control-dependent noise. First, the nonzero-sum LQ differential games are formulated by applying the results of stochastic LQ problems. Second, under the assumption of mean-square stabilizability of stochastic systems, necessary and sufficient conditions for the existence of the Nash strategy are presented by means of four coupled stochastic algebraic Riccati equations. Moreover, in order to demonstrate the usefulness of the obtained results, the stochastic H-two/H-infinity control with state, control and external disturbance-dependent noise is discussed as an immediate application.

Positive definite solution of a class of nonlinear matrix equation

In this paper, we consider a new nonlinear matrix equation , which is a special stochastic algebraic Riccati equation arising in stochastic control theory. Based on Brower’s fixed point theorem and Bhaskar–Lakshmikantham’s fixed point theroem, we derive some new sufficient conditions for the existence of a (unique) positive definite solution, which are easy to check. An iterative method is proposed to compute the unique positive definite solution. The error estimation of the iterative method is also given. Numerical examples show that the iterative method is feasible.

Infinite horizon linear quadratic stochastic Nash differential games of Markov jump linear systems with its application

In this paper, we deal with the problem of stochastic Nash differential games of Markov jump linear systems governed by Itô-type equation. Combining the stochastic stabilizability with the stochastic systems, a necessary and sufficient condition for the existence of the Nash strategy is presented by means of a set of cross-coupled stochastic algebraic Riccati equations. Moreover, the stochastic H2/H∞ control for stochastic Markov jump linear systems is discussed as an immediate application and an illustrative example is presented.

Residual bounds of the stochastic algebraic Riccati equation

In this paper we consider a class of continuous-time algebraic Riccati equations with a constraint of positive definiteness, which occurs in the indefinite stochastic linear quadratic control problems and stochastic H∞ control problems, respectively. The normwise local and non-local residual bounds are derived for a symmetric solution which approximates the unique stabilizing solution to the stochastic algebraic Riccati equation. A numerical example is presented to illustrate the sharpness of ours residual bound.

Nash strategy for multiparameter singularly perturbed Markov jump stochastic systems

This study investigates Nash games for a class of multiparameter singularly perturbed stochastic systems governed by Ito’s differential equation with Markov jump parameters. First, in order to obtain Nash equilibrium strategies, cross-coupled stochastic algebraic Riccati equations (CSAREs) are formulated. Moreover, necessary condition for the existence of solution for CSAREs is also developed. It is noteworthy that this is the first time that conditions for the existence of stochastic equilibria have been derived based on the solutions of sets of CSAREs. After establishing an asymptotic structure with positive definiteness for CSAREs solutions, feasible numerical algorithms that are based on Newton’s method and the linear matrix inequality (LMI) for solving CSAREs is considered. Finally, the authors provide a numerical example to verify the efficiency of the proposed algorithms.

Computational Method for Solving a Stochastic Linear-Quadratic Control Problem Given an Unsolvable Stochastic Algebraic Riccati Equation

We discuss a stochastic linear-quadratic control problem in which a stochastic algebraic Riccati equation derived from the problem is unsolvable. The Riccati equation has no solution when the state and control weighting matrices in the objective function of the problem are indefinite, and the conventional methods cannot solve the problem when the Riccati equation itself is unsolvable. We first show that the optimal value of the problem is finite. Next, we formulate a compromise solution to the stochastic algebraic Riccati equation and show that the problem can be solved via this compromise solution under certain assumptions. Moreover, we propose a novel computational method for finding the compromise solution based on iterative techniques for a convex optimization problem over the fixed point set of a certain nonexpansive mapping. Numerical examples demonstrate the effectiveness of this method.

Discrete‐Time Indefinite Stochastic LQ Control via SDP and LMI Methods

This paper studies a discrete‐time stochastic LQ problem over an infinite time horizon with state‐and control‐dependent noises, whereas the weighting matrices in the cost function are allowed to be indefinite. We mainly use semidefinite programming (SDP) and its duality to treat corresponding problems. Several relations among stability, SDP complementary duality, the existence of the solution to stochastic algebraic Riccati equation (SARE), and the optimality of LQ problem are established. We can test mean square stabilizability and solve SARE via SDP by LMIs method.

Indefinite LQ Control for Discrete‐Time Stochastic Systems via Semidefinite Programming

This paper is concerned with a discrete‐time indefinite stochastic LQ problem in an infinite‐time horizon. A generalized stochastic algebraic Riccati equation (GSARE) that involves the Moore‐Penrose inverse of a matrix and a positive semidefinite constraint is introduced. We mainly use a semidefinite‐programming‐ (SDP‐) based approach to study corresponding problems. Several relations among SDP complementary duality, the GSARE, and the optimality of LQ problem are established.

The Linear Quadratic Regulator Problem for a Class of Controlled Systems Modeled by Singularly Perturbed Itô Differential Equations

This paper discusses an infinite-horizon linear quadratic (LQ) optimal control problem involving state- and control-dependent noise in singularly perturbed stochastic systems. First, an asymptotic structure along with a stabilizing solution for the stochastic algebraic Riccati equation (ARE) are newly established. It is shown that the dominant part of this solution can be obtained by solving a parameter-independent system of coupled Riccati-type equations. Moreover, sufficient conditions for the existence of the stabilizing solution to the problem are given. A new sequential numerical algorithm for solving the reduced-order AREs is also described. Based on the asymptotic behavior of the ARE, a class of $O(\sqrt{\varepsilon})$ approximate controller that stabilizes the system is obtained. Unlike the existing results in singularly perturbed deterministic systems, it is noteworthy that the resulting controller achieves an $O(\varepsilon)$ approximation to the optimal cost of the original LQ optimal control problem. As a result, the proposed control methodology can be applied to practical applications even if the value of the small parameter $\varepsilon$ is not precisely known.