Generalized Algebraic Riccati Equation Research Articles

Pareto efficiency of infinite-horizon cooperative stochastic differential games with Markov jumps and Poisson jumps

This paper investigates a wide class of infinite-horizon cooperative stochastic differential games with Markov jumps and Poisson jumps, which can model uncertainties of internal mode transition and characterize occasional sudden changes in the games, respectively. Firstly, the necessary conditions which guarantee the existence of Pareto solutions are obtained by utilizing the Lagrange multiplier method and the stochastic maximum principle with Markov jumps and Poisson jumps. Then the sufficient conditions which guarantee the existence of Pareto efficient strategies are derived. Secondly, the well-posedness of cooperative stochastic differential games (CSDG) with Markov jumps and Poisson jumps in infinite horizon is established when the solution of generalized algebraic Riccati equation (GARE) exists. Furthermore, we can obtain Pareto solutions by introducing the coupled algebraic Lyapunov equations (ALEs). Finally, the numerical examples verify the theoretical results.

Pareto Optimal Strategy Under H∞ Constraint for the Mean-Field Stochastic Systems in Infinite Horizon.

This article focuses on the mean-field linear-quadratic Pareto (MF-LQP) optimal strategy design for stochastic systems in infinite horizon, which is with the H∞ constraint when the system is disturbed by external interferences. The stochastic bounded real lemma (SBRL) with any initial state in infinite horizon is first investigated based on the stabilizing solution of the generalized algebraic Riccati equation (GARE). Then, by discussing the convexity of the cost functional, the stochastic indefinite MF-LQP control problem is defined and solved based on the MF-LQ theory and Pareto theory. When the worst case disturbance is considered in the collaborative multiplayer system, we show that the Pareto optimal strategy design with H∞ constraint [or robust Pareto optimal strategy, (RPOS)] can be given via solving two coupled GAREs. When the worst case disturbance and the Pareto efficient strategy work, all Pareto solutions are obtained by a generalized Lyapunov equation. Finally, a practical example shows that the obtained results are effective.

Pareto efficiency of finite-horizon mean-field cooperative stochastic differential games with poisson jumps

This paper investigates finite-horizon mean-field cooperative stochastic differential games (CSDG) with Poisson jumps. The game model can describe the case where the game has external disturbances and suffers sudden changes. First, for the fixed initial state, the necessary conditions which guarantee the existence of Pareto solutions can be obtained by utilizing the Lagrange multiplier method and the mean-field type stochastic maximum principle (MF-SMP) with Poisson jumps. Then, the sufficient conditions which guarantee the existence of Pareto efficient strategies are derived. Next, for any initial states, the well-posedness of finite-horizon mean-field CSDG with Poisson jumps is established under the assumption that the solution of generalized algebraic Riccati equations (GAREs) exists. Further, all Pareto solutions can be obtained by constructing an algebraic Lyapunov equations (ALEs). Finally, a numerical example given illustrates our results.

Optimal control and stabilization for linear mean‐field system with indefinite quadratic cost functional

AbstractIn this paper, we consider an optimal control and stabilization problem of linear mean‐field (MF) system, where the quadratic cost functional is allowed to be indefinite. Inspired by the equivalent cost functional method, we introduce a subset, which helps us to investigate the convergence property of generalized differential Riccati equations arising in indefinite mean‐field linear quadratic (MF‐LQ) problems with finite horizon. A coupled generalized algebraic Riccati equation (GARE) is thus obtained. More importantly, the solution pair of corresponding GARE can be decomposed into a solution pair of a coupled standard algebraic Riccati equation (SARE) and a matrix pair in the subset we introduce. Thus, an equivalent relationship is established between GARE and SARE. With this equivalence, we derive necessary and sufficient conditions to stabilize linear MF‐system with indefinite weighting matrices in mean‐square sense.

Mixed H2/H∞-Policy Learning Synthesis

A robustly stabilizing optimal control policy in a model-free mixed H2/H∞-control setting is here put forward for counterbalancing the slow convergence and non-robustness of traditional high-variance policy optimization (and by extension policy gradient) algorithms. Leveraging Itô’s stochastic differential calculus, we iteratively solve the system's continuous-time (closed-loop) generalized algebraic Riccati equation(GARE) whilst updating its admissible controllers in a two-player, zero-sum differential game setting. Our new results are illustrated by learning-enabled control systems which gather previously disseminated results in this field in one holistic data-driven presentation with greater simplification, improvement, and clarity.

Pareto efficiency in the infinite horizon mean-field type cooperative stochastic differential game

This paper discusses the linear quadratic (LQ) mean-field type stochastic differential game with non-homogeneous terms in infinite horizon. Employing the equivalent description of Pareto efficiency, necessary conditions for the existence of Pareto solutions are presented under an assumption on the Lagrange multiplier set. Two conditions are introduced to guarantee that the assumption is established. Further, sufficient conditions for a control to be Pareto efficient are put forward in terms of the necessary conditions, the convexity condition on the homogeneous weighted sum cost functional and a transversality condition. The characterization of Pareto solutions is also studied for the homogeneous case. If the system is MF-L2-stabilizable, then the solvability of the related generalized algebraic Riccati equations (GAREs) provides a sufficient condition under which the cost functionals are convex and all Pareto efficient strategies can be obtained by the weighted sum minimization method. In addition, by introducing two algebraic Lyapunov equations (ALEs), we derive all Pareto solutions.

Discrete-Time Non-Zero-Sum Games With Completely Unknown Dynamics.

In this article, off-policy reinforcement learning (RL) algorithm is established to solve the discrete-time N -player nonzero-sum (NZS) games with completely unknown dynamics. The N -coupled generalized algebraic Riccati equations (GARE) are derived, and then policy iteration (PI) algorithm is used to obtain the N -tuple of iterative control and iterative value function. As the system dynamics is necessary in PI algorithm, off-policy RL method is developed for discrete-time N -player NZS games. The off-policy N -coupled Hamilton-Jacobi (HJ) equation is derived based on quadratic value functions. According to the Kronecker product, the N -coupled HJ equation is decomposed into unknown parameter part and the system operation data part, which makes the N -coupled HJ equation solved independent of system dynamics. The least square is used to calculate the iterative value function and N -tuple of iterative control. The existence of Nash equilibrium is proved. The result of the proposed method for discrete-time unknown dynamics NZS games is indicated by the simulation examples.

Solvability and optimal stabilization controls of discrete-time mean-field stochastic system with infinite horizon

The paper addresses the optimal control and stabilization problems for the indefinite discrete-time mean-field system over infinite horizon. Firstly, we show the convergence of the generalized algebraic Riccati equations (GAREs) and establish their compact form GARE. By dealing with the GARE, we derive the existence of the maximal solution to the original GAREs along with the fact that the maximal solution is the stabilizing solution. Then, the maximal solution is employed to design the linear-quadratic (LQ) optimal controller and the optimal value of the control problem. Specifically, we deduce that under the assumption of exact observability, the mean-field system is L^{2}-stabilizable if and only if the GAREs have a solution, which is also the maximal solution. By semi-definite programming (SDP) method, the solvability of the GAREs is discussed. Our results generalize and improve previous results. Finally, some numerical examples are exploited to illustrate the validity of the obtained results.

Feedback Stackelberg strategies for the discrete-time mean-field stochastic systems in infinite horizon

This paper deals with the feedback Stackelberg strategies for the discrete-time mean-field stochastic systems in infinite horizon. The optimal control problem of the follower is first studied. Employing the discrete-time linear quadratic (LQ) mean-field stochastic optimal control theory, the sufficient conditions for the solvability of the optimization of the follower are presented and the optimal control is obtained based on the stabilizing solutions of two coupled generalized algebraic Riccati equations (GAREs). Then, the optimization of the leader is transformed into a constrained optimal control problem. Applying the Karush-Kuhn-Tucker (KKT) conditions, the necessary conditions for the existence and uniqueness of the Stackelberg strategies are derived and the Stackelberg strategies are expressed as linear feedback forms involving the state and its mean based on the solutions (Ki,K^i),i=1,2 of a set of cross-coupled stochastic algebraic equations (CSAEs). An iterative algorithm is put forward to calculate efficiently the solutions of the CSAEs. Finally, an example is solved to show the effectiveness of the proposed algorithm.

Infinite horizon linear quadratic Pareto game of the stochastic singular systems

This paper is concerned with the linear quadratic (LQ) Pareto game of the stochastic singular systems in infinite horizon. Firstly, the optimal control problem of the weighted sum cost functional is discussed. Utilizing the equivalent transformation method, the weighted sum LQ optimal control problem is transformed into a stochastic LQ optimization problem. Based on the classical stochastic LQ optimal control theory, the necessary and sufficient condition for the solvability of the indefinite weighted sum LQ optimal control is put forward. Then, the LQ Pareto game of the stochastic singular systems is studied. By the discussion of the convexity of the cost functionals, a sufficient condition for the existence of the Pareto solutions is obtained via the solvability of the corresponding generalized algebraic Riccati equation (GARE). Moreover, we derive all Pareto solutions based on the solution of a Lyapunov equation. Finally, an example is given to show the effectiveness of the proposed results.

Stochastic Consentability of Linear Systems With Time Delays and Multiplicative Noises

This paper develops stochastic consentability of linear multiagent systems with time delays and multiplicative noises. First, the stochastic stability for stochastic differential delay equations driven by multiplicative noises is examined, and the existence of the positive definite solution for a class of generalized algebraic Riccati equations (GAREs) is established. Then, sufficient conditions are deduced for the mean square and almost sure consentability and stabilization based on the developed stochastic stability and GAREs. Consensus protocols are designed for linear multiagent systems with undirected and leader-following topologies. It is revealed that multiagent consentability depends on certain characterizing system parameters, including linear system dynamics, communication graph, channel uncertainties, and time delay of the deterministic term. It is shown that a second-order integrator multiagent system is unconditionally mean square and almost surely consentable for any given noise intensities and time delay, and that the mean square and almost sure consensus can be achieved by carefully choosing the control gain according to certain explicit conditions.

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.

Consensus robust output regulation of discrete-time linear multi-agent systems

This paper deals with consensus robust output regulation of discrete-time linear multi-agent systems under a directed interaction topology. The digraph is assumed to contain a spanning tree. Every agent or subsystem is identical and uncertain, but subsystems have different external disturbances. Based on the internal model and general discrete-time algebraic Riccati equation, a distributed consensus protocol is proposed to solve the regulator problem. A numerical simulation demonstrates the effectiveness of the proposed theoretical results.

Stability and Linear Quadratic Differential Games of Discrete-Time Markovian Jump Linear Systems with State-Dependent Noise

We mainly consider the stability of discrete-time Markovian jump linear systems with state-dependent noise as well as its linear quadratic (LQ) differential games. A necessary and sufficient condition involved with the connection between stochasticTn-stability of Markovian jump linear systems with state-dependent noise and Lyapunov equation is proposed. And using the theory of stochasticTn-stability, we give the optimal strategies and the optimal cost values for infinite horizon LQ stochastic differential games. It is demonstrated that the solutions of infinite horizon LQ stochastic differential games are concerned with four coupled generalized algebraic Riccati equations (GAREs). Finally, an iterative algorithm is presented to solve the four coupled GAREs and a simulation example is given to illustrate the effectiveness of it.

Feedback Stabilization for a Class of Nonlinear Stochastic Systems with State- and Control-Dependent Noise

This paper mainly studies the state feedback stabilizability of a class of nonlinear stochastic systems with state- and control-dependent noise. Some sufficient conditions on local and global state feedback stabilizations are given in linear matrix inequalities (LMIs) and generalized algebraic Riccati equations (GAREs). Some obtained results improve the previous work.

Formula omitted] control with unstable and nonproper weights for descriptor systems

This paper is concerned with a nonstandard H∞ output feedback control problem for continuous-time descriptor systems, where unstable and nonproper weighting functions are used. Based on two generalized Sylvester equations and two generalized algebraic Riccati equations (GAREs) together with a spectral radius condition, an explicit parametrization of all desirable controllers is deduced. A numerical example is included to illustrate the validity of the proposed result.

Critical stability and stabilization of discrete-time stochastic systems and its applications

In this paper, critical stability and critical stabilization for discrete stochastic systems with both state and control dependent noise are discussed via the spectrum technique. The Popov-Belevitch-Hautus (PBH) criterion for exact observability in a discrete version is presented. As applications, some interesting results on a class of generalized Lyapunov equations (GLE), unremovable spectra and discrete generalized algebraic Riccati equation (GARE) are obtained. Finally, the problem of assigning the spectra of discrete stochastic systems in a specified disk is considered and some numerical examples are given to demonstrate our results.

Extended ${\rm H}_{2}$ Controller Synthesis for Continuous Descriptor Systems

This technical note presents a complete solution to the nonstandard H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> output feedback control problem for continuous descriptor systems where unstable and nonproper weighting functions are used. In such a problem, the desired controller has to satisfy two conditions simultaneously: (i) the closed-loop is admissible and has a minimum H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm, (ii) only the internal stability of a part of the closed-loop is sought. The condition of the existence of such a controller is deduced. An explicit characterization of the optimal solution is also formulated, based on two generalized algebraic Riccati equations (GAREs) and two generalized Sylvester equations. A numerical example is included to illustrate the validity of the proposed results.

Discrete-time Indefinite Stochastic LQ Optimal Control: Infinite Horizon Case

The asymptotic analysis method is applied to the infinite horizon discrete-time indefinite stochastic LQ problem. This paper carries out the research on the basis of the results of its finite horizon counterpart and the mean-square stabilizing assumption. Properties of the solutions to the generalized algebraic Riccati equation (GARE) are also considered. Finally, two examples are provided to justify the developed theory.

Infinite horizon linear quadratic optimal control for discrete‐time stochastic systems

AbstractThis paper is concerned with the infinite horizon linear quadratic optimal control for discrete‐time stochastic systems with both state and control‐dependent noise. Under assumptions of stabilization and exact observability, it is shown that the optimal control law and optimal value exist, and the properties of the associated discrete generalized algebraic Riccati equation (GARE) are also discussed. Copyright © 2008 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society