Generalized Riccati Differential Equation Research Articles

Saddle-Point Equilibrium Strategy for Linear Quadratic Uncertain Stochastic Hybrid Differential Games Based on Subadditive Measures

This paper describes a kind of linear quadratic uncertain stochastic hybrid differential game system grounded in the framework of subadditive measures, in which the system dynamics are described by a hybrid differential equation with Wiener–Liu noise and the performance index function is quadratic. Firstly, we introduce the concept of hybrid differential games and establish the Max–Min Lemma for the two-player zero-sum game scenario. Next, we discuss the analysis of saddle-point equilibrium strategies for linear quadratic hybrid differential games, addressing both finite and infinite time horizons. Through the incorporation of a generalized Riccati differential equation (GRDE) and guided by the principles of the Itô–Liu formula, we prove that that solving the GRDE is crucial and serves as both a sufficient and necessary precondition for identifying equilibrium strategies within a finite horizon. In addition, we also acquire the explicit formulations of equilibrium strategies in closed forms, alongside determining the optimal values of the cost function. Through the adoption of a generalized Riccati equation (GRE) and applying a similar approach to that used for the finite horizon case, we establish that the ability to solve the GRE constitutes a sufficient criterion for the emergence of equilibrium strategies in scenarios extending over an infinite horizon. Moreover, we explore the dynamics of a resource extraction problem within a finite horizon and separately delve into an H∞ control problem applicable to an infinite horizon. Finally, we present the conclusions.

Convergence with rates for a Riccati-based discretization of SLQ problems with SPDEs

Abstract We consider a new discretization in space (parameter $h>0$) and time (parameter $\tau>0$) of a stochastic optimal control problem, where a quadratic functional is minimized subject to a linear stochastic heat equation with linear noise. Its construction is based on the perturbation of a generalized difference Riccati equation to approximate the related feedback law. We prove a convergence rate of almost ${\mathcal O}(h^{2}+\tau )$ for its solution, and conclude from it a rate of almost ${\mathcal O}(h^{2}+\tau )$ resp. ${\mathcal O}(h^{2}+\tau ^{1/2})$ for computable approximations of the optimal state and control with additive resp. multiplicative noise.

Relationship between Nash Equilibrium Strategies and H2/H∞ Control of Mean-Field Stochastic Differential Equations with Multiplicative Noise

The relationship between finite-horizon mean-field stochastic H2/H∞ control and Nash equilibrium strategies is investigated in this technical note. First, the finite-horizon mean-field stochastic bounded real lemma (SBRL) is established, which is key to developing the H∞ theory. Second, for mean-field stochastic differential equations (MF-SDEs) with control- and state-dependent noises, it is revealed that the existence of Nash equilibrium strategies is equivalent to the solvability of generalized differential Riccati equations (GDREs). Furthermore, the existence of Nash equilibrium strategies is equivalent to the solvability of H2/H∞ control for MF-SDEs with control- and state-dependent noises. However, for mean-field stochastic systems with disturbance-dependent noises, these two problems are not equivalent. Finally, a sufficient and necessary condition is presented via coupled matrix-valued equations for the finite-horizon H2/H∞ control of mean-field stochastic differential equations with disturbance-dependent noises.

Linear Quadratic Optimal Control of Discrete-Time Stochastic Systems Driven by Homogeneous Markov Processes

Random terms in many natural and social science systems have distinct Markovian characteristics, such as Markov jump-taking values in a finite or countable set, and Wiener process-taking values in a continuous set. In general, these systems can be seen as Markov-process-driven systems, which can be used to describe more complex phenomena. In this paper, a discrete-time stochastic linear system driven by a homogeneous Markov process is studied, and the corresponding linear quadratic (LQ) optimal control problem for this system is solved. Firstly, the relations between the well-posedness of LQ problems and some linear matrix inequality (LMI) conditions are established. Then, based on the equivalence between the solvability of the generalized difference Riccati equation (GDRE) and the LMI condition, it is proven that the solvability of the GDRE is sufficient and necessary for the well-posedness of the LQ problem. Moreover, the solvability of the GDRE and the feasibility of the LMI condition are established, and it is proven that the LQ problem is attainable through a certain feedback control when any of the four conditions is satisfied, and the optimal feedback control of the LQ problem is given using the properties of homogeneous Markov processes and the smoothness of the conditional expectation. Finally, a practical example is used to illustrate the validity of the theory.

A Deterministic Setting for the Numerical Computation of the Stabilizing Solutions to Stochastic Game-Theoretic Riccati Equations

In this paper, we are interested in the numerical aspects of the class of generalized Riccati difference equations which are involved in linear quadratic (LQ) stochastic difference games. More specifically, we address the problem of the numerical computation of the stabilizing solutions for this class of nonlinear difference equations. We propose an iterative deterministic algorithm for the computation of such a global solution. The performances of the proposed algorithm are illustrated with some numerical examples.

A New Analytic Solution Method for a Class of Generalized Riccati Differential Equations

We give a useful and practicable solution method for the general Riccati differential equation of the form $w^{\prime }\left( x\right) =p\left( x\right) +q\left( x\right) w\left( x\right) +r\left( x\right) w^{2}\left( x\right) $. In order to get the general solution many authors have been interested this type equation. They show that if there exists some relation about the coefficients $p\left( x\right),$ $q\left( x\right),$ and $r\left( x\right) $ then the general solution of this equation can be given in a closed form. We also determine some relations between these coefficients and find the general solutions to the given equation. Finally, we give some examples to illustrate the importance of the presented method.

Stochastic optimal control problems of discrete‐time Markov jump systems

AbstractIn this paper, we consider the indefinite stochastic optimal control problems of discrete‐time Markov jump linear systems. Firstly, we establish the new stochastic maximum principle, and by solving the forward‐backward stochastic difference equations with Markov jump (FBSDEs‐MJ), we derive the necessary and sufficient solvability condition of the indefinite control problem with non‐discounted cost, which is in an explicit analytical expression. Then, the optimal control is designed by a series of coupled generalized Riccati difference equations with Markov jump (GRDEs‐MJ) and linear recursive equations with Markov jump (LREs‐MJ). Moreover, based on the non‐discounted cost case, we deduce the optimal control problem with discounted cost. Finally, a numerical example for defined‐benefit (DB) pension fund with regime switching is exploited to illustrate the validity of the obtained results.

Indefinite Mean-Field Stochastic Cooperative Linear-Quadratic Dynamic Difference Game With Its Application to the Network Security Model.

In this article, we show how to obtain all of the Pareto optimal decision vectors and solutions for the finite horizon indefinite mean-field stochastic cooperative linear-quadratic (LQ) difference game. First, the equivalence between the solvability of the introduced N coupled generalized difference Riccati equations (GDREs) and the solvability of the multiobjective optimization problem is established. However, it is difficult to obtain Pareto optimal decision vectors based on the N coupled GDREs because the optimal joint strategy adopted by all players to optimize the performance criterion of some players in the game is different from the strategies of other players, which rely on the weighted matrices of cost functionals that may be different among players. Second, a necessary and sufficient condition is developed to guarantee the convexity of the costs, which makes the weighting technique not only sufficient but also necessary for searching Pareto optimal decision vectors. It is then shown that the mean-field Pareto optimality algorithm (MF-POA) is presented to identify, in principle, all of the Pareto optimal decision vectors and solutions via the solutions to the weighted coupled GDREs and the weighted coupled generalized difference Lyapunov equations (GDLEs), respectively. Finally, a cooperative network security game is reported to illustrate the results presented. Simulation results validate the solvability, correctness, and efficiency of the proposed algorithm.

Pareto Optimal Strategy under H∞ Constraint for Discrete-Time Stochastic Systems

This paper investigates the Pareto optimal strategy of discrete-time stochastic systems under H∞ constraint, in which the weighting matrices of the weighted sum cost function can be indefinite. Combining the H∞ control theory with the indefinite LQ control theory, the generalized difference Riccati equations (GDREs) are obtained. By means of the solution of the GDREs, the Pareto optimal strategy with H∞ constraint is derived, and the necessary and sufficient conditions for the existence of the strategy are presented. Then the Pareto optimal solution under the worst-case disturbance is solved. Finally, the efficiency of the obtained results is illustrated by a numerical example.

Optimal control of stochastic singular affine systems with Markovian jumps

We consider an optimal control problem for a class of stochastic singular affine systems with Markovian jumps. We establish the existence and uniqueness of the solution to stochastic singular affine systems with Markovian jumps for the first time. Via square completion technique and the generalized Itô’s formula, we derive new kinds of generalized differential Riccati equations (GDREs) and generalized backward differential equations (GBDEs), which give sufficient conditions for the well-posedness of the optimal control problem, and present an explicit representation of optimal control. Also, we discuss the solvability of the GDREs in two cases. As an application, we present a leader-follower differential game to demonstrate the practicability of our results.

Necessary/sufficient conditions for Pareto optimality in finite horizon mean-field type stochastic differential game

This paper is concerned with necessary and sufficient conditions for the existence of Pareto solutions in finite horizon mean-field type stochastic cooperative differential game. Based on the equivalent characterization of Pareto optimality, the problem is transformed into a set of constrained mean-field type stochastic optimal control problems with a special structure. Utilizing the mean-field type stochastic minimum principle, the necessary conditions are put forward. Under certain convex assumptions, it is shown that the necessary conditions are also sufficient ones. Next, the indefinite linear quadratic (LQ) case is studied. It is pointed out that the solvability of two related generalized differential Riccati equations (GDREs) provides a sufficient condition under which Pareto efficient strategies are equivalent to weighted sum optimal controls. In addition, all Pareto solutions are obtained based on the solutions of two generalized differential Lyapunov equations (GDLEs). At last, an example sheds light on the effectiveness of the theoretical results.

Discrete‐time mean‐field stochastic linear‐quadratic optimal control problem with finite horizon

AbstractIn this paper we consider the discrete‐time mean‐field stochastic linear‐quadratic (MF‐LQ) optimal control problem with indefinite weighting matrices. First, we establish the maximum principle, and by the solvability of mean‐field forward‐backward stochastic difference equations derived from the maximum principle, we characterize the existence of the open‐loop optimal control for the MF‐LQ problem. Then, by virtue of introducing the linear matrix inequalities condition, we obtain the solvability of the generalized difference Riccati equations (GDREs). Moreover, we show that the indefinite MF‐LQ problem is well‐posed if and only if the GDREs are solvable. Finally, a numerical example is used to show the effectiveness of the obtained results.

On the bounded and stabilizing solution of a generalized Riccati differential equation arising in connection with a zero‐sum linear quadratic stochastic differential game

SummaryWe study a class of coupled nonlinear matrix differential equations arising in connection with the solution of a zero‐sum two‐player linear quadratic (LQ) differential game for a dynamical system modeled by an Itô differential equation subject to random switching of its coefficients. The system of differential equations under consideration contains as special cases the game‐theoretic Riccati differential equations arising in the solution of the H∞ control problem from the deterministic and stochastic cases. A set of sufficient conditions that guarantee the existence of the bounded and stabilizing solution of this kind of Riccati differential equations is provided. We show how such stabilizing solution is involved in the construction of the equilibrium strategy of a zero‐sum LQ stochastic differential game on an infinite‐time horizon and give as a byproduct the solution of such a control problem.

Pareto optimal strategy for linear stochastic systems with H∞ constraint in finite horizon

In this paper, we derive conditions for the existence of Pareto efficient strategy and Pareto solution under H∞ constraint for the linear quadratic (LQ) finite horizon cooperative differential game of stochastic systems with state-, control- and disturbance-multiplicative noise. Firstly, we present a stochastic bounded real lemma (SBRL) with any initial condition for the considered stochastic system. Next, a necessary and sufficient condition for Pareto optimal strategy under the H∞ constraint is researched by two cross-coupled generalized differential Riccati equations (GDREs), which is the key contribution of this paper. Another contribution is that the Pareto solution for every controller is given under the Pareto efficient strategy and worst-case external disturbance. Finally, a practical example is given to illustrate the effectiveness of our results.

Finite horizon [formula omitted] control for SDEs with infinite Markovian jumps

This paper concerns the finite horizon H2∕H∞ control for a broad class of linear Itô stochastic differential equations (SDEs) with infinite Markovian jumps and (x,u,v)-dependent noise. We derive stochastic bounded real lemma (SBRL) and linear quadratic (LQ) optimal control result for the considered system at first. Further, a necessary and sufficient condition, which is represented by the solution of a countably infinite set of coupled generalized difference Riccati equations (GDREs), is proposed for the existence of the mixed H2∕H∞ control. Moreover, an iterative algorithm is given to solve GDREs.

[formula omitted] index for continuous-time stochastic systems with Markov jump and multiplicative noise

This paper investigates the problem of ℋ− index for stochastic linear continuous-time systems involving Markov jump and multiplicative noise. A set of generalized differential Riccati equations (GDREs) are presented and it is shown that the solvability of GDREs is necessary and sufficient for the feasibility of an ℋ−index larger than γ>0. Our results extend the corresponding deterministic cases to stochastic systems. Furthermore, the infinite horizon ℋ− index problem for square systems is considered. Finally, the effectiveness of the obtained results is tested by two examples.

Linearized filtering of affine processes using stochastic Riccati equations

We consider an affine process X which is only observed up to an additive white noise, and we ask for the law of Xt, for some t>0, conditional on all observations up to time t. This is a general, possibly high dimensional filtering problem which is not even locally approximately Gaussian, whence essentially only particle filtering methods remain as solution techniques. In this work we present an efficient numerical solution by introducing an approximate filter for which conditional characteristic functions can be calculated by solving a system of generalized Riccati differential equations depending on the observation and the process characteristics of X. The quality of the approximation can be controlled by easily observable quantities in terms of a macro location of the signal in state space. Asymptotic techniques as well as maximization techniques can be directly applied to the solutions of the Riccati equations leading to novel very tractable filtering formulas. The efficiency of the method is illustrated with numerical experiments for Cox–Ingersoll–Ross and Wishart processes, for which Gaussian approximations usually fail.

Homotopy Perturbation Method for Solving Generalized Riccati Differential Equation

In this work, we have studied a class of nonlinear Riccati differential equation which is used as mathematical models in many physically significant fields and applied science. The homotopy perturbation method, has been modified for solving generalized linear and nonlinear Riccati differential equation, which is a second-order ordinary differential equation. Also, we have tested the modified homotopy perturbation method on the solving of different implementations which are show the efficiency and accuracy of the proposed method. The approximated solutions are agreed well with analytical solutions for the tested problems.

<italic>H</italic>_ Index for Linear Time-Varying Markov Jump Stochastic Systems and Its Application to Fault Detection

In this paper, the H _ index for Markov jump linear time-varying stochastic systems and its application to robust H _ fault detection filter (FDF) are under consideration. First, a set of finite horizon backward generalized differential Riccati equations (GDREs) and a set of matrix inequalities are introduced. Based on the introduced backward GDREs and matrix inequalities, for nominal Markov jump linear time-varying stochastic systems, two necessary and sufficient conditions for the finite horizon H _ index larger than a given prescribed level $\beta > 0$ are given. Second, for norm uncertain Markov jump linear time-varying stochastic systems, sufficient conditions are presented in terms of matrix inequalities. As applications, two equivalent conditions for the existence of robust H _ FDF are obtained for nominal linear stochastic systems. In particular, under the case of norm uncertainties, a robust H _ FDF is designed based on the feasibility of linear matrix inequalities.

Time-Inconsistent Mean-Field Stochastic LQ Problem: Open-Loop Time-Consistent Control

This paper is concerned with the open-loop time-consistent solution of time-inconsistent mean-field stochastic linear-quadratic (LQ) optimal control. Different from standard stochastic linear-quadratic problems, both the system matrices and the weighting matrices are depending on the initial times, and the conditional expectations of the control and state enter quadratically into the cost functional. Such features will ruin Bellman's principle of optimality and result in the time inconsistency of optimal control. Based on the dynamical nature of the systems involved, a kind of open-loop time-consistent equilibrium control is investigated in this paper. It is shown that the existence of open-loop equilibrium control for a fixed initial pair is equivalent to the solvability of a set of forward–backward stochastic difference equations with stationary condition and convexity condition. By decoupling the forward–backward stochastic difference equations, necessary and sufficient conditions in terms of linear difference equations and generalized difference Riccati equations are given for the existence of open-loop equilibrium control for a fixed initial pair. Moreover, the existence of open-loop time-consistent equilibrium controls for all the initial pairs is shown to be equivalent to the solvability of a set of coupled constrained generalized difference Riccati equations and two sets of constrained linear difference equations.