Coupled Riccati Differential Equations Research Articles

Infinite Horizon H2/H∞ Control for Discrete-Time Mean-Field Stochastic Systems

In this paper, we deal with the H2/H∞ control problem in the infinite horizon for discrete-time mean-field stochastic systems with (x,u,v)-dependent noise. First of all, a stochastic-bounded real lemma, which is the core of H∞ analysis, is derived. Secondly, a sufficient condition in terms of the solution of coupled difference Riccati equations (CDREs) is obtained for solving the H2/H∞ control problem above. In addition, an iterative algorithm for solving CDREs is proposed and a numerical example is given for verification of the feasibility of the developed results.

Indefinite LQ optimal control for stochastic Takagi–Sugeno fuzzy system under sensor data scheduling: Finite‐horizon case

AbstractThe present paper considers the finite‐horizon indefinite linear quadratic (LQ) control problem for stochastic Takagi–Sugeno (T‐S) fuzzy systems with input delay. In this paper, we consider the presence of sensor data scheduling, which imposes a communication energy constraint and necessitates optimal state estimation for measurements. Then, by utilizing dynamic programming principles, the stochastic LQ problem under consideration can be solved, while the optimal control policy is developed in terms of the unique solutions to a set of coupled difference Riccati equations (CDREs). Specifically, for simple delay‐free case, the linear matrix inequalities based conditions are also proposed, whose feasibility is shown to be equivalent to the well‐posedness of the indefinite LQ control under consideration. As an application, our theoretic analysis is extended to study the intermittent observation model caused by random denial‐of‐service attack.

Linear–quadratic stochastic leader–follower differential games for Markov jump-diffusion models

This paper considers the linear–quadratic (LQ) leader–follower Stackelberg differential game for Markov jump-diffusion stochastic differential equations (SDEs). We first obtain the open-loop type optimal solutions for the leader and the follower by establishing the general stochastic maximum principle for (indefinite) LQ control with Markovian jumps. Then we obtain the state-feedback representation of the open-loop type optimal solutions for the leader and the follower in terms of the coupled Riccati differential equations (CRDEs) by generalizing the classical Four-Step Scheme to the case with Markovian jumps. Unlike the existing literature, the Four-Step Scheme in our paper is not symmetric due to the presence of the nonsymmetric quadratic variation induced by the Markov chain. We develop the decoupling approach to find the simplified expression of the corresponding quadratic variation. Under the well-posedness of the CRDEs, the state-feedback type optimal solutions for the leader and the follower constitute the Stackelberg equilibrium. We also show the well-posedness of the CRDEs for the follower under suitable assumptions of the coefficients. Finally, we demonstrate the well-posedness of the CRDEs for the leader and the follower via numerical simulations for both indefinite and definite cost parameter cases.

Control for networked control systems with multiplicative noises, packet dropouts and multiple delays

This paper mainly focuses on the optimal output feedback control problem for networked control systems (NCSs) involving multiplicative noises, packet dropouts, input delays and measurement delays. The main contributions of this paper can be concluded as follows. Firstly, different from the previous results, this paper overcomes the barrier of the packet dropouts and measurement delays. Based on the measurement data, the optimal estimator is given. Secondly, by using maximum principle, a sufficient and necessary condition for the optimal control problem is presented. Moreover, the explicit output feedback controller is derived with feedback gain based on the coupled Riccati difference equations. Numerical example is illustrated to show the effectiveness of the proposed algorithm.

Optimal output feedback control for discrete‐time Markov jump linear system with input delay and packet losses

SummaryThis article investigates the optimal output feedback control problem for discrete‐time Markov jump linear system (MJLS) with input delay and packet losses in finite horizon. There are three main contributions. First, we assume that the state variable and the jump variable are available to the controller, and then we propose a new version of stochastic maximum principle. Based on the new proposed tool, an optimal state feedback controller is obtained by solving a set of delayed coupled difference Riccati equations. Second, under the condition that the state variable is unavailable to the controller, it is desired to design a dynamic Markovian jump controller such that the closed‐loop system minimizes the performance index. Thus, an optimal Markovian jump linear filter is proposed for MJLS with input delay and packet losses. Finally, based on the optimal filter, we prove that the separation principle holds for this case and then an optimal output feedback controller is presented. Some numerical examples are presented to illustrate the proposed approach.

Linear–quadratic mean field stochastic zero-sum differential games

In this paper, we consider the two-player linear–quadratic mean field stochastic differential game (LQ-MF-SZSDG), where the expected values of state and players’ control variables are included in the corresponding mean field stochastic system and the objective functional. We characterize the explicit condition under which the feedback saddle-point equilibrium for the LQ-MF-SZSDG exists. The results are obtained by using the “completion of squares” method and the “ordered interchangeability” property of multiple saddle points. We show that the corresponding saddle-point equilibrium is linear in state, which is characterized by the two coupled Riccati differential equations (CRDEs). We also discuss the solvability conditions of the CRDEs. Numerical results are presented to verify the theoretical results of the paper.

Linear quadratic differential game approach for attitude takeover control of failed spacecraft

Spacecraft is mostly abandoned after its fuel exhausts or actuator fails. Using microsatellites to implement attitude takeover control for failed spacecraft is a cost efficient way to recycle on-board valuable payloads at a low cost. In order to achieve collaboration between independent microsatellites, a differential game based method is proposed to stabilize the combined spacecraft composed of multiple microsatellites and the failed spacecraft, in which each microsatellite can independently calculate its own control strategy. Firstly, based on the attitude model of combined spacecraft and the performance index function of each microsatellite, the linear quadratic differential game model is formulated. By optimizing individual performance index functions, the control strategies of multiple microsatellites can be obtained. Secondly, in order to find the Nash equilibrium strategies of microsatellites, the coupled Hamilton-Jacobi(HJ) equations are transferred to a set of coupled Riccati differential equations over finite time horizon. In order to avoid computational overhead, a set of coupled algebraic Riccati equations over infinite horizon is also derived, which can be solved easily with high precision by the Lyapunov iterations method. Finally, numerical simulations verify the effectiveness of the differential game method for the attitude takeover control of failed spacecraft.

Linear-Quadratic Time-Inconsistent Mean-Field Type Stackelberg Differential Games: Time-Consistent Open-Loop Solutions

In this article, we consider the linear-quadratic time-inconsistent mean-field type leader-follower Stackelberg differential game with an adapted open-loop information structure. The objective functionals of the leader and the follower include conditional expectations of state and control (mean field) variables, and the cost parameters could be general nonexponential discounting depending on the initial time. As stated in the existing literature, these two general settings of the objective functionals induce time inconsistency in the optimal solutions. Given an arbitrary control of the leader, we first obtain the follower's (time consistent) equilibrium control and its state feedback representation in terms of the nonsymmetric coupled Riccati differential equations (RDEs) and the backward stochastic differential equation (SDE). This provides the rational behavior of the follower, characterized by the forward-backward SDE (FBSDE). We then obtain the leader's explicit (time consistent) equilibrium control and its state feedback representation in terms of the nonsymmetric coupled RDEs under the FBSDE constraint induced by the follower. With the solvability of the nonsymmetric coupled RDEs, the equilibrium controls of the leader and the follower constitute the time-consistent Stackelberg equilibrium. Finally, the numerical examples are provided to check the solvability of the nonsymmetric coupled RDEs.

A Sufficient Condition for Linear-Quadratic Stochastic Zero-Sum Differential Games for Markov Jump Systems

In this note, we consider the linear-quadratic stochastic zero-sum differential game (LQ-SZSDG) for the Markov jump system (MJS) driven by Brownian motion. Unlike previous work considered in the literature, the diffusion term of the MJS is dependent on the state and the control of both players, and the cost parameters need not be definite matrices. We obtain a sufficient condition under which a feedback saddle point for the LQ-SZSDG exists. We show that the corresponding feedback saddle point is linear in the state and can be characterized in terms of a set of coupled Riccati differential equations (CRDEs). We also discuss the solvability of the CRDEs and verify the solvability through numerical examples under the existence condition of the saddle point.

Optimal filter for MJL system with delayed modes and observations

This study investigates the optimal filter for Markov jump linear (MJL) system with multiple delayed modes and observations, in which the data losses are introduced naturally and the Markov chains are assumed to be known up to the present time. To deal with the mode and observation delays, a reorganised observation sequence is defined. Moreover, the optimal filter problem is transformed into one of the delay-free system which is just with jumping parameters and multiplicative noises. An optimal MJL filter is presented based on the mean squared method, where the filter gains are determined by solving a set of generalised coupled Riccati difference equations based on a set of coupled Lyapunov equations. Alternatively, an optimal stationary MJL filter is developed, where the filter gains are derived by solving a set of generalised coupled algebraic Riccati equations based on coupled algebraic Lyapunov equations. It can be shown that the difference Riccati equations developed in the former filter converge to the stationary ones under appropriate conditions.

Finite Time Robust Feedback Nash Equilibrium for Linear Quadratic Games

In this paper we consider the solution for an N players non-cooperative differential game affected by some sort of uncertainties. The problem analyzed is linear quadratic in nature, and the uncertainty affecting the game is square integrable, which is seen as a malicious fictitious player trying to maximize the cost function of each player. In order to find the solution to this problem we solve a robust form of the Hamilton-Jacobi-Bellman equation, which allows us to find the robust equilibrium strategies for each player and in turn to solve a Coupled Riccati Differential Equation.

Risk-sensitive control of Markov jump linear systems: Caveats and difficulties

In this technical note, we revisit the risk-sensitive optimal control problem for Markov jump linear systems (MJLSs). We first demonstrate the inherent difficulty in solving the risk-sensitive optimal control problem even if the system is linear and the cost function is quadratic. This is due to the nonlinear nature of the coupled set of Hamilton-Jacobi-Bellman (HJB) equations, stemming from the presence of the jump process. It thus follows that the standard quadratic form of the value function with a set of coupled Riccati differential equations cannot be a candidate solution to the coupled HJB equations. We subsequently show that there is no equivalence relationship between the problems of risk-sensitive control and H∞ control of MJLSs, which are shown to be equivalent in the absence of any jumps. Finally, we show that there does not exist a large deviation limit as well as a risk-neutral limit of the risk-sensitive optimal control problem due to the presence of a nonlinear coupling term in the HJB equations.

Robust H∞ Filtering and Deconvolution for Continuous Time Delay Systems Based on Game‐Theoretic Approach

AbstractMany dynamical systems involve not only process and measurement noise signals but also parameter uncertainty and unknown input signals. This paper aims to estimate the state and unknown input for linear continuous time‐varying systems subject to time delay in state, norm‐bounded parameter uncertainty, and a known input. Such a problem is reformulated into a two‐player differential game whose saddle point solution gives rise to one sufficient solvable condition for the estimation problem. The possible optimal estimators are obtained by solving the two coupled Riccati differential equations. We demonstrate, through two examples, how the proposed estimator is valid for estimating state and unknown input.

Stochastic Optimal Tracking with Preview for Linear Continuous-Time Markovian Jump Systems by Output Feedback

The author has already presented stochastic optimal tracking control theory with preview for linear continuous-time Markovian jump systems on the finite time interval by state feedback ([21]). He has also presented optimal estimation theory for linear continuoustime Markovian jump systems on the finite time interval ([22]). In this paper we study stochastic optimal tracking control theory with preview for linear continuous-time Markovian jump systems on the finite time interval by output feedback. The necessary and sufficient conditions for the solvability of the stochastic optimal tracking problem with preview by state feedback are given by coupled Riccati differential equations with terminal conditions. In order to decide the gains of output feedback controllers, we need solutions of another type of coupled Riccati differential equations with initial conditions.

Hybrid optimal filtering for linear continuous-time Markovian jump systems with non-Gaussian noises by MPT approach

In this paper, we study the optimal filtering problems for a class of linear continuoustime Markovian jump systems with non-Gaussian noises by most probable trajectory (MPT) approach based on principle of hybrid optimality. The systems are described by the switching systems with Markovian mode transitions. In this paper we consider both cases (A) the modes are accessible and (B) the modes are inaccessible. The necessary and sufficient conditions for the solvability of the optimal filtering problems are given by coupled Riccati differential equations with initial conditions. Finally, we give numerical examples.

Game-Theoretic Design for Robust $H_{\infty}$ Filtering and Deconvolution With Consideration of Known Input

This paper aims at the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> estimation of a linear combination of state and unknown input for linear continuous time-varying systems, which is subjected to both norm-bounded parameter uncertainty and a known input. Such a problem is reformulated into a two-player differential game, whose saddle point solution gives rise to both one sufficient solvable condition to the estimation problem and one possible optimal estimator in terms of solution to two coupled Riccati differential equations. It is demonstrated, through one example, the proposed estimator is of superior robust performance with respect to either parameter uncertainty or known input, when compared with those based on the nominal design. Note to Practitioners-Generally speaking, estimation can be divided into two kinds of problems including state observation (filtering, smoothing, and prediction) and deconvolution (unknown input estimation). This paper will consider state and unknown input hybrid estimation (abbreviated to "hybrid estimation"). The estimated signal is linear combination of state and unknown input. Hybrid estimation also reveals useful in many industrial applications. One example is load current estimation of unin terruptible power supply, where load current signal is a linear function of capacity voltage (state) and back electromotive force (unknown input). Chaos synchronization in secure communication is another example. The practical systems are subjected to both norm-bounded parameter uncertainty and a known input, and the known input will incur new estimation error due to parameter uncertainty. In this work, we utilize differential game approach to solve hybrid estimation problem. The method is based on optimal estimator in terms of solution to two coupled Riccati differential equations. Based on these tools, we propose a complete solvable framework for hybrid estimation systems.

H&lt;sub&gt;∞&lt;/sub&gt; Tracking with Preview for Linear Continuous-Time Markovian Jump Systems by Output Feedback

In this paper we study the H∞ tracking problems with preview for a class of linear continuous-time Markovian jump systems. The systems are described by the linear continuous-time switching systems with Markovian mode transition. The necessary and sufficient conditions for the solvability of the H∞ tracking problem are given by coupled Riccati differential equations with terminal conditions. Correspondingly feedforward compensators introducing future information are given by coupled differential equations with terminal conditions. In this paper we consider the case that the measured output is only partially observed and present the design theory by output feedback.

H&lt;sub&gt;∞&lt;/sub&gt; Estimation for Linear Continuous-Time Markovian Jump Systems by Game Theoretic Approach

In this paper we study the H∞ state estimation problems for a class of linear continuous-time Markovian jump systems. We adopt a game theoretic approach and stochastic variational calculus method to derive estimates and forms of dynamic estimators on the fixed time interval. The necessary and sufficient conditions for the solvability of the H∞ state estimation problems are given by the coupled Riccati differential equations with initial conditions.

離散時間線形マルコフジャンプシステムに対する状態フィードバックによる予見Ｈ∞追従制御

In order to design tracking control systems for a class of systems with rapid or abrupt changes, it is effective in improving tracking performance to construct preview control systems considering future information of reference signals. In this paper we study the H∞ tracking problems with preview for a class of linear discrete-time Markovian jump systems by game theoretic approach. The systems are described by a class of switching systems with Markovian mode transitions. The necessary and sufficient conditions for the solvability of the H∞ tracking problems are given by coupled Riccati difference equations with terminal conditions. Correspondingly feedforward compensators introducing future information are given by coupled difference equations with terminal conditions. We consider three different tracking problems depending on the property of the reference signals. Finally we give numerical examples and verify the effectiveness of the theory presented in this paper.

A Separation Principle for the Continuous-Time LQ-Problem With Markovian Jump Parameters

In this paper, we devise a separation principle for the finite horizon quadratic optimal control problem of continuous-time Markovian jump linear systems driven by a Wiener process and with partial observations. We assume that the output variable and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed loop system minimizes the quadratic functional cost of the system over a finite horizon period of time. As in the case with no jumps, we show that an optimal controller can be obtained from two coupled Riccati differential equations, one associated to the optimal control problem when the state variable is available, and the other one associated to the optimal filtering problem. This is a separation principle for the finite horizon quadratic optimal control problem for continuous-time Markovian jump linear systems. For the case in which the matrices are all time-invariant we analyze the asymptotic behavior of the solution of the derived interconnected Riccati differential equations to the solution of the associated set of coupled algebraic Riccati equations as well as the mean square stabilizing property of this limiting solution. When there is only one mode of operation our results coincide with the traditional ones for the LQG control of continuous-time linear systems.