Indefinite Linear Quadratic Research Articles

Indefinite LQ optimal control for stochastic Takagi–Sugeno fuzzy system under sensor data scheduling: Finite‐horizon case

AbstractThe present paper considers the finite‐horizon indefinite linear quadratic (LQ) control problem for stochastic Takagi–Sugeno (T‐S) fuzzy systems with input delay. In this paper, we consider the presence of sensor data scheduling, which imposes a communication energy constraint and necessitates optimal state estimation for measurements. Then, by utilizing dynamic programming principles, the stochastic LQ problem under consideration can be solved, while the optimal control policy is developed in terms of the unique solutions to a set of coupled difference Riccati equations (CDREs). Specifically, for simple delay‐free case, the linear matrix inequalities based conditions are also proposed, whose feasibility is shown to be equivalent to the well‐posedness of the indefinite LQ control under consideration. As an application, our theoretic analysis is extended to study the intermittent observation model caused by random denial‐of‐service attack.

On the stochastic linear quadratic optimal control problem by piecewise constant controls: The infinite horizon time case

This paper is devoted to the problem of indefinite stochastic linear quadratic (LQ) optimal control by piecewise constant controls in an infinite horizon case. By restricting the set of admissible controls to the class of piecewise constant stochastic processes, we reformulated the above control problem under the setting of systems modeled by Itô differential equations controlled by impulses. We show that the solution in a state feedback form of the indefinite stochastic LQ control problem is equivalent to the existence of a global stabilizing solution associated to a class of backward matrix linear differential equations with a Riccati type jumping operators.

Linear–quadratic stochastic leader–follower differential games for Markov jump-diffusion models

This paper considers the linear–quadratic (LQ) leader–follower Stackelberg differential game for Markov jump-diffusion stochastic differential equations (SDEs). We first obtain the open-loop type optimal solutions for the leader and the follower by establishing the general stochastic maximum principle for (indefinite) LQ control with Markovian jumps. Then we obtain the state-feedback representation of the open-loop type optimal solutions for the leader and the follower in terms of the coupled Riccati differential equations (CRDEs) by generalizing the classical Four-Step Scheme to the case with Markovian jumps. Unlike the existing literature, the Four-Step Scheme in our paper is not symmetric due to the presence of the nonsymmetric quadratic variation induced by the Markov chain. We develop the decoupling approach to find the simplified expression of the corresponding quadratic variation. Under the well-posedness of the CRDEs, the state-feedback type optimal solutions for the leader and the follower constitute the Stackelberg equilibrium. We also show the well-posedness of the CRDEs for the follower under suitable assumptions of the coefficients. Finally, we demonstrate the well-posedness of the CRDEs for the leader and the follower via numerical simulations for both indefinite and definite cost parameter cases.

Indefinite linear quadratic optimal control for discrete time‐varying linear rectangular descriptor systems

AbstractIn this article, the indefinite linear quadratic (LQ) optimal control problem for discrete time‐varying rectangular descriptor systems is investigated, in which the indefinite weight matrices are considered in the quadratic cost function. With a suite of equivalent transformations, the indefinite LQ problem for discrete time‐varying rectangular descriptor systems can be substituted by an equivalent indefinite LQ problem for standard state space systems. Then, the transformed equivalent indefinite LQ problem is solved by establishing a dual Krein space model, and the optimal control and optimal cost value are obtained by invoking the Kalman filter of Krein space. The form of state feedback optimal control corresponding to the original system is given. Next, to guarantee the dynamic part for the resulting optimal closed‐loop system has a unique solution, some necessary and sufficient conditions are proposed. Finally, a numerical example is carried out to demonstrate the availability of established results.

Indefinite linear quadratic optimal control for continuous-time rectangular descriptor Markov jump systems: infinite-time case

The indefinite linear quadratic (ILQ) optimal control problem has many important applications in financial, economic systems, etc., which has been widely researched in stochastic systems and descriptor systems, etc. However, for ILQ problem of rectangular descriptor Markov jump systems (DMJSs), because there are impulses simultaneously in descriptor subsystems and at the switching time, it is theoretically difficult to research and there is no result yet. This paper discusses the ILQ optimal control problem for continuous-time linear rectangular DMJSs. Firstly, under some rank conditions and inequality conditions, the ILQ problem for rectangular DMJSs can be equivalently transformed into standard LQ problem for Markov jump systems (MJSs) by using elementary linear algebra method. Then based on the LQ theory of MJSs, the solvable sufficient condition of the ILQ problem for rectangular DMJSs and the non-negative optimal cost value are obtained. The optimal control can be synthesised as state feedback, and the resulting optimal closed-loop system has the stochastically stable solution. In addition, with some rank inequality assumptions, the differential subsystem of the resulting optimal closed-loop system can be ensured to have a unique solution. Finally, two numerical examples are provided to illustrate the effectiveness of the methods proposed in this paper.

Finite and infinite horizon indefinite linear quadratic optimal control for discrete-time singular Markov jump systems

This paper concerns the indefinite linear quadratic (LQ) optimal control problem for discrete-time singular Markov jump systems (MJSs) with finite and infinite horizon, where the weight matrices for state and control of cost function are all indefinite. Firstly, the indefinite LQ problem for singular MJSs is equivalently transformed into indefinite LQ problem for MJSs under a series of equivalent transformations. Then, the sufficient and necessary condition is proposed for the solvability of finite horizon case, the optimal control and optimal cost value are given, and the resulting optimal closed-loop system is regular, casual. Next, some sufficient and necessary conditions are obtained to ensure the transformed equivalent LQ problem for MJSs to be definite one, which can guarantee the generalized algebraic Riccati equation with Markov jump has a unique semi-positive definite solution. Meanwhile, the optimal control and nonnegative optimal cost value in infinite horizon are acquired, and the resulting optimal closed-loop system is stochastically admissible. Finally, three examples are presented to illustrate the theoretical results.

Linear-Quadratic Stochastic Stackelberg Differential Games for Jump-Diffusion Systems

This paper considers linear-quadratic (LQ) stochastic leader-follower Stackelberg differential games for jump-diffusion systems with random coefficients. We first solve the LQ problem of the follower using the stochastic maximum principle and obtain the state-feedback representation of the open-loop optimal solution in terms of the integro-stochastic Riccati differential equation (ISRDE), where the state-feedback type control is shown to be optimal via the completion of squares method. Next, we establish the stochastic maximum principle for the indefinite LQ stochastic optimal control problem of the leader using the variational method. However, to obtain the state-feedback representation of the open-loop solution for the leader, there is a technical challenge due to the jump process. To overcome this limitation, we consider two different cases, in which the state-feedback type optimal control for the leader in terms of the ISRDE can be characterized by generalizing the Four-Step Scheme. Finally, in these two cases, we show that the state-feedback representation of the open-loop optimal solutions for the leader and the follower constitutes the Stackelberg equilibrium. Note that the (indefinite) LQ control problem of the leader is new and nontrivial due to the coupled FBSDE constraint induced by the rational behavior of the follower.

Infinite horizon LQ Nash Games for SDEs with infinite jumps

AbstractIn this paper, we consider infinite horizon linear‐quadratic (LQ) Nash games for stochastic differential equations (SDEs) with infinite Markovian jumps and (x,u,v)‐dependent noise. An indefinite stochastic LQ result is first derived for the considered system. Then, under the condition of strong detectability, a necessary and sufficient condition for the existence of a Nash equilibrium is put forward in terms of the solvability of a countably infinite set of coupled generalized algebraic Riccati equations (ICGAREs). Moreover, the mixed H2/H∞ control is investigated by Nash game approach as an important application. At last, we present an iterative algorithm to solve the ICGAREs, and a numerical simulation is given to illustrate its efficiency.

Indefinite LQ optimal control for discrete-time uncertain systems

This paper is concerned with a linear quadratic (LQ) optimal control for discrete-time uncertain systems, with indefinite state and control weighting matrices in the cost function. Firstly, a recurrence equation of general optimal control problem for discrete-time uncertain systems is obtained by applying Bellman’s principle of optimality. Then, the optimal state feedback control is obtained based on the recurrence equation. Moreover, a sufficient condition of well-posedness for the LQ problem is proposed and a general expression for the optimal control set is given. Furthermore, a numerical example is presented by using the obtained results. Finally, as an application of the indefinite LQ optimal control, an optimal production inventory problem of uncertain environment is solved.

Indefinite LQ optimal control with cross term for discrete‐time uncertain systems

Recently, there has been an increasing interest in the study on uncertain optimal control problems. In this paper, a linear quadratic (LQ) optimal control with cross term for discrete‐time uncertain systems is considered, whereas the weighting matrices in the cost function are allowed to be indefinite. Firstly, a recurrence equation for the problem is presented based on Bellman's principle of optimality in dynamic programming. Then, a necessary condition for the existence of an optimal linear state feedback control of the indefinite LQ problem is given by the recurrence equation. Moreover, a sufficient condition of well‐posedness for the indefinite LQ problem is presented by introducing a linear matrix inequality (LMI) condition. Furthermore, it is shown that the well‐posedness of the indefinite LQ problem, the solvability of the indefinite LQ problem, the LMI condition, and the solvability of the constrained difference equation are equivalent to each other. Finally, an example is presented to illustrate the results obtained.

Indefinite LQ optimal control with process state inequality constraints for discrete-time uncertain systems

Uncertainty theory is a branch of axiomatic mathematics that deals with human uncertainty. Based on uncertainty theory, this paper discusses linear quadratic (LQ) optimal control with process state inequality constraints for discrete-time uncertain systems, where the weighting matrices in the cost function are assumed to be indefinite. By means of the maximum principle with mixed inequality constraints, we present a necessary condition for the existence of optimal state feedback control that involves a constrained difference equation. Moreover, the existence of a solution to the constrained difference equation is equivalent to the solvability of the indefinite LQ problem. Furthermore, the well-posedness of the indefinite LQ problem is proved. Finally, an example is provided to demonstrate the effectiveness of our theoretical results.

Optimistic Value Model of Indefinite LQ Optimal Control for Discrete‐Time Uncertain Systems

AbstractUncertainty theory is a branch of mathematics which provides a new tool to deal with the human uncertainty. Based on uncertainty theory, this paper proposes an optimistic value model of discrete‐time linear quadratic (LQ) optimal control, whereas the state and control weighting matrices in the cost function are indefinite, the system dynamics are disturbed by uncertain noises. With the aid of the Bellman's principle of optimality in dynamic programming, we first present a recurrence equation. Then, a necessary condition for the state feedback control of the indefinite LQ problem is derived by using the recurrence equation. Moreover, a sufficient condition of well‐posedness for the indefinite LQ optimal control is given. Finally, a numerical example is presented by using the obtained results.

Partial stabilizability and hidden convexity of indefinite LQ problem

Generalization of linear system stability theory and LQ control theory are presented. It is shown that the partial stabilizability problem is equivalent to a Linear Matrix Inequality (LMI). Also, the set of all initial conditions for which the system is stabilizable by an open-loop control (the stabilizability subspace) is characterized in terms of a semi-definite programming (SDP). Next, we give a complete theory for an infinite-time horizon Linear Quadratic (LQ) problem with possibly indefinite weighting matrices for the state and control. Necessary and sufficient convex conditions are given for well-posedness as well as attainability of the proposed (LQ) problem. There is no prior assumption of complete stabilizability condition as well as no assumption on the quadratic cost. A generalized algebraic Riccati equation is introduced and it is shown that it provides all possible optimal controls. Moreover, we show that the solvability of the proposed indefinite LQ problem is equivalent to the solvability of a specific SDP problem.

Indefinite LQ optimal control with equality constraint for discrete-time uncertain systems

Based on uncertainty theory, this paper studies a kind of discrete-time uncertain linear quadratic (LQ) optimal control with equality constraint for the terminal state, allowing the state and control weighting matrices in the cost function to be indefinite. First, we transform the uncertain LQ optimal control problem into an equivalent deterministic optimal control problem. Then, a necessary condition for the existence of optimal linear state feedback control is presented by means of matrix minimum principle. Moreover, the well-posedness of the uncertain LQ problem is proved by applying the technique of completing squares. Finally, an example is provided to demonstrate the effectiveness of our theoretical results.

Linear Quadratic Optimal Control of Continuous-Time LTI Systems With Random Input Gains

This note studies the linear quadratic (LQ) optimal control of continuous-time linear time-invariant systems with random gains imposed on the input channels. We start from the indefinite LQ problem, in which the cost weighting matrix can be indefinite. The definite LQ problem is discussed as a special case. The main novelty originates from the point of view that in networked control, designing the channels and controller jointly often leads to an easier problem and achieves better performance than designing them separately. Specifically, we formulate the LQ problem as a channel/controller co-design problem assuming that the channel capacities can be allocated among the input channels subject to an overall capacity constraint. Necessary and sufficient conditions are obtained for the well-posedness and the attainability of the indefinite LQ problem under a given channel capacity allocation satisfying the stabilization requirement. The optimal controller is given by a linear state feedback associated with the mean-square stabilizing solution of a modified algebraic Riccati equation.

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.

Solvability of indefinite stochastic Riccati equations and linear quadratic optimal control problems

A new approach to study the indefinite stochastic linear quadratic (LQ) optimal control problems, which we called the “equivalent cost functional method”, is introduced by Yu (2013) in the setup of Hamiltonian system. On the other hand, another important issue along this research direction, is the possible state feedback representation of optimal control and the solvability of associated indefinite stochastic Riccati equations. As the response, this paper continues to develop the equivalent cost functional method by extending it to the Riccati equation setup. Our analysis is featured by its introduction of some equivalent cost functionals which enable us to have the bridge between the indefinite and positive-definite stochastic LQ problems. With such bridge, some solvability relation between the indefinite and positive-definite Riccati equations is further characterized. It is remarkable the solvability of the former is rather complicated than the latter, hence our relation provides some alternative but useful viewpoint. Consequently, the corresponding indefinite linear quadratic problem is discussed for which the unique optimal control is derived in terms of state feedback via the solution of the Riccati equation. In addition, some example is studied using our theoretical results.

Study on Indefinite Stochastic Linear Quadratic Optimal Control with Inequality Constraint

This paper studies the indefinite stochastic linear quadratic (LQ) optimal control problem with an inequality constraint for the terminal state. Firstly, we prove a generalized Karush-Kuhn-Tucker (KKT) theorem under hybrid constraints. Secondly, a new type of generalized Riccati equations is obtained, based on which a necessary condition (it is also a sufficient condition under stronger assumptions) for the existence of an optimal linear state feedback control is given by means of KKT theorem. Finally, we design a dynamic programming algorithm to solve the constrained indefinite stochastic LQ issue.

Indefinite linear quadratic optimal control problem for singular discrete-time system with multiple input delays

An indefinite linear quadratic (ILQ) optimal control problem is discussed for singular discrete-time-varying linear systems with multiple input delays. The problem is transformed to the one for standard systems by normalizability decomposition. An explicit controller is obtained by computing the gain of the smoothing estimation of dual systems. Necessary and sufficient conditions guaranteeing the existence of unique solution are given simultaneously. A numerical example illustrates the presented method.

Indefinite Linear Quadratic Optimal Control Problem for Singular Linear Discrete-time System: Krein Space Method

The finite time horizon indefinite linear quadratic(LQ) optimal control problem for singular linear discrete time-varying systems is discussed. Indefinite LQ optimal control problem for singular systems can be transformed to that for standard state-space systems under a reasonable assumption. It is shown that the indefinite LQ optimal control problem is dual to that of projection for backward stochastic systems. Thus, the optimal LQ controller can be obtained by computing the gain matrices of Kalman filter. Necessary and sufficient conditions guaranteeing a unique solution for the indefinite LQ problem are given. An explicit solution for the problem is obtained in terms of the solution of Riccati difference equations.