Stochastic Linear Quadratic Optimal Control Research Articles

Backward Stochastic Riccati Equation with Jumps Associated with Stochastic Linear Quadratic Optimal Control with Jumps and Random Coefficients

In this paper, we investigate the solvability of matrix valued backward stochastic Riccati equations with jumps (BSREJ), which are associated with a stochastic linear quadratic (SLQ) optimal control problem with random coefficients and driven by both Brownian motion and a Poisson process. By the dynamic programming principle, Doob--Meyer decomposition, and inverse flow technique, the existence and uniqueness of the solution for the BSREJ are established. The difficulties addressed in this issue not only are brought from the high nonlinearity of the generator of the BSREJ like the case driven only by Brownian motion, but also from that (i) the inverse flow of the controlled linear stochastic differential equation driven by Poisson process may not exist without additional technical conditions, and (ii) showing that the inverse matrix term involving a jump process in the generator is well-defined. Utilizing the structure of the optimal control problem, we overcome these difficulties and establish the existence of the solution. In addition, we show the construction of the optimal feedback control with the help of the Riccati equation and the relation between the solution of the Riccati equation and the value function of the SLQ problem, which also implies the uniqueness of BSREJ.

Stochastic linear quadratic optimal control problems for mean-field stochastic evolution equations

We study a linear quadratic optimal control problem for mean-field stochastic evolution equation with the assumption that all the coefficients concerned in the problem are deterministic. We show that the existence of optimal feedback operators is equivalent to that of regular solution to the system which is coupled by two Riccati equations and an explicit formula of the optimal feedback control operator is given via the regular solution. We also show that the mentioned Riccati equations admit a unique strongly regular solution when the cost functional is uniformly convex.

Optimal control for controllable stochastic linear systems

This paper is concerned with a constrained stochastic linear-quadratic optimal control problem, in which the terminal state is fixed and the initial state is constrained to lie in a stochastic linear manifold. The controllability of stochastic linear systems is studied. Then the optimal control is explicitly obtained by considering a parameterized unconstrained backward LQ problem and an optimal parameter selection problem. A notable feature of our results is that, instead of solving an equation involving derivatives with respect to the parameter, the optimal parameter is characterized by a matrix equation.

Stochastic linear quadratic optimal control problem for systems driven by fractional Brownian motions

SummaryThis paper is concerned with stochastic linear control systems driven by fractional Brownian motions (fBms) with Hurst parameter H∈(1/2,1) and the cost functional is quadratic with respect to the state and control variables. Here, the integrals with respect to fBms are the type of Stratonovich integrals. A stochastic maximum principle as a necessary condition of the optimal control is derived. The adjoint backward stochastic differential equation (BSDE) is driven by the fBms and its underlying standard Brownian motions. The existence and uniqueness of the solution of adjoint BSDE is proved. The explicit form of the unique optimal control is obtained.

Feedback Stackelberg strategies for the discrete-time mean-field stochastic systems in infinite horizon

This paper deals with the feedback Stackelberg strategies for the discrete-time mean-field stochastic systems in infinite horizon. The optimal control problem of the follower is first studied. Employing the discrete-time linear quadratic (LQ) mean-field stochastic optimal control theory, the sufficient conditions for the solvability of the optimization of the follower are presented and the optimal control is obtained based on the stabilizing solutions of two coupled generalized algebraic Riccati equations (GAREs). Then, the optimization of the leader is transformed into a constrained optimal control problem. Applying the Karush-Kuhn-Tucker (KKT) conditions, the necessary conditions for the existence and uniqueness of the Stackelberg strategies are derived and the Stackelberg strategies are expressed as linear feedback forms involving the state and its mean based on the solutions (Ki,K^i),i=1,2 of a set of cross-coupled stochastic algebraic equations (CSAEs). An iterative algorithm is put forward to calculate efficiently the solutions of the CSAEs. Finally, an example is solved to show the effectiveness of the proposed algorithm.

A discrete-time mean-field stochastic linear-quadratic optimal control problem with financial application

This paper is concerned with a discrete-time mean-field stochastic linear-quadratic optimal control problem arising from financial application. Through matrix dynamical optimisation method, a group of linear feedback controls is investigated. The problem is then reformulated as an operator stochastic linear-quadratic optimal control problem by a sequence of bounded linear operators over Hilbert space, the optimal control with six algebraic Riccati difference equations is obtained by backward induction. The two above approaches are proved to be coincided by the classical method of completing the square. Finally, after discussing the solution of the problem under multidimensional noises, a financial application example is given.

Well-posedness of stochastic Riccati equations and closed-loop solvability for stochastic linear quadratic optimal control problems

We study the closed-loop solvability of a stochastic linear quadratic optimal control problem for systems governed by stochastic evolution equations. This solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. At last, conditions ensuring the uniform convexity of the cost functional are discussed.

The Optimal Control of Fully-Coupled Forward-Backward Doubly Stochastic Systems Driven by Itô-Lévy Processes

This paper studies the optimal control of a fully-coupled forward-backward doubly stochastic system driven by Ito-Levy processes under partial information. The existence and uniqueness of the solution are obtained for a type of fully-coupled forward-backward doubly stochastic differential equations (FBDSDEs in short). As a necessary condition of the optimal control, the authors get the stochastic maximum principle with the control domain being convex and the control variable being contained in all coefficients. The proposed results are applied to solve the forward-backward doubly stochastic linear quadratic optimal control problem.

Linear quadratic stochastic optimal control problems with operator coefficients: open-loop solutions

An optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic (LQ, for short) optimal control problems for mean-field stochastic differential equations. Open-loop solvability of the problem is characterized as the solvability of a system of linear coupled forward-backward stochastic differential equations (FBSDE, for short) with operator coefficients, together with a convexity condition for the cost functional. Under proper conditions, the well-posedness of such an FBSDE, which leads to the existence of an open-loop optimal control, is established. Finally, as applications of our main results, a general mean-field LQ control problem and a concrete mean-variance portfolio selection problem in the open-loop case are solved.

A stochastic maximum principle for linear quadratic problem with nonconvex control domain

This paper considers the stochastic linear quadratic optimal control problem in which the control domain is nonconvex. By the functional analysis and convex perturbation methods, we establish a novel maximum principle. The application of the proposed maximum principle is illustrated through a work-out example.

Weak closed-loop solvability of stochastic linear-quadratic optimal control problems

Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed-loop optimal strategy for LQ problems so that its existence is equivalent to the open-loop solvability of the LQ problem. Moreover, there is at least one open-loop optimal control admitting a state feedback representation. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies.

An Open-Loop Stackelberg Strategy for the Linear Quadratic Mean-Field Stochastic Differential Game

This paper is concerned with the open-loop linear–quadratic (LQ) Stackelberg game of the mean-field stochastic systems in finite horizon. By means of two generalized differential Riccati equations, the follower first solves a mean-field stochastic LQ optimal control problem. Then, the leader turns to solve an optimization problem for a linear mean-field forward–backward stochastic differential equation. By introducing new state and costate variables, we present a sufficient condition for the existence and uniqueness of the Stackelberg strategy in terms of the solvability of some Riccati equations and a convexity condition. Furthermore, it is shown that the open-loop Stackelberg equilibrium admits a feedback representation involving the new state and its mean. Finally, two examples are given to show the effectiveness of the proposed results.

Solution to Mixed H2/H∞ Control for Discrete–time Systems with (x,u,v)–dependent Noise

In this paper, the stochastic H2/H∞ control problem for linear discrete–time systems with (x,u,v)–dependent noise is studied. By applying the leader–follower stochastic game approach, the disturbance is treated as the follower and the control input is treated as the leader, respectively. Necessary and sufficient conditions for the mixed control problem are presented which guarantee the existence and uniqueness of the optimal solution. By applying the stochastic maximum principle, the follower first solves a stochastic linear quadratic optimal control problem which is given in the form of H∞–norm with the aid of stochastic Riccati equations. Then the leader solves a stochastic linear quadratic problem with the aid of forward and backward equations. The main technique is to introduce two new co–states to capture the future information, the encountered difficulty is to establish a homogeneous relationship between the new co–states.

Solving Stochastic LQR Problems by Polynomial Chaos

We consider the infinite dimensional stochastic linear quadratic optimal control problem for the infinite horizon case. We provide a numerical framework for solving this problem using a polynomial chaos expansion approach. By applying the method of chaos expansions to the state equation, we obtain a system of deterministic partial differential equations in terms of the coefficients of the state and the control variables. We set up a control problem for each equation, which results in a set of infinite horizon deterministic linear quadratic regulator problems. We prove the optimality of the solution expressed in terms of the expansion of these coefficients compared to the direct approach. We perform numerical experiments which validate our approach and compare the finite and infinite horizon case.

Necessary conditions in stochastic linear quadratic problems and their applications

Stochastic linear quadratic optimal control problems are considered. A unified approach is proposed to treat the necessary optimality conditions of closed-loop optimal strategies and open-loop optimal controls. Notice that the former notion does not rely on initial wealth, while the later one does. Our conclusions of closed-loop optimal strategies are directly derived by suitable variational methods, the approach to which is different from [12], [11]. Moreover, the necessary conditions for closed-loop optimal strategies happen to be sufficient which takes us by surprise. Finally, two applications are given as illustration.

Time-Inconsistent Mean-Field Stochastic LQ Problem: Open-Loop Time-Consistent Control

This paper is concerned with the open-loop time-consistent solution of time-inconsistent mean-field stochastic linear-quadratic (LQ) optimal control. Different from standard stochastic linear-quadratic problems, both the system matrices and the weighting matrices are depending on the initial times, and the conditional expectations of the control and state enter quadratically into the cost functional. Such features will ruin Bellman's principle of optimality and result in the time inconsistency of optimal control. Based on the dynamical nature of the systems involved, a kind of open-loop time-consistent equilibrium control is investigated in this paper. It is shown that the existence of open-loop equilibrium control for a fixed initial pair is equivalent to the solvability of a set of forward–backward stochastic difference equations with stationary condition and convexity condition. By decoupling the forward–backward stochastic difference equations, necessary and sufficient conditions in terms of linear difference equations and generalized difference Riccati equations are given for the existence of open-loop equilibrium control for a fixed initial pair. Moreover, the existence of open-loop time-consistent equilibrium controls for all the initial pairs is shown to be equivalent to the solvability of a set of coupled constrained generalized difference Riccati equations and two sets of constrained linear difference equations.

Constrained LQ Problem with a Random Jump and Application to Portfolio Selection

This paper deals with a constrained stochastic linear-quadratic (LQ for short) optimal control problem where the control is constrained in a closed cone. The state process is governed by a controlled SDE with random coefficients. Moreover, there is a random jump of the state process. In mathematical finance, the random jump often represents the default of a counter party. Thanks to the Ito-Tanaka formula, optimal control and optimal value can be obtained by solutions of a system of backward stochastic differential equations (BSDEs for short). The solvability of the BSDEs is obtained by solving a recursive system of BSDEs driven by the Brownian motions. The author also applies the result to the mean variance portfolio selection problem in which the stock price can be affected by the default of a counterparty.

Optimal control of mean-field backward doubly stochastic systems driven by Itô-Lévy processes

ABSTRACT In this paper, we introduce a new class of backward doubly stochastic differential equations (in short BDSDE) called mean-field backward doubly stochastic differential equations (in short MFBDSDE) driven by Itô-Lévy processes and study the partial information optimal control problems for backward doubly stochastic systems driven by Itô-Lévy processes of mean-field type, in which the coefficients depend on not only the solution processes but also their expected values. First, using the method of contraction mapping, we prove the existence and uniqueness of the solutions to this kind of MFBDSDE. Then, by the method of convex variation and duality technique, we establish a sufficient and necessary stochastic maximum principle for the stochastic system. Finally, we illustrate our theoretical results by an application to a stochastic linear quadratic optimal control problem of a mean-field backward doubly stochastic system driven by Itô-Lévy processes.

Equilibrium Controls in Time Inconsistent Stochastic Linear Quadratic Problems

This paper deals with a class of time inconsistent stochastic linear quadratic optimal control problems in Markovian framework. Three notions, i.e., closed-loop equilibrium strategies, open-loop equilibrium controls and open-loop equilibrium strategies, are characterized in unified manners. These results indicate clearer and deeper distinctions among these notions. For example, in particular time consistent setting, the open-loop equilibrium controls are fully characterized by first-order, second-ordernecessaryoptimalityconditions, and are not optimal in general, while the closed-loop equilibrium controls naturally reduce into closed-loopoptimalcontrols.

Singularly Perturbed Forward-Backward Stochastic Differential Equations: Application to the Optimal Control of Bilinear Systems

We study linear-quadratic stochastic optimal control problems with bilinear state dependence where the underlying stochastic differential equation (SDE) has multiscale features. We show that, in the same way in which the underlying dynamics can be well approximated by a reduced-order dynamics in the scale separation limit (using classical homogenization results), the associated optimal expected cost converges to an effective optimal cost in the scale separation limit. This entails that we can approximate the stochastic optimal control for the whole system by a reduced-order stochastic optimal control, which is easier to compute because of the lower dimensionality of the problem. The approach uses an equivalent formulation of the Hamilton-Jacobi-Bellman (HJB) equation, in terms of forward-backward SDEs (FBSDEs). We exploit the efficient solvability of FBSDEs via a least squares Monte Carlo algorithm and show its applicability by a suitable numerical example.