Stochastic Linear-quadratic Optimal Control Problems Research Articles

Decentralized stochastic linear-quadratic optimal control with risk constraint and partial observation

This paper addresses a risk-constrained decentralized stochastic linear-quadratic optimal control problem with one remote controller and one local controller, where the risk constraint is posed on the cumulative state weighted variance in order to reduce the oscillation of system trajectory. In this model, local controller can only partially observe the system’s state, and sends the estimate of state to remote controller through an unreliable channel, whereas the channel from remote controller to local controller is perfect; here, the unreliable channel means the Bernoulli channel, that is, when data is transmitted on this channel, there is a certain probability of failure. For the considered constrained optimization problem with Bernoulli channel, we first punish the risk constraint into cost function through Lagrange multiplier method, and the resulting augmented cost function will include a quadratic mean-field term of state. In the sequel, for any but fixed multiplier, explicit solutions to finite-horizon and infinite-horizon mean-field decentralized linear-quadratic problems are derived together with necessary and sufficient condition on the mean-square stability of optimal system. Then, approach to find the optimal Lagrange multiplier is presented based on bisection method. Finally, two numerical examples are given to show the efficiency of the obtained results.

Stochastic Linear-Quadratic Optimal Control Problems with Random Coefficients and Markovian Regime Switching System

Stochastic Linear-Quadratic Optimal Control Problems with Random Coefficients and Markovian Regime Switching System

Zero-Sum Stackelberg Stochastic Linear-Quadratic Differential Games

The paper is concerned with a zero-sum Stackelberg stochastic linear-quadratic (LQ) differential game over finite horizons. Under a fairly weak condition, the Stackelberg equilibrium is explicitly obtained by first solving a forward stochastic LQ optimal control problem (SLQ problem) and then a backward SLQ problem. Two Riccati equations are derived for constructing the Stackelberg equilibrium. An interesting finding is that the difference of these two Riccati equations coincides with the Riccati equation associated with the zero-sum Nash stochastic LQ differential game, which implies that under the uniform convexity-concavity condition, the Stackelberg equilibrium and the Nash equilibrium are actually identical. Consequently, the Stackelberg equilibrium admits a linear state feedback representation, and the Nash game can be solved in a leader-follower manner.

Erratum to: Turnpike Properties for Stochastic Linear-Quadratic Optimal Control Problems

Erratum to: Turnpike Properties for Stochastic Linear-Quadratic Optimal Control Problems

A stochastic linear-quadratic optimal control problem with jumps in an infinite horizon

<abstract><p>In this paper, a stochastic linear-quadratic (LQ, for short) optimal control problem with jumps in an infinite horizon is studied, where the state system is a controlled linear stochastic differential equation containing affine term driven by a one-dimensional Brownian motion and a Poisson stochastic martingale measure, and the cost functional with respect to the state process and control process is quadratic and contains cross terms. Firstly, in order to ensure the well-posedness of our stochastic optimal control of infinite horizon with jumps, the $ L^2 $-stabilizability of our control system with jump is introduced. Secondly, it is proved that the $ L^2 $-stabilizability of our control system with jump is equivalent to the non-emptiness of the admissible control set for all initial state and is also equivalent to the existence of a positive solution to some integral algebraic Riccati equation (ARE, for short). Thirdly, the equivalence of the open-loop and closed-loop solvability of our infinite horizon optimal control problem with jumps is systematically studied. The corresponding equivalence is established by the existence of a $ stabilizing\ solution $ of the associated generalized algebraic Riccati equation, which is different from the finite horizon case. Moreover, any open-loop optimal control for the initial state $ x $ admiting a closed-loop representation is obatined.</p></abstract>

Indefinite Backward Stochastic Linear-Quadratic Optimal Control Problems

This paper is concerned with a backward stochastic linear-quadratic (LQ, for short) optimal control problem with deterministic coefficients. The weighting matrices are allowed to be indefinite, and cross-product terms in the control and state processes are presented in the cost functional. Based on a Hilbert space method, necessary and sufficient conditions are derived for the solvability of the problem, and a general approach for constructing optimal controls is developed. The crucial step in this construction is to establish the solvability of a Riccati-type equation, which is accomplished under a fairly weak condition by investigating the connection with forward stochastic LQ optimal control problems.

Closed-Loop Solvability of Stochastic Linear-Quadratic Optimal Control Problems with Poisson Jumps

The stochastic linear–quadratic optimal control problem with Poisson jumps is addressed in this paper. The coefficients in the state equation and the weighting matrices in the cost functional are all deterministic but are allowed to be indefinite. The notion of closed-loop strategies is introduced, and the sufficient and necessary conditions for the closed-loop solvability are given. The optimal closed-loop strategy is characterized by a Riccati integral–differential equation and a backward stochastic differential equation with Poisson jumps. A simple example is given to demonstrate the effectiveness of the main result.

Turnpike Properties for Stochastic Linear-Quadratic Optimal Control Problems

This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time-horizon [0, T] as T → ∞. The so-called turnpike properties are established for such problems, under stabilizability condition which is weaker than the controllability, normally imposed in the similar problem for ordinary differential systems. In dealing with the turnpike problem, a crucial issue is to determine the corresponding static optimization problem. Intuitively mimicking the deterministic situations, it seems to be natural to include both the drift and the diffusion expressions of the state equation to be zero as constraints in the static optimization problem. However, this would lead us to a wrong direction. It is found that the correct static problem should contain the diffusion as a part of the objective function, which reveals a deep feature of the stochastic turnpike problem.

Constrained stochastic LQ control with regime switching and application to portfolio selection

This paper is concerned with a stochastic linear-quadratic optimal control problem with regime switching, random coefficients and cone control constraint. The randomness of the coefficients comes from two aspects: the Brownian motion and the Markov chain. Using Itô’s lemma for Markov chain, we obtain the optimal state feedback control and optimal cost value explicitly via two new systems of extended stochastic Riccati equations (ESREs). We prove the existence and uniqueness of the two ESREs using tools including multidimensional comparison theorem, truncation function technique, log transformation and the John–Nirenberg inequality. These results are then applied to study mean-variance portfolio selection problems with and without short-selling prohibition with random parameters depending on both the Brownian motion and the Markov chain. Finally, the efficient portfolios and efficient frontiers are presented in closed forms.

A mean-field stochastic linear-quadratic optimal control problem with jumps under partial information

In this article, the stochastic linear-quadratic optimal control problem of mean-field type with jumps under partial information is discussed. The state equation which contains affine terms is a SDE with jumps driven by a multidimensional Brownian motion and a Poisson stochastic martingale measure, and the quadratic cost function contains cross terms. In addition, the state and the control as well as their expectations are contained both in the state equation and the cost functional. This is the so-called optimal control problem of mean-field type. Firstly, the existence and uniqueness of the optimal control is proved. Secondly, the adjoint processes of the state equation is introduced, and by using the duality technique, the optimal control is characterized by the stochastic Hamiltonian system. Thirdly, by applying a decoupling technology, we deduce two integro-differential Riccati equations and get the feedback representation of the optimal control under partial information. Fourthly, the existence and uniqueness of the solutions of two Riccati equations are proved. Finally, we discuss a special case, and establish the corresponding feedback representation of the optimal control by means of filtering technique.

General indefinite backward stochastic linear-quadratic optimal control problems

A general backward stochastic linear-quadratic optimal control problem is studied, in which both the state equation and the cost functional contain the nonhomogeneous terms. The main feature of the problem is that the weighting matrices in the cost functional are allowed to be indefinite, and the cross-product terms in the control and the state processes are present. Necessary and sufficient conditions for the solvability of the problem are obtained, and a characterization of the optimal control in terms of forward-backward stochastic differential equations is derived. By a Riccati equation approach, a general procedure for constructing optimal controls is developed and the value function is obtained explicitly.

Conditional LQ time-inconsistent Markov-switching stochastic optimal control problem for diffusion with jumps

The paper presents a characterization of equilibrium in a game-theoretic description of discounting conditional stochastic linear-quadratic (LQ for short) optimal control problem, in which the controlled state process evolves according to a multidimensional linear stochastic differential equation, when the noise is driven by a Poisson process and an independent Brownian motion under the effect of a Markovian regime-switching. The running and the terminal costs in the objective functional are explicitly dependent on several quadratic terms of the conditional expectation of the state process as well as on a nonexponential discount function, which create the time-inconsistency of the considered model. Open-loop Nash equilibrium controls are described through some necessary and sufficient equilibrium conditions. A state feedback equilibrium strategy is achieved via certain differential-difference system of ODEs. As an application, we study an investment–consumption and equilibrium reinsurance/new business strategies for mean-variance utility for insurers when the risk aversion is a function of current wealth level. The financial market consists of one riskless asset and one risky asset whose price process is modeled by geometric Lévy processes and the surplus of the insurers is assumed to follow a jump-diffusion model, where the values of parameters change according to continuous-time Markov chain. A numerical example is provided to demonstrate the efficacy of theoretical results.

Constrained stochastic LQ control on infinite time horizon with regime switching

This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem explicitly. Finally, we apply these results to solve a lifetime portfolio selection problem of tracking a given wealth level with regime switching and portfolio constraint.

Strong error estimates for a space-time discretization of the linear-quadratic control problem with the stochastic heat equation with linear noise

Abstract We propose a time-implicit, finite-element-based space-time discretization of the necessary and sufficient optimality conditions for the stochastic linear-quadratic optimal control problem with the stochastic heat equation driven by linear noise of type $[X(t)+\sigma (t)]\,\,\textrm{d}W(t)$ and prove optimal convergence w.r.t. both space and time discretization parameters. In particular, we employ the stochastic Riccati equation as a proper analytical tool to handle the linear noise, and thus extend the applicability of the earlier work by Prohl & Wang (2021, Strong rates of convergence for a space-time discretization of the backward stochastic heat equation, and of a linear-quadratic control problem for the stochastic heat equation. ESAIM Control Optim. Calc. Var., 27, 54), where the error analysis was restricted to additive noise.

Indefinite stochastic linear-quadratic optimal control problems with random coefficients: Closed-loop representation of open-loop optimal controls

This paper is concerned with a stochastic linear-quadratic optimal control problem in a finite time horizon, where the coefficients of the control system are allowed to be random, and the weighting matrices in the cost functional are allowed to be random and indefinite. It is shown, with a Hilbert space approach, that for the existence of an open-loop optimal control, the convexity of the cost functional (with respect to the control) is necessary; and the uniform convexity, which is slightly stronger, turns out to be sufficient, which also leads to the unique solvability of the associated stochastic Riccati equation. Further, it is shown that the open-loop optimal control admits a closed-loop representation. In addition, some sufficient conditions are obtained for the uniform convexity of the cost functional, which are strictly more general than the classical conditions that the weighting matrix-valued processes are positive (semi-) definite.

Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability

<p style='text-indent:20px;'>This paper is concerned with mean-field stochastic linear-quadratic (MF-SLQ, for short) optimal control problems with deterministic coefficients. The notion of weak closed-loop optimal strategy is introduced. It is shown that the open-loop solvability is equivalent to the existence of a weak closed-loop optimal strategy. Moreover, when open-loop optimal controls exist, there is at least one of them admitting a state feedback representation, which is the outcome of a weak closed-loop optimal strategy. Finally, an example is presented to illustrate the procedure for finding weak closed-loop optimal strategies.

Optimal control for controllable stochastic linear systems

This paper is concerned with a constrained stochastic linear-quadratic optimal control problem, in which the terminal state is fixed and the initial state is constrained to lie in a stochastic linear manifold. The controllability of stochastic linear systems is studied. Then the optimal control is explicitly obtained by considering a parameterized unconstrained backward LQ problem and an optimal parameter selection problem. A notable feature of our results is that, instead of solving an equation involving derivatives with respect to the parameter, the optimal parameter is characterized by a matrix equation.

A discrete-time mean-field stochastic linear-quadratic optimal control problem with financial application

This paper is concerned with a discrete-time mean-field stochastic linear-quadratic optimal control problem arising from financial application. Through matrix dynamical optimisation method, a group of linear feedback controls is investigated. The problem is then reformulated as an operator stochastic linear-quadratic optimal control problem by a sequence of bounded linear operators over Hilbert space, the optimal control with six algebraic Riccati difference equations is obtained by backward induction. The two above approaches are proved to be coincided by the classical method of completing the square. Finally, after discussing the solution of the problem under multidimensional noises, a financial application example is given.

Weak closed-loop solvability of stochastic linear-quadratic optimal control problems

Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed-loop optimal strategy for LQ problems so that its existence is equivalent to the open-loop solvability of the LQ problem. Moreover, there is at least one open-loop optimal control admitting a state feedback representation. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies.

An Open-Loop Stackelberg Strategy for the Linear Quadratic Mean-Field Stochastic Differential Game

This paper is concerned with the open-loop linear–quadratic (LQ) Stackelberg game of the mean-field stochastic systems in finite horizon. By means of two generalized differential Riccati equations, the follower first solves a mean-field stochastic LQ optimal control problem. Then, the leader turns to solve an optimization problem for a linear mean-field forward–backward stochastic differential equation. By introducing new state and costate variables, we present a sufficient condition for the existence and uniqueness of the Stackelberg strategy in terms of the solvability of some Riccati equations and a convexity condition. Furthermore, it is shown that the open-loop Stackelberg equilibrium admits a feedback representation involving the new state and its mean. Finally, two examples are given to show the effectiveness of the proposed results.