Stochastic Linear Quadratic Control Problem Research Articles

Optimal Reinsurance and Investment Strategies Under Mean-Variance Criteria: Partial and Full Information

This paper is concerned with an optimal reinsurance and investment problem for an insurance firm under the criterion of mean-variance. The driving Brownian motion and the rate in return of the risky asset price dynamic equation cannot be directly observed. And the short-selling of stocks is prohibited. The problem is formulated as a stochastic linear-quadratic control problem where the control variables are constrained. Based on the separation principle and stochastic filtering theory, the partial information problem is solved. Efficient strategies and efficient frontier are presented in closed forms via solutions to two extended stochastic Riccati equations. As a comparison, the efficient strategies and efficient frontier are given by the viscosity solution to the HJB equation in the full information case. Some numerical illustrations are also provided.

Optimal solution of the liquidation problem under execution and price impact risks

We consider an investor that trades continuously and wants to liquidate an initial asset position within a prescribed time interval. As a consequence of his trading activity, during the execution of the liquidation order, the investor has no guarantees that the placed order is executed immediately; it may go unfilled, partially filled or filled in excess. The uncertainty in the execution affects the trading activity of the investor and the asset share price dynamics generating additional sources of noise: the execution risk and the price impact risk, respectively. Assuming the two sources of noise correlated and driven by the cumulative effect of the investor trading strategy, we study the problem of finding the optimal liquidation strategy adopted by the investor in order to maximize the expected revenue resulting from the liquidation. The mathematical model of the liquidation problem presented here extends the model of Almgren and Chriss [Optimal execution of portfolio transactions. J. Risk, 2000, 3(2), 5–39] to include execution and price impact risks. The liquidation problem is modeled as a linear quadratic stochastic optimal control problem with the finite horizon and, under some assumptions about the functional form for the magnitude of execution and price impact risks, is solved explicitly. The derived solution coincides with the optimal trading strategy obtained in the absence of execution uncertainty for an asset price with a modified growth rate. This suggests that the uncertainty in the execution modifies the directional view of the investor about the future growth rate of the asset price.

Conditional LQ time-inconsistent Markov-switching stochastic optimal control problem for diffusion with jumps

The paper presents a characterization of equilibrium in a game-theoretic description of discounting conditional stochastic linear-quadratic (LQ for short) optimal control problem, in which the controlled state process evolves according to a multidimensional linear stochastic differential equation, when the noise is driven by a Poisson process and an independent Brownian motion under the effect of a Markovian regime-switching. The running and the terminal costs in the objective functional are explicitly dependent on several quadratic terms of the conditional expectation of the state process as well as on a nonexponential discount function, which create the time-inconsistency of the considered model. Open-loop Nash equilibrium controls are described through some necessary and sufficient equilibrium conditions. A state feedback equilibrium strategy is achieved via certain differential-difference system of ODEs. As an application, we study an investment–consumption and equilibrium reinsurance/new business strategies for mean-variance utility for insurers when the risk aversion is a function of current wealth level. The financial market consists of one riskless asset and one risky asset whose price process is modeled by geometric Lévy processes and the surplus of the insurers is assumed to follow a jump-diffusion model, where the values of parameters change according to continuous-time Markov chain. A numerical example is provided to demonstrate the efficacy of theoretical results.

Constrained stochastic LQ control on infinite time horizon with regime switching

This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem explicitly. Finally, we apply these results to solve a lifetime portfolio selection problem of tracking a given wealth level with regime switching and portfolio constraint.

Indefinite Linear-Quadratic Stochastic Control Problem for Jump-Diffusion Models with Random Coefficients: A Completion of Squares Approach

In this paper, we study the indefinite linear-quadratic (LQ) stochastic optimal control problem for stochastic differential equations (SDEs) with jump diffusions and random coefficients driven by both the Brownian motion and the (compensated) Poisson process. In our problem setup, the coefficients in the SDE and the objective functional are allowed to be random, and the jump-diffusion part of the SDE depends on the state and control variables. Moreover, the cost parameters in the objective functional need not be (positive) definite matrices. Although the solution to this problem can also be obtained through the stochastic maximum principle or the dynamic programming principle, our approach is simple and direct. In particular, by using the Itô-Wentzell’s formula, together with the integro-type stochastic Riccati differential equation (ISRDE) and the backward SDE (BSDE) with jump diffusions, we obtain the equivalent objective functional that is quadratic in control u under the positive definiteness condition, where the approach is known as the completion of squares method. Then the explicit optimal solution, which is linear in state characterized by the ISRDE and the BSDE jump diffusions, and the associated optimal cost are derived by eliminating the quadratic term of u in the equivalent objective functional. We also verify the optimality of the proposed solution via the verification theorem, which requires solving the stochastic HJB equation, a class of stochastic partial differential equations with jump diffusions.

Error analysis of a discretization for stochastic linear quadratic control problems governed by SDEs

Abstract In this work, a time-implicit discretization for stochastic linear quadratic problems subject to stochastic differential equations with control-dependence noises is proposed, and the convergence rate of this discretization is proved. Compared to the existing results, the control variables are stochastic processes and can be contained in systems’ diffusion term. Based on this discretization, a gradient descent algorithm and its convergence rate are presented. Finally, a numerical example is provided to support the theoretical finding.

Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system

<p style="text-indent:20px;">This paper investigates the mean-field stochastic linear quadratic optimal control problem of Markov regime switching system (M-MF-SLQ, for short). The representation of the cost functional for the M-MF-SLQ is derived using the technique of operators. It is shown that the convexity of the cost functional is necessary for the finiteness of the M-MF-SLQ problem, whereas uniform convexity of the cost functional is sufficient for the open-loop solvability of the problem. By considering a family of uniformly convex cost functionals, a characterization of the finiteness of the problem is derived and a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. We demonstrate with a few examples that our results can be employed for tackling some financial problems such as mean-variance portfolio selection problem.</p>

Stochastic Linear Quadratic Control Problem on Time Scales

This paper addresses a version of the stochastic linear quadratic control problem on time scales S Δ LQ , which includes the discrete time and continuous time as special cases. Riccati equations on time scales are given, and the optimal control can be expressed as a linear state feedback. Furthermore, we present the uniqueness and existence of the solution to the Riccati equation on time scales. Furthermore, we give an example to illustrate the theoretical results.

A BSDE approach to stochastic linear quadratic control problem

AbstractIn this article, we study a kind of linear quadratic optimal control problem driven by forward–backward stochastic differential equations (FBSDEs in short) with deterministic coefficients. The cost functional is defined by the solution of FBSDEs. By means of the Girsanov transformation, the original issue is turned equivalently into the classical LQ problem. By functional analysis approach, some necessary and sufficient conditions for the existence of optimal controls have been obtained. Moreover, we investigate the relationship between two groups of first‐order and second‐order adjoint equations. A new stochastic Riccati equation is derived, which leads to the state feedback form of optimal control. By introducing a new Hamiltonian function with an exponential factor, we establish the stochastic maximum principle to deal with the stochastic linear quadratic problem for forward–backward stochastic system with nonconvex control domain using first‐order adjoint equation. An illustrative example is given as well.

Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markovian regime switching system

This paper investigates the stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. The representation of the cost functional for the stochastic LQ optimal control problem of Markovian regime switching system is derived by the technique of Itô’s formula with jumps. For the stochastic LQ optimal control problem of Markovian regime switching system, we establish the equivalence between the open-loop (closed-loop, resp.) solvability and the existence of an adapted solution to the corresponding forward-backward stochastic differential equation with constraint. (i.e., the existence of a regular solution to Riccati equations). Also, we analyze the interrelationship between the strongly regular solvability of Riccati equations and the uniform convexity of the cost functional. Finally, we present an example which is open-loop solvable but not closed-loop solvable.

A concise introduction to control theory for stochastic partial differential equations

<p style='text-indent:20px;'>The aim of this notes is to give a concise introduction to control theory for systems governed by stochastic partial differential equations. We shall mainly focus on controllability and optimal control problems for these systems. For the first one, we present results for the exact controllability of stochastic transport equations, null and approximate controllability of stochastic parabolic equations and lack of exact controllability of stochastic hyperbolic equations. For the second one, we first introduce the stochastic linear quadratic optimal control problems and then the Pontryagin type maximum principle for general optimal control problems. It deserves mentioning that, in order to solve some difficult problems in this field, one has to develop new tools, say, the stochastic transposition method introduced in our previous works.

Mean-field linear-quadratic stochastic differential games in an infinite horizon

This paper is concerned with two-person mean-field linear-quadratic non-zero sum stochastic differential games in an infinite horizon. Both open-loop and closed-loop Nash equilibria are introduced. The existence of an open-loop Nash equilibrium is characterized by the solvability of a system of mean-field forward-backward stochastic differential equations in an infinite horizon and the convexity of the cost functionals, and the closed-loop representation of an open-loop Nash equilibrium is given through the solution to a system of two coupled non-symmetric algebraic Riccati equations. The existence of a closed-loop Nash equilibrium is characterized by the solvability of a system of two coupled symmetric algebraic Riccati equations. Two-person mean-field linear-quadratic zero-sum stochastic differential games in an infinite horizon are also considered. Both the existence of open-loop and closed-loop saddle points are characterized by the solvability of a system of two coupled generalized algebraic Riccati equations with static stabilizing solutions. Mean-field linear-quadratic stochastic optimal control problems in an infinite horizon are discussed as well, for which it is proved that the open-loop solvability and closed-loop solvability are equivalent.

Stochastic linear quadratic control via random parameter‐dependent truncated balanced realization

AbstractThis article deals with stochastic linear quadratic (LQ) control by using random parameter‐dependent truncated balanced realization (RPD‐TBR). The main contribution of the article is theoretical formulation to approximate high‐order stochastic LQ control problems, using the RPD‐TBR instead of the original RPD full state model. The whole approach uses the nonintrusive and intrusive generalized polynomial chaos (GPC) approximations to generate an RPD‐TBR and the corresponding reduced‐order stochastic LQ control formulation. The latter is then rewritten in the GPC space using an intrusive Galerkin‐type projection. Solving the reduced‐order Riccati equation for the resulting deterministic LQ problem enables to suitably approximate the solution of the original full‐order stochastic LQ control problem. The feasibility and effectiveness of the proposed approach are assessed by considering the LQ control of the tip‐tilt mirror for an Extremely Large Telescope‐class adaptive optics system, when the actuator's dynamics depends on an uncertain parameter with known distribution. Through numerical simulations, the proposed approach ability to perform LQ gain selection and performance evaluation for plants with probabilistic uncertainty more efficiently than the standard Monte Carlo approach is illustrated.

Solvability for indefinite mean‐field stochastic linear quadratic optimal control with random jumps and its applications

SummaryIn this article, we study the open‐loop and closed‐loop solvability for indefinite mean‐field stochastic linear quadratic (MF‐SLQ) optimal control problem and its application in finance, where the controlled stochastic system is driven by a Brownian motion and a Poisson random martingale measure and also disturbed by some stochastic processes. The intrinsic property of stochastic systems results in the inequivalence of those two solvabilities, which is different from deterministic case. Based on a well‐posedness result of the problem, it is shown that the uniform convexity of cost functional is sufficient for the open‐loop solvability of the problem. By a matrix minimum principle, the necessity condition of regular solvability for a decoupled Riccati equation is established, meanwhile, the closed‐loop solvability is turned out to be equivalent to the regular solvability of Riccati equations with some constraints on the solutions of disturbances equations, moreover, the optimal closed‐loop strategy is characterized by regular solutions of Riccati equations and adapted solutions of disturbances equations. And then a mean‐variance model is considered to solve optimal portfolio selection strategy problem of the insurance company with liability. Our study fills a gap in the field of solvability for mean‐field stochastic optimal control with random jumps and provides a different way for solving mean‐variance portfolio selection model.

Maximum Principle for Near-Optimality of Mean-Field FBSDEs

The present paper concerns with a near-optimal control problem for systems governed by mean-field forward-backward stochastic differential equations (FBSDEs) with mixed initial-terminal conditions. Utilizing Ekeland’s variational principle as well as the reduction method, the necessary and sufficient near-optimality conditions are established in the form of Pontryagin’s type. The results are obtained under restriction on the convexity of the control domain. As an application, a linear-quadratic stochastic control problem is solved explicitly.

The Delayed Doubly Stochastic Linear Quadratic Optimal Control Problem

In this paper, the delayed doubly stochastic linear quadratic optimal control problem is discussed. It deduces the expression of the optimal control for the general delayed doubly stochastic control system which contained time delay both in the state variable and in the control variable at the same time and proves its uniqueness by using the classical parallelogram rule. The paper is concerned with the generalized matrix value Riccati equation for a special delayed doubly stochastic linear quadratic control system and aims to give the expression of optimal control and value function by the solution of the Riccati equation.

Continuous Viscosity Solutions to Linear-Quadratic Stochastic Control Problems with Singular Terminal State Constraint

This paper establishes the existence of a unique nonnegative continuous viscosity solution to the HJB equation associated with a linear-quadratic stochastic control problem with singular terminal state constraint and possibly unbounded cost coefficients. The existence result is based on a novel comparison principle for semi-continuous viscosity sub- and supersolutions for PDEs with singular terminal value. Continuity of the viscosity solution is enough to carry out the verification argument.

Optimization of Constrained Stochastic Linear-Quadratic Control on an Infinite Horizon: A Direct-Comparison Based Approach

In this paper we study the optimization of the discrete-time stochastic linear-quadratic (LQ) control problem with conic control constraints on an infinite horizon, considering multiplicative noises. Stochastic control systems can be formulated as Markov Decision Problems (MDPs) with continuous state spaces and therefore we can apply the direct-comparison based optimization approach to solve the problem. We first derive the performance difference formula for the LQ problem by utilizing the state separation property of the system structure. Based on this, we successfully derive the optimality conditions and the stationary optimal feedback control. By introducing the optimization, we establish a general framework for infinite horizon stochastic control problems. The direct-comparison based approach is applicable to both linear and nonlinear systems. Our work provides a new perspective in LQ control problems; based on this approach, learning based algorithms can be developed without identifying all of the system parameters.

Weak Closed-Loop Solvability of Stochastic Linear Quadratic Optimal Control Problems of Markovian Regime Switching System

In this paper, we investigate the open-loop and weak closed-loop solvabilities of stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. Interestingly, these two solvabilities are equivalent. We first provide an alternative characterization of the open-loop solvability of the LQ problem using the perturbation approach. Then, we study the weak closed-loop solvability of the LQ problem of Markovian regime switching system, and establish the equivalent relationship between open-loop and weak closed-loop solvabilities. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies within the framework of Markovian regime switching system.

ON COERCIVITY AND THE FREQUENCY DOMAIN CONDITION IN INDEFINITE LQ-CONTROL

We introduce a coercivity condition as a time domain analogue of the frequency criterion provided by the famous Kalman-YakubovichPopov lemma. For a simple stochastic linear quadratic control problem we show how the coercivity condition characterizes the solvability of Riccati equations.