Stochastic Linear Quadratic Control Research Articles

Uncertain stochastic linear quadratic control subject to forward and backward multi-stage systems

Uncertain stochastic linear quadratic control subject to forward and backward multi-stage systems

ON SENSITIVITY OF SOLUTIONS OF RICCATI EQUATIONS UNDER SMALL PARAMETER PERTURBATIONS AND OPTIMALITY IN LINEAR STOCHASTIC CONTROL SYSTEMS

We investigate sensitivity of solutions of Riccati equations under asymptotically small perturbations of their coefficients. Upper bound on the difference between solutions of algebraic and differential Riccati equations is derived. The result is applied to study optimality in the stochastic linear-quadratic control problem over an infinite time-horizon for an asymptotically autonomous system. We also treat an issue related to performance of invariant control strategy.

Stochastic linear–quadratic control problems with affine constraints

This paper investigates the stochastic linear–quadratic control problems with affine constraints, in which both equality and inequality constraints are involved. With the help of the Pontryagin maximum principle and Lagrangian duality theory, both the dual problem and the state feedback form of the solution are obtained for the primal problem. Under the Slater condition, the strong duality is proved between the dual problem and the primal problem, and the KKT condition is also provided for solving the primal problem. Moreover, a new sufficient condition is given for the invertibility assumption, which ensures the uniqueness of the solutions to the dual problem. Finally, two numerical examples are provided to illustrate our main results.

Reducing Obizhaeva–Wang-type trade execution problems to LQ stochastic control problems

We start with a stochastic control problem where the control process is of finite variation (possibly with jumps) and acts as integrator both in the state dynamics and in the target functional. Problems of such type arise in the stream of literature on optimal trade execution pioneered by Obizhaeva and Wang (J. Financ. Mark. 16:1–32, 2013) (models with finite resilience). We consider a general framework where the price impact and the resilience are stochastic processes. Both are allowed to have diffusive components. First we continuously extend the problem from processes of finite variation to progressively measurable processes. Then we reduce the extended problem to a linear–quadratic (LQ) stochastic control problem. Using the well-developed theory on LQ problems, we describe the solution to the obtained LQ one and translate it back to the solution for the (extended) initial trade execution problem. Finally, we illustrate our results by several examples. Among other things, the examples discuss the Obizhaeva–Wang model with random (terminal and moving) targets, the necessity to extend the initial trade execution problem to a reasonably large class of progressively measurable processes (even going beyond semimartingales), and the effects of diffusive components in the price impact process and/or the resilience process.

Optimal Feedback for Stochastic Linear Quadratic Control and Backward Stochastic Riccati Equations in Infinite Dimensions

It is a longstanding unsolved problem to characterize the optimal feedbacks for general SLQs (i.e., stochastic linear quadratic control problems) with random coefficients in infinite dimensions; while the same problem but in finite dimensions was just addressed very recently. This paper is devoted to giving a solution to this problem under some assumptions which can be verified for interesting concrete models. More precisely, under these assumptions, we establish the equivalence between the existence of optimal feedback operator for infinite dimensional SLQs and the solvability of the corresponding operator-valued, backward stochastic Riccati equations. A key contribution of this work is to introduce a suitable notion of solutions (i.e., transposition solutions to the aforementioned Riccati equations), which plays a crucial role in both the statement and the proof of our main results.

Stochastic controls of fractional Brownian motion

Abstract We consider a stochastic control problem for a non-linear forward-backward stochastic differential equation driven by fractional Brownian motion, with Hurst parameter H ∈ ( 0 , 1 ) {H\in(0,1)} , in the case where the set of the control domain is convex. We provide an estimation of the solution and establish the necessary and sufficient optimality conditions in the form of the stochastic maximum principle. We apply the theory to solve a linear quadratic stochastic control problem.

Constrained mean-variance investment-reinsurance under the Cramér–Lundberg model with random coefficients

In this paper, we study an optimal mean-variance investment-reinsurance problem for an insurer (she) under the Cramér–Lundberg model with random coefficients. At any time, the insurer can purchase reinsurance or acquire new business and invest her surplus in a security market consisting of a risk-free asset and multiple risky assets, subject to a general convex cone investment constraint. We reduce the problem to a constrained stochastic linear-quadratic control problem with jumps whose solution is related to a system of partially coupled stochastic Riccati equations (SREs). Then we devote ourselves to establishing the existence and uniqueness of solutions to the SREs by pure backward stochastic differential equation (BSDE) techniques. We achieve this with the help of approximation procedure, comparison theorems for BSDEs with jumps, log transformation and BMO martingales. The efficient investment-reinsurance strategy and efficient mean-variance frontier are explicitly given through the solutions of the SREs, which are shown to be a linear feedback form of the wealth process and a half-line, respectively.

Stochastic linear quadratic optimal control problem with terminal state constraints

AbstractThis paper addresses the stochastic linear quadratic (LQ) control problem with first‐ and second‐order moment constraints on the terminal state. The problem is a modified version of the optimal covariance control problem, where the terminal state is steered to a given probability distribution. Studying a multiplicative‐noise stochastic system rather than an additive‐noise system is a salient feature. Unlike the existing ideas in the optimal steering, by using the Lagrange multipliers method and establishing the stochastic maximum principle, our problem is converted into solving forward–backward stochastic difference equations (FBSDEs), which is a special stochastic two‐point boundary‐value problem (TPBVP). We provide the optimal closed‐loop controller and necessary and sufficient solvability conditions by developing a nonhomogeneous relationship between the optimal state and costate in FBSDEs. Finally, numerical examples are given to demonstrate our results.

Time-inconsistent stochastic linear-quadratic control problem with indefinite control weight costs

Time-inconsistent stochastic linear-quadratic control problem with indefinite control weight costs

Error Analysis of the Feedback Controls Arising in the Stochastic Linear Quadratic Control Problems

Error Analysis of the Feedback Controls Arising in the Stochastic Linear Quadratic Control Problems

Optimal regulator for a class of nonlinear stochastic systems with random coefficients

We consider an optimal regulator problem for a class of nonlinear stochastic systems with a square-root nonlinearity and random coefficients, and using the quadratic-linear criterion. This represents a certain nonlinear generalisation of the stochastic linear-quadratic control problem with random coefficients. The solution if found in an explicit closed-form as an affine state-feedback control in terms of a Riccati and linear backward stochastic differential equations. As an application, we give the solution to an optimal investment problem in a market with random coefficients.

Temporal Semi-discretizations of a Backward Semilinear Stochastic Evolution Equation

This paper studies the convergence of three temporal semi-discretizations for a backward semilinear stochastic evolution equation. For general terminal value and general coefficient with Lipschitz continuity, the convergence of the first two temporal semi-discretizations is established, and an explicit convergence rate is derived for the third temporal semi-discretization. The third temporal semi-discretization is applied to a general stochastic linear quadratic control problem, and the convergence of a temporally semi-discrete approximation to the optimal control is established.

On the stochastic linear quadratic optimal control problem by piecewise constant controls: The infinite horizon time case

This paper is devoted to the problem of indefinite stochastic linear quadratic (LQ) optimal control by piecewise constant controls in an infinite horizon case. By restricting the set of admissible controls to the class of piecewise constant stochastic processes, we reformulated the above control problem under the setting of systems modeled by Itô differential equations controlled by impulses. We show that the solution in a state feedback form of the indefinite stochastic LQ control problem is equivalent to the existence of a global stabilizing solution associated to a class of backward matrix linear differential equations with a Riccati type jumping operators.

Optimal feedback controls of stochastic linear quadratic control problems in infinite dimensions with random coefficients

It is a longstanding unsolved problem to characterize the optimal feedback controls for general linear quadratic optimal control problem of stochastic evolution equation with random coefficients. A solution to this problem is given in [22] under some assumptions which can be verified for interesting concrete models, such as controlled stochastic wave equations, controlled stochastic Schrödinger equations, etc. More precisely, the authors establish the equivalence between the existence of an optimal feedback operator and the solvability of the corresponding operator-valued, backward stochastic Riccati equations. However, their result cannot cover some important stochastic partial differential equations, such as stochastic heat equations, stochastic stokes equations, etc. A key contribution of the current work is to relax the C0-group assumption of unbounded linear operator A in [22] and using the contraction semigroup assumption instead. Therefore, our result can be well applicable in the linear quadratic problem of stochastic parabolic equations. To this end, we introduce a suitable notion to the aforementioned Riccati equation, and some delicate techniques which are even new in the finite dimensional case.

Stochastic linear-quadratic control with a jump and regime switching on a random horizon

In this paper, we study a stochastic linear-quadratic control problem with random coefficients and regime switching on a random horizon $ [0,T\wedge\tau] $, where $ \tau $ is a given random jump time for the underlying state process and $ T $ a constant. We obtain the explicit optimal feedback control and explicit optimal value of the problem by solving a system of stochastic Riccati equations (SREs) with jumps on the random horizon $ [0,T\wedge\tau] $. By the decomposition approach stemming from filtration enlargement theory, we express the solution to the system of SREs with jumps in terms of another system of SREs involving only Brownian filtration on the deterministic horizon $ [0,T] $. Solving the latter system is the key theoretical contribution of this paper and we accomplish this under three different conditions, one of which seems to be new in the literature. The above results are then applied to study a mean-variance hedging problem with random parameters that depend on both Brownian motion and Markov chain. The optimal portfolio and optimal value are presented in closed forms with the aid of a system of linear backward stochastic differential equations with jumps and unbounded coefficients in addition to the SREs with jumps.

Risk-Aware Linear Quadratic Control Using Conditional Value-at-Risk

Stochastic linear quadratic control problems are considered from the viewpoint of risks. In particular, a worst-case conditional value-at-risk (CVaR) of quadratic objective function is minimized subject to additive disturbances whose first two moments of the distribution are known. The study focuses on three problems of finding the optimal feedback gain that minimizes the quadratic cost of: stationary distribution, one-step, and infinite time horizon. For the stationary distribution problem, it is proved that the optimal control gain that minimizes the worst-case CVaR of the quadratic cost is equivalent to that of the standard (stochastic) linear quadratic regulator. For the one-step problem, an approach to an optimal solution as well as analytical suboptimal solutions are presented. For the infinite time horizon problem, two suboptimal solutions that bound the optimal solution and an approach to an optimal solution for a special case are discussed. The presented theorems are illustrated with numerical examples.

Optimal control on investment and reinsurance strategies with delay and common shock dependence in a jump-diffusion financial market

<p style='text-indent:20px;'>In this paper, we consider an optimal mean-variance investment and reinsurance problem with delay and Common Shock Dependence. An insurer can control the claim risk by purchasing proportional reinsurance. He/she invests his/her wealth on a risk-free asset and a risky asset, which follows the jump-diffusion process. By introducing a capital flow related to the historical performance of the insurer, the wealth process described by a stochastic differential equation with delay is obtained. By stochastic linear-quadratic control theory and stochastic control theory with delay, we achieve the explicit expression of the optimal strategy and value function in the framework of the viscosity solution. Furthermore, an efficient strategy and its efficient frontier are derived by Lagrange dual method. Finally, we analyze the influence of the parameters of our model on the efficient frontier by a numerical example.</p>

Optimal order execution under price impact: a hybrid model

In this paper we explore optimal liquidation in a market populated by a number of heterogeneous market makers that have limited inventory-carrying and risk-bearing capacity. We derive a reduced form model for the dynamics of their aggregated inventory considering a proper scaling limit. The resulting price impact profile is shown to depend on the characteristics and relative importance of their inventories. The model is flexible enough to reproduce the empirically documented power law behavior of the price impact function. For any choice of the market makers characteristics, optimal execution within this modeling approach can be recast as a linear-quadratic stochastic control problem. The value function and the associated optimal trading rate can be obtained semi-explicitly subject to solving a differential matrix Riccati equation. Numerical simulations are conducted to illustrate the performance of the resulting optimal liquidation strategy in relation to standard benchmarks. Remarkably, they show that the increase in performance is determined by a substantial reduction of higher order moment risk.

Linear quadratic Gaussian control with advanced continuous-time disturbance models for building thermal regulation

This paper introduces a linear quadratic control scheme for a continuous-time system with observations taken at discrete times. Particular attention is given to the derivation of the disturbance terms in the model. Control performance may depend critical on accurate disturbance forecasts. This is the case for building climate control, where solar rays pass through e.g. windows and deliver significant amounts of energy and the dynamics can be very fast, fluctuating, and spontaneous. We thus argue that it is critical for control performance to sufficiently describe and include disturbances in the control description to obtain satisfactory control accuracy. We suggest and derive in details a control framework based on continuous-time stochastic differential equations (SDEs) and linear quadratic Gaussian control using an advanced continuous-time disturbance model to supply disturbance forecasts. The numerical simulation results suggest that control with embedded forecasts handles uncertainties well and provides up to 26% performance improvements compared to standard disturbance mitigation techniques. Furthermore, we demonstrate that the quadratic controller has a useful trade-off between variability in the control signal, economic cost, and variability around the reference point.

Newton Method for Stochastic Control Problems

We develop a new iterative method based on Pontryagin principle to solve stochastic control problems. This method is nothing else than the Newton method extended to the framework of stochastic controls, where the state dynamics is given by an ODE with stochastic coefficients. Each iteration of the method is made of two ingredients: computing the Newton direction, and finding an adapted step length. The Newton direction is obtained by solving an affine-linear Forward-Backward Stochastic Differential Equation (FBSDE) with random coefficients. This is done in the setting of a general filtration. We prove that solving such an FBSDE reduces to solving a Riccati Backward Stochastic Differential Equation (BSDE) and an affine-linear BSDE, as expected in the framework of linear FBSDEs or Linear-Quadratic stochastic control problems. We then establish convergence results for this Newton method. In particular, sufficient regularity of the second-order derivative of the cost functional is required to obtain (local) quadratic convergence. A restriction to the space of essentially bounded stochastic processes is needed to obtain such regularity. To choose an appropriate step length while fitting our choice of space of processes, an adapted backtracking line-search method is developed. We then prove global convergence of the Newton method with the proposed line-search procedure, which occurs at a quadratic rate after finitely many iterations. An implementation with regression techniques to solve BSDEs arising in the computation of the Newton step is developed. We apply it to the control problem of a large number of batteries providing ancillary services to an electricity network.