Stochastic Linear Quadratic Control Problem Research Articles

An Indefinite Stochastic Linear Quadratic Optimal Control Problem with Delay and Related Forward–Backward Stochastic Differential Equations

In this paper, we will study an indefinite stochastic linear quadratic optimal control problem, where the controlled system is described by a stochastic differential equation with delay. By introducing the relaxed compensator as a novel method, we obtain the well-posedness of this linear quadratic problem for indefinite case. And then, we discuss the uniqueness and existence of the solutions for a kind of anticipated forward–backward stochastic differential delayed equations. Based on this, we derive the solvability of the corresponding stochastic Hamiltonian systems, and give the explicit representation of the optimal control for the linear quadratic problem with delay in an open-loop form. The theoretical results are validated as well on the control problems of engineering and economics under indefinite condition.

Solution to stochastic LQ control problem for Itô systems with state delay or input delay

This paper is concerned with the linear quadratic (LQ) control problem for Itô stochastic systems with state delay or input delay. The main contribution is to give the explicit optimal LQ controllers for Itô stochastic systems with input delay and with state delay respectively. For the case of state delay, the optimal LQ controller is a linear function of the state at the current time and the past time. For the case of input delay, the optimal LQ controller is a linear function of the state and the past inputs. The key technology is to define the value function and complete the square based on the coupled differential equations.

Linear Quadratic Stochastic Control Problems with Stochastic Terminal Constraint

We study a linear quadratic optimal control problem with stochastic coefficients and a terminal state constraint, which may be in force merely on a set with positive, but not necessarily full, probability. Under such a partial terminal constraint, the usual approach via a coupled system of a backward stochastic Riccati equation and a linear backward equation breaks down. As a remedy, we introduce a family of auxiliary problems parametrized by the supersolutions to this Riccati equation alone. The target functional of these problems dominates the original constrained one and allows for an explicit description of both the optimal control policy and the auxiliary problem's value in terms of a suitably constructed optimal signal process. This suggests that, for the minimal supersolution of the Riccati equation, the minimizers of the auxiliary problem coincide with those of the original problem, a conjecture that we see confirmed in all examples understood so far.

A Global Stochastic Maximum Principle for Fully Coupled Forward-Backward Stochastic Systems

We study a stochastic optimal control problem for fully coupled forward-backward stochastic control systems with a nonempty control domain. For our problem, the first-order and second-order variational equations are fully coupled linear forward-backward stochastic differential equations. Inspired by Hu [Probab. Uncertain. Quant. Risk, 2 (2017), pp. 1--20], we develop a new decoupling approach by introducing an adjoint equation which is a quadratic backward stochastic differential equation. By revealing the relations among the terms of the first-order Taylor expansions, we estimate the orders of them and derive a global stochastic maximum principle which includes a completely new term. Applications to stochastic linear quadratic control problems are investigated.

Constrained Quadratic Risk Minimization via Forward and Backward Stochastic Differential Equations

In this paper we study a continuous-time stochastic linear quadratic control problem arising from mathematical finance. We model the asset dynamics with random market coefficients and portfolio strategies with convex constraints. Following the convex duality approach, we show that the necessary and sufficient optimality conditions for both the primal and dual problems can be written in terms of processes satisfying a system of forward and backward stochastic differential equations (FBSDEs) together with other conditions. We characterize explicitly the optimal wealth and portfolio processes as functions of adjoint processes from the dual FBSDEs in a dynamic fashion, and vice versa. We apply the results to solve quadratic risk minimization problems with cone constraints and derive the explicit representations of solutions to the extended stochastic Riccati equations for such problems.

Existence of a Mean-Square Stabilizing Solution to a Modified Algebraic Riccati Equation

In this paper, the existence of a mean-square stabilizing solution to a discrete-time modified algebraic Riccati equation (MARE), which arises in the study of some stochastic linear quadratic optimal control problems, is investigated. The theory of cone-invariant operators is employed as the mathematical tool to tackle this problem. We provide some criteria for the cone stability, cone observability, and cone detectability of a class of cone-invariant systems in terms of the associated distinguished eigenvalues. Then two scenarios concerning the problem of existence in terms of the positive definiteness of the input weighting matrix are considered. We first study the MARE under the assumption that the input weighting matrix in the cost function is positive definite. Then an explicit necessary and sufficient condition is obtained. Such a condition is derived for the very first time and it indicates that the common condition of observability or detectability of certain stochastic systems is unnecessary. Onl...

Linear Stochastic Differential Equations Driven by Gauss-Volterra Processes and Related Linear-Quadratic Control Problems

A stochastic linear-quadratic control problem is formulated and solved for some stochastic equations in an infinite dimensional Hilbert space for both finite and infinite time horizons. The equations are bilinear in the state and the noise process where the noise is a scalar Gauss-Volterra process. The Gauss-Volterra noise processes are obtained from the integral of a Brownian motion with a suitable kernel function. These noise processes include fractional Brownian motions with the Hurst parameter $$H \in (\frac{1}{2},1)$$ , Liouville fractional Brownian motions with $$H \in (\frac{1}{2},1)$$ , and some multifractional Brownian motions. The family of admissible controls for the quadratic costs is a family of linear feedback controls. This restriction on the family of controls allows for a feasible implementation of the optimal controls. The bilinear equations have drift terms that are linear evolution operators. These equations can model stochastic partial differential equations of parabolic and hyperbolic types and two families of examples are given.

Stochastic Linear-Quadratic Control with State Dependent Fractional Brownian Noise and Stochastic Coefficients

Stochastic linear-quadratic control problems for bilinear equations with some stochastic coefficients and driven by fractional Brownian motions with the Hurst parameters H ϵ (½,1) are formulated and solved. The family of admissible controls is the collection of linear feedback controls. An optimal control in this family is not optimal in the admissible family of all adapted controls. However an optimal linear feedback control obtained in this paper can be relatively easily implemented whereas an optimal control in the family of adapted controls has a term that predicts the future noise so it is more difficult to implement. An optimal control is determined from the solution of a Riccati equation which differs from the Riccati equation for the linear-quadratic control problem with a Brownian motion replacing the noise term. Since the Riccati equation has some stochastic coefficients it is solved as a backward stochastic differential equation. Given the solution of the stochastic Riccati equation, a direct method is used to determine the optimal linear feedback control.

Linear quadratic optimal control of nonhomogeneous vector differential equations with FBM

In this paper, the solution of stochastic linear quadratic optimal control problem of nonhomogeneous vector differential equation driven by fractional Brownian motion of Hurst parameter \( H(>0.5)\) is established. A stochastic Riccati equation is derived by considering adjoint equations satisfying backward stochastic differential equations. The feedback form of the optimal control and the optimal cost are derived using completion of squares technique. The results established are illustrated with examples.

Time-inconsistent stochastic linear quadratic control for discrete-time systems

This paper is mainly concerned with the time-inconsistent stochastic linear quadratic (LQ) control problem in a more general formulation for discrete-time systems. The time-inconsistency arises from three aspects: the coefficient matrices depending on the initial pair, the terminal of the cost function involving the initial pair together with the nonlinear terms of the conditional expectation. The main contributions are: firstly, the maximum principle is derived by using variational methods, which forms a flow of forward and backward stochastic difference equations (FBSDE); secondly, in the case of the system state being one-dimensional, the equilibrium control is obtained by solving the FBSDE with feedback gain based on several nonsymmetric Riccati equations; finally, the necessary and sufficient solvability condition for the time-inconsistent LQ control problem is presented explicitly. The key techniques adopted are the maximum principle and the solution to the FBSDE developed in this paper.

Information Transfer of Control Strategies: Dualities of Stochastic Optimal Control Theory and Feedback Capacity of Information Theory

The control-coding capacity of stochastic control systems is introduced, and its operational meaning is established using randomized control strategies, which simultaneously control output processes encode information, and communicate information from control processes to output processes. The control-coding capacity is the analog Shannon's coding-capacity of noisy channels. Furthermore, duality relations to stochastic optimal control problems with deterministic and randomized control strategies are identified including the following. First, extremum problems of stochastic optimal control with directed information payoff are equivalent to feedback capacity problems of information theory, in which the control system act as a communication channel. Second, for Gaussian linear decision models with average quadratic constraints, it is shown that optimal randomized strategies are Gaussian, and decompose into a deterministic part and a random part. The deterministic part is precisely the optimal strategy of the linear quadratic Gaussian stochastic optimal control problem, whereas the random part is the solution of an water-filling information transmission problem that encodes information, which is estimated by a decoder.

Characterization of optimal feedback for stochastic linear quadratic control problems

One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the study of deterministic linear quadratic control problems is exactly to construct the desired feedbacks. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a suitable sense. We also give a counterexample showing the nonexistence of feedback controls to a solvable stochastic linear quadratic control problem. This is a new phenomenon in the stochastic setting, significantly different from its deterministic counterpart.

A Class of Linear Quadratic Gaussian Hybrid Optimal Control Problems with Realization–Independent Riccati Equations

A class of stochastic linear quadratic hybrid optimal control problems is presented for which the Hamiltonian boundary conditions appearing in the associated necessary optimality conditions of the Stochastic Hybrid Minimum Principle and Stochastic Hybrid Dynamic Programming are path-independent. Consequently, the linear quadratic Gaussian regulation problem associated with this class of stochastic hybrid optimal control problems can be solved via (stochastic) Riccati equations which are independent of the realization of stochastic diffusion terms. An analytic example of a scalar hybrid system is provided to illustrate the results, and the relation to the deterministic case is discussed.

Numerical solution of the finite horizon stochastic linear quadratic control problem

The treatment of the stochastic linear quadratic optimal control problem with finite time horizon requires the solution of stochastic differential Riccati equations. We propose efficient numerical methods, which exploit the particular structure and can be applied for large-scale systems. They are based on numerical methods for ordinary differential equations such as Rosenbrock methods, backward differentiation formulas, and splitting methods. The performance of our approach is tested in numerical experiments.

Stochastic Linear Quadratic Optimal Control Problems in Infinite Horizon

This paper is concerned with stochastic linear quadratic (LQ, for short) optimal control problems in an infinite horizon with constant coefficients. It is proved that the non-emptiness of the admissible control set for all initial state is equivalent to the $L^2$-stabilizability of the control system, which in turn is equivalent to the existence of a positive solution to an algebraic Riccati equation (ARE, for short). Different from the finite horizon case, it is shown that both the open-loop and closed-loop solvabilities of the LQ problem are equivalent to the existence of a static stabilizing solution to the associated generalized ARE. Moreover, any open-loop optimal control admits a closed-loop representation. Finally, the one-dimensional case is worked out completely to illustrate the developed theory.

The distributionally robust complementarity problem

We investigate the linear complementarity problem with uncertain parameters (ULCP) which affect the linear mapping affinely or quadratically. Assuming that the distribution of the uncertain parameters belongs to some ambiguity set with prescribed partial information, we formulate the ULCP as a distributionally robust optimization reformulation named as the distributionally robust complementarity problem (DRCP), which minimizes the worst case of an expected complementarity measure with a joint chance constraint that the probability of the linear mapping being nonnegative is not less than a given level. Applying the cone dual theory and S-procedure, we conservatively approximate the DRCP as a nonlinear semidefinite programming (NSDP) with bilinear matrix inequalities, which can be solved by the NSDP solver PENLAB. The preliminary numerical test on a constrained stochastic linear quadratic control problem shows that the DRCP as well as the corresponding solution method is promising.

The Stochastic Linear Quadratic Control Problem with Singular Estimates

We study an infinite dimensional finite horizon stochastic linear quadratic control problem in an abstract setting. We assume that the dynamics of the problem are generated by a strongly continuous semigroup, while the control operator is unbounded and the multiplicative noise operators for the state and the control are bounded. We prove an optimal feedback synthesis along with well posedness of the Riccati equation for the finite horizon case. Our results extend the ones proposed in [C. Hafizoglu, Ph.D. Thesis, University of Virginia, Charlottesville, VA, 2006.] to the case in which disturbance in the control is considered and a final time penalization term is included in the quadratic cost functional.

Linear Quadratic Stochastic Optimal Control of Forward Backward Stochastic Control System Associated with Lévy Process

This paper analyzes one kind of linear quadratic (LQ) stochastic control problem of forward backward stochastic control system associated with Lévy process. We obtain the explicit form of the optimal control, then prove it to be unique, and get the linear feedback regulator by introducing one kind of generalized Riccati equation. Finally, we discuss the solvability of the generalized Riccati equation, and its existence and uniqueness of the solutions are proved in a special case.

Dynamic Programming for Optimal Control of Stochastic McKean--Vlasov Dynamics

We study the optimal control of general stochastic McKean-Vlasov equation. Such problem is motivated originally from the asymptotic formulation of cooperative equilibrium for a large population of particles (players) in mean-field interaction under common noise. Our first main result is to state a dynamic programming principle for the value function in the Wasserstein space of probability measures, which is proved from a flow property of the conditional law of the controlled state process. Next, by relying on the notion of differentiability with respect to probability measures due to P.L. Lions [32], and It{\^o}'s formula along a flow of conditional measures, we derive the dynamic programming Hamilton-Jacobi-Bellman equation, and prove the viscosity property together with a uniqueness result for the value function. Finally, we solve explicitly the linear-quadratic stochastic McKean-Vlasov control problem and give an application to an interbank systemic risk model with common noise.

A direct approach to linear-quadratic stochastic control

A direct approach is used to solve some linear-quadratic stochastic control problems for Brownian motion and other noise processes. This direct method does not require solving Hamilton-Jacobi-Bellman partial differential equations or backward stochastic differential equations with a stochastic maximum principle or the use of a dynamic programming principle. The appropriate Riccati equation is obtained as part of the optimization problem. The noise processes can be fairly general including the family of fractional Brownian motions.