Recursive Optimal Control Problem Research Articles

Dynamic programming principle for one kind of stochastic recursive optimal control problem with Markovian switching

Dynamic programming principle for one kind of stochastic recursive optimal control problem with Markovian switching

Dynamic programming principle for backward doubly stochastic recursive optimal control problem and sobolev weak solution of the stochastic Hamilton-Jacobi-Bellman equation

In this paper, we investigate a backward doubly stochastic recursive optimal control problem wherein the cost function is expressed as the solution to a backward doubly stochastic differential equation. We present the dynamical programming principle for this type of optimal control problem and establish that the value function is the unique Sobolev weak solution to the associated stochastic Hamilton-Jacobi-Bellman equation.

Stochastic maximum principle for recursive optimal control problems with varying terminal time

This paper introduces a new recursive stochastic optimal control problem driven by a forward-backward stochastic differential equations (FBSDEs), where the terminal time varies according to the constraints of the state of the forward equation. We can use this optimal control problem to describe the investment portfolio problems with the varying investment period. Based on novel ρ-moving variational and adjoint equations, we establish the stochastic maximum principle for this optimal control problem including the classical recursive optimal control problem as a particular case. Furthermore, we propose an example to verify the main results of this study.

Relationship between maximum principle and dynamic programming principle for stochastic recursive optimal control problem under volatility uncertainty

AbstractIn this article, we study the relationship between maximum principle (MP) and dynamic programming principle (DPP) for stochastic recursive optimal control problem driven by ‐Brownian motion. Under the smooth assumption for the value function, we obtain the connection between MP and DPP under a reference probability . Within the framework of viscosity solution, we establish the relation between the first‐order super‐jet, sub‐jet of the value function and the solution to the adjoint equation respectively.

A Modified Method of Successive Approximations for Stochastic Recursive Optimal Control Problems

Based on the stochastic maximum principle for the partially coupled forward-backward stochastic control system (FBSCS for short), a modified method of successive approximations (MSA for short) is established for stochastic recursive optimal control problems. The second-order adjoint processes are introduced in the augmented Hamiltonian minimization step since the control domain is not necessarily convex. Thanks to the theory of bounded mean oscillation martingales (BMO martingales for short), we give a delicate proof of the error estimate and then prove the convergence of the modified MSA algorithm. In a special case, we obtain a logarithmic convergence rate. When the control domain is convex and compact, a sufficient condition which makes the control returned from the MSA algorithm be a near-optimal control is given for a class of linear FBSCSs.

Maximum principle for partially observed stochastic recursive optimal control problems involving impulse controls

AbstractThis article focuses on a stochastic recursive optimal control problem involving impulse control under partially observed information and obtains the related maximum principle. Unlike the classical literature, both the regular control (considering the nonconvexity of the domain) and the impulse control (where the domain is convex) are considered in the framework. In virtue of two types of variational methods, spike variation and convex variation, the Pontryagin maximum principle is established. In addition, to demonstrate the validity of the results, this article investigates a differential game problem as an application.

Dynamic programming principle and Hamilton-Jacobi-Bellman equation under nonlinear expectation

In this paper, we study a stochastic recursive optimal control problem in which the value functional is defined by the solution of a backward stochastic differential equation (BSDE) under G-expectation. Under standard assumptions, we establish the comparison theorem for this kind of BSDE and give a novel and simple method to obtain the dynamic programming principle. Finally, we prove that the value function is the unique viscosity solution to a type of fully nonlinear HJB equation.

Stochastic recursive optimal control problem with mixed delay under viscosity solution's framework

SummaryThis article is concerned with the stochastic recursive optimal control problem with mixed delay. The connection between Pontryagin's maximum principle and Bellman's dynamic programming principle is discussed. Without containing any derivatives of the value function, relations among the adjoint processes and the value function are investigated by employing the notions of super‐ and sub‐jets introduced in defining the viscosity solutions. Stochastic verification theorem is also given to verify whether a given admissible control is really optimal.

Stochastic recursive optimal control problem with obstacle constraint involving diffusion type control

This paper concerns a kind of stochastic optimal control problem with recursive utility described by a reflected backward stochastic differential equation (RBSDE, for short) involving diffusion type control which covers regular control problem, singular control problem and impulse control problem. To begin with, the existence and uniqueness of solution for RBSDEs involving diffusion type control is derived. Then, for the related recursive optimal control problem with obstacle constraint, a sufficient condition to obtain the optimal regular control and diffusion type control is provided. Hence, based on the connection between RBSDE and optimal stopping problem, a class of recursive optimal mixed control problem involving diffusion type control is considered to illustrate our theoretical result, and here the explicit optimal control as well as the stopping time are obtained.

Probabilistic interpretation of HJB equations by the representation theorem for generators of BSDEs

The purpose of this note is to propose a new approach for the probabilistic interpretation of Hamilton-Jacobi-Bellman equations associated with stochastic recursive optimal control problems, utilizing the representation theorem for generators of backward stochastic differential equations. The key idea of our approach for proving this interpretation lies in the identity between solutions and generators given by the representation theorem. Compared with existing methods, our approach seems to be a feasible unified method for different frameworks and be more applicable to general settings. This can also be regarded as a new application of such representation theorem.

Maximum Principle for Stochastic Recursive Optimal Control Problem under Model Uncertainty

In this paper, we consider a stochastic recursive optimal control problem under model uncertainty. In this framework, the cost function is described by solutions of a family of backward stochastic differential equations with uncertainty parameter $\theta$, which is used to represent different market conditions. With the help of linearization techniques and weak convergence methods, we derive the corresponding stochastic maximum principle. Moreover, a linear quadratic robust control problem is also studied.

Stochastic recursive optimal control problem of reflected stochastic differential systems

ABSTRACT In this paper, we study one kind of stochastic recursive optimal control problem in which the control system is stochastic differential equations reflected in a domain and the cost functional is defined by generalised backward stochastic differential equations with reflection. We establish the dynamic programming principle for the value function and show that it is a viscosity solution of the associated obstacle problem for the corresponding Hamilton-Jacobi-Bellman equation with a nonlinear Neumann boundary condition.

The stochastic maximum principle in singular optimal control with recursive utilities

In this paper, we consider stochastic recursive optimal control problem, in which the control variable has two components with the first absolutely continuous and the second singular. The control domain of the first component needs not to be convex. By using a spike variation on the absolutely continuous part of the control and a convex perturbation on the singular one respectively, we obtain a stochastic maximum principle of the optimal control. Also, we give the relationship of the backward variational equation, the adjoint equation and forward variational equation.

A sufficient stochastic maximum principle for a kind of recursive optimal control problem with obstacle constraint

In this paper, we study a kind of recursive optimal control problem whose utility functional is described by the solution of a reflected backward stochastic differential equation (BSDE). We obtain a sufficient stochastic maximum principle of optimal controls. Moreover, a mixed optimal control problem is considered to illustrate the application of our theoretical result and the optimal control and stopping strategy are given.

Near-maximum principle for general recursive utility optimal control problem

ABSTRACTIn this study, we introduce a general recursive utility optimal control problem driven by a fully nonlinear stochastic differential system. In order to investigate Pontryagin's stochastic maximum principle for this new optimal control problem, we establish the near-optimal control system for the original problem and obtain the related near-maximum principle. Two examples are also provided to illustrate the near-maximum principle.

Stochastic global maximum principle for optimization with recursive utilities

In this paper, we study the recursive stochastic optimal control problems. The control domain does not need to be convex, and the generator of the backward stochastic differential equation can contain z. We obtain the variational equations for backward stochastic differential equations, and then obtain the maximum principle which solves completely Peng’s open problem.

Connection between MP and DPP for Stochastic Recursive Optimal Control Problems: Viscosity Solution Framework in the General Case

This paper deals with a stochastic recursive optimal control problem, where the diffusion coefficient depends on the control variable and the control domain is not necessarily convex. We focus on the connection between the general maximum principle and the dynamic programming principle for such a control problem without the assumption that the value is smooth enough; the set inclusions among the sub- and super-jets of the value function and the first-order and second-order adjoint processes as well as the generalized Hamiltonian function are established. Moreover, by comparing these results with the classical ones in Yong and Zhou [Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999], it is natural to obtain the first- and second-order adjoint equations of Hu [Probability, Uncertainty and Quantitative Risk, 2 (2017), 1].

Dynamic programming principle for stochastic recursive optimal control problem driven by a [formula omitted]-Brownian motion

In this paper, we study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related fully nonlinear HJB equation in the framework of G-expectation. Finally, we show that the value function is the viscosity solution of the obtained HJB equation.

Stochastic Maximum Principle for Stochastic Recursive Optimal Control Problem Under Volatility Ambiguity

We study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by $G$-Brownian motion. Some of the economic and financial optimization problems with volatility ambiguity can be formulated as such problems. Unlike the classical variational approach, we establish the maximum principle by the linearization and weak convergence methods.

Maximum principle for forward–backward stochastic control system under [formula omitted]-expectation and relation to dynamic programming

In this paper, based on the theory of stochastic differential equations on a sublinear expectation space (Ω,H,Eˆ), we develop a stochastic maximum principle for a general forward–backward stochastic control system under G-expectation. Under some convexity assumptions, we also obtain sufficient conditions for the optimality. Furthermore, relations between the adjoint processes and the value function for stochastic recursive optimal control problems are given. Finally, applications of our main results to the recursive utility portfolio optimization problem in financial market are discussed.