Dynamic Programming Principle Research Articles

Continuity of the value function for deterministic optimal impulse control with terminal state constraint

Deterministic optimal impulse control problem with terminal state constraint is considered. Due to the appearance of the terminal state constraint, the value function might be discontinuous in general. The main contribution of this paper is the introduction of an intrinsic condition under which the value function is proved to be continuous. Then by a Bellman dynamic programming principle, the corresponding Hamilton-Jacobi-Bellman type quasi-variational inequality (QVI, for short) is derived. The value function is proved to be a viscosity solution to such a QVI. The issue of whether the value function is characterized as the unique viscosity solution to this QVI is carefully addressed and the answer is left open challengingly.

Singular optimal controls for stochastic recursive systems under convex control constraint

In this paper, we study two kinds of singular optimal controls (SOCs for short) problems where the systems governed by forward-backward stochastic differential equations (FBSDEs for short), in which the control has two components: the regular control, and the singular one. Both drift and diffusion terms may involve the regular control variable. The regular control domain is postulated to be convex. Under certain assumptions, in the framework of the Malliavin calculus, we derive the pointwise second-order necessary conditions for stochastic SOC in the classical sense. This condition is described by two adjoint processes, a maximum condition on the Hamiltonian supported by an illustrative example. A new necessary condition for optimal singular control is obtained as well. Besides, as a by-product, a verification theorem for SOCs is derived via viscosity solutions without involving any derivatives of the value functions. It is worth pointing out that this theorem has wider applicability than the restrictive classical verification theorems. Finally, we focus on the connection between the maximum principle and the dynamic programming principle for such SOCs problem without the assumption that the value function is smooth enough.

Representation of limit values for nonexpansive stochastic differential games

A classical problem in ergodic control theory consists in the study of the limit behaviour of λVλ(⋅) as λ↘0, when Vλ is the value function of a deterministic or stochastic control problem with discounted cost functional with infinite time horizon and discount factor λ. We study this problem for the lower value function Vλ of a stochastic differential game with recursive cost, i.e., the cost functional is defined through a backward stochastic differential equation with infinite time horizon. But unlike the ergodic control approach, we are interested in the case where the limit can be a function depending on the initial condition. For this we extend the so-called non-expansivity assumption from the case of control problems to that of stochastic differential games and we derive that λVλ(⋅) is bounded and Lipschitz uniformly with respect to λ>0. Using PDE methods and assuming radial monotonicity of the Hamiltonian of the associated Hamilton-Jacobi-Bellman-Isaacs equation we obtain the monotone convergence of λVλ(.) and we characterize its limit W0 as maximal viscosity subsolution of a limit PDE. Using BSDE methods we prove that W0 satisfies a uniform dynamic programming principle involving the supremum and the infimum with respect to the time, and this is the key for an explicit representation formula for W0.

Relative Performance Concern on DC Pension Plan under Heston Model with Inflation Risk

With the global outbreak of new coronavirus pneumonia, more and more countries have entered the state of sealing off cities. After the epidemic, with the shortage of some materials, the economy is very likely to enter the state of inflation. Thereby, it is necessary and urgent for us to reconsider investment problems involving inflation risk. In this paper, we mainly study the optimal investment strategy of two defined contribution (DC) pension managers with strategy interaction under inflation risk. The traditional portfolio literatures mainly focus on DC pension plan and try to maximize the expected utility of terminal nominal wealth. In this paper, we consider the more complicated situation that pension managers have, both concerns on relative wealth and relative risk aversion. Then, the objective function is constructed to satisfy these two concerns. The dynamic programming principle method is employed to solve the above problems, and a series of analytical solutions to this problem are obtained. Finally, some numerical examples are discussed for the economic implications to support our theoretical results.

Non-anticipative strategies in guarantee optimization problems under functional constraints on disturbances

For a dynamical system controlled under conditions of disturbances, a problem of optimizing the guaranteed result is considered. A feature of the problem is the presence of functional constraints on disturbances, under which, in general, the set of admissible disturbances is not closed with respect to the operation of “gluing up” of two of its elements. This circumstance does not allow to apply directly the methods developed within the differential games theory for studying the problem and, thus, leads to the necessity of modifying them appropriately. The paper provides a new notion of a non-anticipative control strategy. It is proved that the corresponding functional of the optimal guaranteed result satisfies the dynamic programming principle. As a consequence, so-called properties of $u$- and $v$-stability of this functional are established, which may allow, in the future, to obtain a constructive solution of the problem in the form of feedback (positional) controls.

A Continuous-Time Version of a Delegated Asset Management Problem

This paper develops a continuous-time model to study the widely used investment mandates in the institutional asset management industry. In this paper, just like He and Xiong (2013), we suppose that the asset management industry has a two-layered incentive structure, and fund families charging investors fixed management fees while compensating individual fund managers based on fund performance. Different from He and Xiong (2013), we suppose that the fund family aims to select an optimal incentive strategy to maximize its terminal benefits, while the fund manager needs to select the optimal effort level and the optimal investment portfolio to maximize his terminal net discounted compensation in a continuous-time model. By using dynamic programming principle and stochastic differential game theory, the optimal strategies and value functions of both sides are derived. At last, numerical studies are provided to illustrate the effects of all the parameters on the optimal strategies. The result reveals that the optimal incentive mechanism will redistribute both the benefit of the fund families and the cost of the fund managers’ effort.

Optimal Parametric Control of Nonlinear Random Vibrating Systems

Abstract Due to the great progresses in the fields of smart structures, especially smart soft materials and structures, the parametric control of nonlinear systems attracts extensive attentions in scientific and industrial communities. This paper devotes to the derivation of the optimal parametric control strategy for nonlinear random vibrating systems, in which the excitations are confined to Gaussian white noises. For a prescribed performance index balancing the control performance and control cost, the stochastic dynamic programming equation with respect to the value function is first derived by the principle of dynamic programming. The optimal feedback control law is established according to the extremum condition. The explicit expression of the value function is determined by approximately expressing as a quadratic function of state variables and by solving the final dynamic programming equation. The application and efficacy of the optimal parametric control are illustrated by a random-excited Duffing oscillator and a dielectric elastomer balloon with random pressure. The numerical results show that the optimal parameter control possesses good effectiveness, high efficiency, and high robustness to excitation intensity, and is superior than the associated optimal bounded parametric control.

Stochastic differential investment and reinsurance games with nonlinear risk processes and VaR constraints

This paper investigates a class of non-zero-sum stochastic differential investment and reinsurance games between two insurance companies. We allow both insurers to purchase a proportional reinsurance contract and invest in risky and risk-free assets. When applying the generalized mean–variance premium principle in determining reinsurance premium, the surplus process becomes quadratic in the retained proportion of the claims. The optimization criterion of each insurer is to maximize the expected utility of the insurer’s terminal performance relative to that of his competitor. In addition, we incorporate dynamic Value-at-Risk (VaR) constraints in the optimization problems of both insurers to satisfy the capital requirements from regulators. The results show that this game problem can be converted to solving a system of nonlinear equations by means of dynamic programming principle and Karush–Kuhn–Tucker (KKT) conditions. Specifically, when both insurers are constant absolute risk aversion (CARA) institutions and the reinsurance premium principle reduces to the expected value principle, we derive the simplified expressions for the Nash equilibrium strategies. Finally, we use some numerical examples to illustrate the effects of several model parameters on the Nash equilibrium strategies under three different scenarios.

Optimal Asset Allocation Subject to Liquidity and Withdrawal Risks

This study investigates the optimal asset allocation of a financial institution subject to liquidity risks and whose customers are free to withdraw their capital-guaranteed financial contracts at any time. Accounting for constraints on the solvency of the institution, we present a general optimization problem and provide a dynamic programming principle for the optimal dynamic investment strategies. Furthermore, we consider an explicit context, including the interest rate and credit intensity fluctuations, and show, by numerical results, that the optimal strategy improves the solvency and the asset returns of the institution compared to the baseline asset allocation.

Dynamic programming for semi-Markov modulated SDEs

ABSTRACT We consider a stochastic optimal control problem with state variable dynamics described by a stochastic differential equation of diffusive type modulated by a semi-Markov process with a finite state space. The time horizon is both deterministic and finite. Within such setup, we provide a detailed proof of the dynamic programming principle and use it to characterize the value function as a viscosity solution of the corresponding Hamilton-Jacobi-Bellman equation. We illustrate our results with an application to Mathematical Finance: the generalization of Merton's optimal consumption-investment problem to financial markets with semi-Markov switching.

Approximate optimal method for controlling the angular motion of a spacecraft as part of an orbital tether system

An approximate optimal method for controlling the movement relative to the center of mass of a spacecraft as part of an orbital tether system is considered. The purpose of the control is to stabilize the spatial angular movement of the spacecraft relative to the direction of the tether connecting it to another spacecraft. If there is no control, the angular movement of the spacecraft relative to the direction of the tether may be unstable. This is primarily due to the disturbances that occur during the formation (deployment) of tether systems, as well as the internal small mass asymmetry of the spacecraft that inevitably occurs during its manufacture. An approximate optimal method for stabilizing the relative motion of a spacecraft on a tether is proposed, based on the Bellman dynamic programming principle and the averaging method. The averaging method is used for approximate solution of the Bellman partial differential equation, which allows in many cases to obtain the regulator’s equations in analytical form. An example of the application of this approach to stabilize the movement of a small spacecraft when deploying an orbital tether system is given.

Lifetime Ruin Under High-Water Mark Fees and Drift Uncertainty

This paper aims to study lifetime ruin minimization problem by considering investment in two hedge funds with high-watermark fees and drift uncertainty. Due to multi-dimensional performance fees that are charged whenever each fund profit exceeds its historical maximum, the value function is expected to be multi-dimensional. New mathematical challenges arise as the standard dimension reduction cannot be applied, and the convexity of the value function and Isaacs condition may not hold in our probability minimization problem with drift uncertainty. We propose to employ the stochastic Perron’s method to characterize the value function as the unique viscosity solution to the associated Hamilton–Jacobi–Bellman (HJB) equation without resorting to the proof of dynamic programming principle. The required comparison principle is also established in our setting to close the loop of stochastic Perron’s method.

General fully coupled FBSDES involving the value function and related nonlocal HJB equations combined with algebraic equations

Recently, Hao and Li [Fully coupled forward-backward SDEs involving the value function. Nonlocal Hamilton–Jacobi–Bellman equations, ESAIM: Control Optim, Calc. Var. 22 (2016) 519–538] studied a new kind of forward-backward stochastic differential equations (FBSDEs), namely the fully coupled FBSDEs involving the value function in the case where the diffusion coefficient [Formula: see text] in forward stochastic differential equations depends on control, but does not depend on [Formula: see text]. In our paper, we generalize their work to the case where [Formula: see text] depends on both control and [Formula: see text], which is called the general fully coupled FBSDEs involving the value function. The existence and uniqueness theorem of this kind of equations under suitable assumptions is proved. After obtaining the dynamic programming principle for the value function [Formula: see text], we prove that the value function [Formula: see text] is the minimum viscosity solution of the related nonlocal Hamilton–Jacobi–Bellman equation combined with an algebraic equation.

Optimal investment and reinsurance policies for an insurer with ambiguity aversion

In this paper, we study the optimal investment and reinsurance problem for an insurer based on the variance premium principle, in which three cases are considered. First, we assume that the financial market does not exist. The insurer only holds an insurance business, and the optimal reinsurance problem is studied. Subsequently, we assume that there exists a financial market with an accurately modeled risky asset. The optimal investment and reinsurance problem is investigated under these conditions. Finally, we consider the general case in which the insurer is concerned about the model ambiguity of both the insurance market and the financial market. In all three cases, the value function is set to maximize the expected utility of terminal wealth. By employing the dynamic programming principle, we derive the Hamilton–Jacobi–Bellman (HJB) equations, which are satisfied by the value functions and obtain closed-form solutions for optimal reinsurance and investment policies and the value functions in all three cases. Most interestingly, we elucidate how investment improves the insurer’s utility and find that the existence of ambiguity can significantly affect the optimal policies and value functions. We also compare the ambiguities in the two markets and find that ambiguity in the insurance market has much more significant impact on the value function than the ambiguity in the financial market. It implies that it is more valuable for insurer to precisely evaluate the insurance risk. We also provide some numerical examples and economic explanations to illustrate our results.

Optimal control for uncertain discrete-time singular systems under expected value criterion

Optimal control problems governed by two different types of uncertain discrete-time singular systems are investigated under expected value criterion. The objective function including uncertain variables is optimized with the help of expected value method provided that the singular systems are both regular and impulse-free. At first, based on the principle of dynamic programming, a recurrence equation is derived to simplify an optimal control model for a class of uncertain discrete-time singular systems. After that, according to uncertainty theory and the recurrence equation, two kinds of optimal control problems subject to an uncertain linear singular system and an uncertain singular system with quadratic input variables are considered in order, and the optimal solutions are both presented by accurate expressions. A numerical example and a dynamic input-output model are settled to illustrate the effectiveness of the results obtained.

Stochastic recursive optimal control problem with mixed delay under viscosity solution's framework

SummaryThis article is concerned with the stochastic recursive optimal control problem with mixed delay. The connection between Pontryagin's maximum principle and Bellman's dynamic programming principle is discussed. Without containing any derivatives of the value function, relations among the adjoint processes and the value function are investigated by employing the notions of super‐ and sub‐jets introduced in defining the viscosity solutions. Stochastic verification theorem is also given to verify whether a given admissible control is really optimal.

Combining Pontryagin's Principle and Dynamic Programming for Linear and Nonlinear Systems

To study optimal control and disturbance attenuation problems, two prominent-and somewhat alternative-strategies have emerged in the last century: dynamic programming (DP) and Pontryagin's minimum principle (PMP). The former characterizes the solution by shaping the dynamics in a closed loop (a priori unknown) via the selection of a feedback input, at the price, however, of the solution to (typically daunting) partial differential equations. The latter, instead, provides (extended) dynamics that must be satisfied by the optimal process, for which boundary conditions (a priori unknown) should be determined. The results discussed in this article combine the two approaches by matching the corresponding trajectories, i.e., combining the underlying sources of information: knowledge of the complete initial condition from DP and of the optimal dynamics from PMP. The proposed approach provides insights for linear as well as nonlinear systems. In the case of linear systems, the derived conditions lead to matrix algebraic equations, similar to the classic algebraic Riccati equations (AREs), although with coefficients defined as polynomial functions of the input gain matrix, with the property that the coefficient of the quadratic term of such equation is sign definite, even if the corresponding coefficient of the original ARE is sign indefinite, as it is typically the case in the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> control problem. This feature is particularly appealing from the computational point of view, since it permits the use of standard minimization techniques for convex functions, such as the gradient algorithm. In the presence of nonlinear dynamics, the strategy leads to algebraic equations that allow us to (locally) construct the optimal feedback by considering the behavior of the closed-loop dynamics at a single point in the state space.

Optimization of sequential excavation method for large-section urban subway tunnel: A case study

In response to the requirement for population increase, the cross-section area of the urban subway is enlarged, resulting in that one-time excavation technology cannot be directly applied to the excavation of large-section tunnels. Consequently, how to partition the cross-section of the large-section tunnels and optimize the corresponding construction parameters is of great significance. In this paper, we establish a unified planar partition optimization model based on the four parameters of the number of horizontal layers, the number of transverse partitions, the height of the step, and the width of sections. Moreover, using the dynamic programming principle, we can further obtain the optimal excavation sequence and the construction parameters of the large-section tunnels by solving the planar partition optimization model. Combined with the case of an extra-large cross-section tunnel excavation of Chongqing Metro Central Park East Station, the paper optimizes the excavation method of the tunnel with the aim of the maximum construction efficiency and tunnel stability to obtain the optimal excavation sequence, the optimal construction parameters, and the optimal comprehensive evaluation index. The practice has proved that the optimization model based on the dynamic programming principle can effectively solve the problem of large-section tunnel construction. The case analysis can provide an effective reference for similar large-section tunnel projects.

Existence and Uniqueness of Viscosity Solutions for Nonlinear Variational Inequalities Associated with Mixed Control

The author investigates the nonlinear parabolic variational inequality derived from the mixed stochastic control problem on finite horizon. Supposing that some sufficiently smooth conditions hold, by the dynamic programming principle, the author builds the Hamilton-Jacobi-Bellman (HJB for short) variational inequality for the value function. The author also proves that the value function is the unique viscosity solution of the HJB variational inequality and gives an application to the quasi-variational inequality.

A Reinsurance and Investment Game between Two Insurers under the CEV Model

In this paper, the problem of nonzero-sum stochastic differential game between two competing insurance companies is considered, i.e., the relative performance concerns. A certain proportion of reinsurance can be taken out by each insurer to control his own risk. Moreover, each insurer can invest in a risk-free asset and risk asset with the price dramatically following the constant elasticity of variance (CEV) model. Based on the principle of dynamic programming, a general framework regarding Nash equilibrium for nonzero-sum games is established. For the typical case of exponential utilization, we, respectively, give the explicit solutions of the equilibrium strategy as well as the equilibrium function. Some numerical studies are provided at last which assist in obtaining some economic explanations.