Hamilton Jacobi Bellman Isaacs Equations Research Articles

Optimal harvesting policy for biological resources with uncertain heterogeneity for application in fisheries management

AbstractConventional harvesting problems for natural resources often assume physiological homogeneity of the body length/weight among individuals. However, such assumptions generally are not valid in real‐world problems, where heterogeneity plays an essential role in the planning of biological resource harvesting. Furthermore, it is difficult to observe heterogeneity directly from the available data. This paper presents a novel optimal control framework for the cost‐efficient harvesting of biological resources for application in fisheries management. The heterogeneity is incorporated into the resource dynamics, which is the population dynamics in this case, through a probability density that can be distorted from reality. Subsequently, the distortion, which is the model uncertainty, is penalized through a divergence, leading to a nonstandard dynamic differential game wherein the Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation has a unique nonlinear partial differential term. Here, the existence and uniqueness results of the HJBI equation are presented along with an explicit monotone finite difference method. Finally, the proposed optimal control is applied to a harvesting problem with recreationally, economically, and ecologically important fish species using collected field data.

Optimal investment and reinsurance policies for the Cramér–Lundberg risk model under monotone mean-variance preference

In this paper, an optimisation problem for the monotone mean-variance (MMV) criterion is considered from the perspective of the insurance company. The MMV criterion is an amended version of the classical mean-variance (MV) criterion which guarantees the monotonicity of the utility function. With this criterion we study the optimal investment and reinsurance problem which is formulated as a zero-sum game between the insurance company and an imaginary player. We apply the dynamic programming principle to obtain the corresponding Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation. As the main conclusion of this paper, by solving the HJBI equation explicitly, the closed forms of the optimal strategy and the value function are obtained. Moreover, the MMV efficient frontier is also provided. At the end of the paper, a numerical example is presented.

Stochastic differential games with random coefficients and stochastic Hamilton–Jacobi–Bellman–Isaacs equations

In this paper, we study a class of zero-sum two-player stochastic differential games with the controlled stochastic differential equations and the payoff/cost functionals of recursive type. As opposed to the pioneering work by Fleming and Souganidis [Indiana Univ. Math. J. 38 (1989) 293–314] and the seminal work by Buckdahn and Li [SIAM J. Control Optim. 47 (2008) 444–475], the involved coefficients may be random, going beyond the Markovian framework and leading to the random upper and lower value functions. We first prove the dynamic programming principle for the game, and then under the standard Lipschitz continuity assumptions on the coefficients, the upper and lower value functions are shown to be the viscosity solutions of the upper and the lower fully nonlinear stochastic Hamilton–Jacobi–Bellman–Isaacs (HJBI) equations, respectively. A stability property of viscosity solutions is also proved. Under certain additional regularity assumptions on the diffusion coefficient, the uniqueness of the viscosity solution is addressed as well.

Optimal sizing of the sediment replenishment capacity based on robust ergodic control of subordinator-driven dynamics

A countermeasure against sediment depletion in rivers is sand replenishment. The cost-efficient sizing of its capacity is critical for sustainable environmental management. However, only temporally discrete replenishment is possible, and sediment transport is intermittent and uncertain. In this paper, we approach this new design problem using a robust ergodic control. First, a stochastic differential equation driven by a (tempered) stable subordinator describes the sediment storage dynamics. Sand replenishment is considered an impulse control, wherein observation/replenishment chances arrive at Poisson jump times. The subordinator is ambiguous since the decision-maker does not precisely know its Lévy measure. Using a dynamic risk measure that balances replenishment cost, disutility triggered by sediment depletion, and penalization of ambiguity, we derived the Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation to determine the optimal storage capacity. We obtained that the HJBI equation has an analytical solution when a stable subordinator is used. Furthermore, we prove it admits a unique continuous viscosity solution and that it is optimal and applied the proposed model to real data.

Hamilton–Jacobi–Bellman–Isaacs equation for rational inattention in the long-run management of river environments under uncertainty

A new stochastic control model for the long-run environmental management of rivers is mathematically and numerically analyzed, focusing on a modern sediment replenishment problem with unique nonsmooth and nonlinear properties. Rational inattention as a novel adaptive strategy to collect information and intervene against the target system is modeled using Erlangization. The system dynamics containing the river discharge following a continuous-state branching with an immigration-type process and the controlled sediment storage dynamics lead to a nonsmooth and nonlocal infinitesimal generator. Modeling uncertainty, which is ubiquitous in certain applications, is considered in a robust control framework in which deviations between the benchmark and distorted models are penalized through relative entropy. The partial integro-differential Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation as an optimality equation is derived, and its uniqueness, existence, and optimality are discussed. A monotone finite difference scheme guaranteeing the boundedness and uniqueness of numerical solutions is proposed to discretize the HJBI equation and is verified based on manufactured solutions. Model applications are also conducted with the parameter values identified from the available data and physical formulae. The computational results suggest that environmental management should be rationally inattentive in a state-dependent and adaptive manner.

Discontinuous Galerkin and C0-IP finite element approximation of periodic Hamilton–Jacobi–Bellman–Isaacs problems with application to numerical homogenization

In the first part of the paper, we study the discontinuous Galerkin (DG) and C0 interior penalty (C0-IP) finite element approximation of the periodic strong solution to the fully nonlinear second-order Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation with coefficients satisfying the Cordes condition. We prove well-posedness and perform abstract a posteriori and a priori analyses which apply to a wide family of numerical schemes. These periodic problems arise as the corrector problems in the homogenization of HJBI equations. The second part of the paper focuses on the numerical approximation to the effective Hamiltonian of ergodic HJBI operators via DG/C0-IP finite element approximations to approximate corrector problems. Finally, we provide numerical experiments demonstrating the performance of the numerical schemes.

Fractional McKean–Vlasov and Hamilton–Jacobi–Bellman–Isaacs Equations

Fractional McKean–Vlasov and Hamilton–Jacobi–Bellman–Isaacs Equations

Robust control in a rough environment

This paper studies robust decision problems in a rough environment that is described by Volterra processes. Various alternative dominated models are introduced to reflect decision makers' concerns regarding model uncertainty. Using a functional Itô calculus approach, we characterize the robust optimal strategy by a path-dependent Hamilton–Jacobi–Bellman–Isaacs equation. Explicit strategies are derived for robust power and exponential utility maximization with the Volterra Heston model and an example from a robust linear-quadratic control problem. Numerical study shows that it becomes harder to distinguish two probability measures in a rougher environment. Robust optimal investment strategies can reduce losses from ignoring volatility roughness and model uncertainty and can improve the stability of portfolio performance.

Robust consumption portfolio optimization with stochastic differential utility

This paper examines a continuous time intertemporal consumption and portfolio choice problem with a stochastic differential utility preference of Epstein–Zin type for a robust investor, who worries about model misspecification and seeks robust decision rules. We provide a verification theorem which formulates the Hamilton–Jacobi–Bellman–Isaacs equation under a non-Lipschitz condition. Then, with the verification theorem, the explicit closed-form optimal robust consumption and portfolio solutions to a Heston model are given. Also we compare our robust solutions with the non-robust ones, and the comparisons shown in a few figures coincide with our common sense.

Differential Games for Fractional-Order Systems: Hamilton–Jacobi–Bellman–Isaacs Equation and Optimal Feedback Strategies

The paper deals with a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order α∈(0,1) and a Bolza-type cost functional. A relationship between the differential game and the Cauchy problem for the corresponding Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant derivatives of the order α and the natural boundary condition is established. An emphasis is given to construction of optimal positional (feedback) strategies of the players. First, a smooth case is studied when the considered Cauchy problem is assumed to have a sufficiently smooth solution. After that, to cope with a general non-smooth case, a generalized minimax solution of this problem is involved.

A BSDE Approach to Stochastic Differential Games with Regime Switching

In this paper, we study a two-player zero-sum stochastic differential game with regime switching in the framework of forward-backward stochastic differential equations on a finite time horizon. By means of backward stochastic differential equation methods, in particular that of the notion from stochastic backward semigroups, we prove a dynamic programming principle for both the upper and the lower value functions of the game. Based on the dynamic programming principle, the upper and the lower value functions are shown to be the unique viscosity solutions of the associated upper and lower Hamilton–Jacobi–Bellman–Isaacs equations.

Stochastic differential games for optimal investment problems in a Markov regime-switching jump-diffusion market

We apply dynamic programming principle to discuss two optimal investment problems by using zero-sum and nonzero-sum stochastic game approaches in a continuous-time Markov regime-switching environment within the frame work of behavioral finance. We represent different states of an economy and, consequently, investors’ floating levels of psychological reactions by a D-state Markov chain. The first application is a zero-sum game between an investor and the market, and the second one formulates a nonzero-sum stochastic differential portfolio game as the sensitivity of two investors’ terminal gains. We derive regime-switching Hamilton–Jacobi–Bellman–Isaacs equations and obtain explicit optimal portfolio strategies with Feynman–Kac representations of value functions. We illustrate our results in a two-state special case and observe the impact of regime switches by comparative results.

G-expected utility maximization with ambiguous equicorrelation

This paper studies a class of expected utility maximization problems with respect to a controlled state process with multiple noises, whose pairwise correlations are equal and ambiguous. Using the G-expectation theory, we solve for the robust stochastic controls explicitly from a Hamilton–Jacobi–Bellman–Isaacs equation and deduce a robust choice of the equicorrelation coefficient. We also generalize the results to a block equicorrelation structure, where we consider more than two ambiguous parameters that could be interactive in general. We manage to derive an analytical solution to the robust stochastic controls under an ambiguous two-block equicorrelated structure via the solution to a system of polynomial equations. The results have significant implications for the investment and reinsurance problems among many others.

Stochastic control of single-species population dynamics model subject to jump ambiguity

A logistic type stochastic control model for cost-effective single-species population management subject to an ambiguous jump intensity is presented based on the modern multiplier-robust formulation. The Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation for finding the optimal control is then derived. Mathematical analysis of the HJBI equation from the viewpoint of viscosity solutions is carried out with an emphasis on the non-linear and non-local term, which is a key term arising due to the jump ambiguity. We show that this term can be efficiently handled in the framework of viscosity solutions by utilizing its monotonicity property. A numerical scheme to discretize the HJBI equation is presented as well. Our model is finally applied to management of algae population in river environment. Optimal management policies ranging from the short-term to long-term viewpoints are numerically investigated.

Robust Utility Maximization Under Model Uncertainty via a Penalization Approach

This paper addresses the problem of utility maximization under uncertain parameters. In contrast with the classical approach, where the parameters of the model evolve freely within a given range, we constrain them via a penalty function. We show that this robust optimization process can be interpreted as a two-player zero-sum stochastic differential game. We prove that the value function satisfies the Dynamic Programming Principle and that it is the unique viscosity solution of an associated Hamilton–Jacobi–Bellman–Isaacs equation. We test this robust algorithm on real market data. The results show that robust portfolios generally have higher expected utilities and are more stable under strong market downturns. To solve for the value function, we derive an analytical solution in the logarithmic utility case and obtain accurate numerical approximations in the general case by three methods: finite difference method, Monte Carlo simulation, and Generative Adversarial Networks.

Continuous-Time Portfolio Choice Under Monotone Mean-Variance Preferences—Stochastic Factor Case

We consider an incomplete market with a nontradable stochastic factor and a continuous-time investment problem with an optimality criterion based on monotone mean-variance preferences. We formulate it as a stochastic differential game problem and use Hamilton–Jacobi–Bellman–Isaacs equations to find an optimal investment strategy and the value function. What is more, we show that our solution is also optimal for the classical Markowitz problem, and every optimal solution for the classical Markowitz problem is optimal also for the monotone mean-variance preferences. These results are interesting because the original Markowitz functional is not monotone, and it was observed that in the case of a static one-period optimization problem, the solutions for those two functionals (monotone mean variance and classical Markowitz) are different. In addition, we determine explicit Markowitz strategies in the square root factor models.

Stability Properties of the Value Function in an Infinite Horizon Optimal Control Problem

Properties of the value function are examined in an infinite horizon optimal control problem with an integrand index appearing in the quality functional with a discount factor. The properties are analyzed in the case when the payoff functional of the control system includes a quality index represented by an unbounded function. An upper estimate is given for the growth rate of the value function. Necessary and sufficient conditions are obtained to ensure that the value function satisfies the infinitesimal stability properties. The question of coincidence of the value function with the generalized minimax solution of the Hamilton–Jacobi–Bellman–Isaacs equation is discussed. The uniqueness of the corresponding minimax solution is shown. The growth asymptotic behavior of the value function is described for the logarithmic, power, and exponential quality functionals, which arise in economic and financial modeling. The obtained results can be used to construct grid approximation methods for the value function as the generalized minimax solution of the Hamilton–Jacobi–Bellman–Isaacs equation. These methods are effective tools in the modeling of economic growth processes.

Robust time-inconsistent stochastic control problems

This paper establishes a general analytical framework for continuous-time stochastic control problems for an ambiguity-averse agent (AAA) with time-inconsistent preference, where the control problems do not satisfy Bellman’s principle of optimality. The AAA is concerned about model uncertainty in the sense that she is not completely confident in the reference model of the controlled Markov state process and rather considers some similar alternative models. The problems of interest are studied within a set of dominated models and the AAA seeks for an optimal decision that is robust with respect to model risks. We adopt a game-theoretic framework and the concept of subgame perfect Nash equilibrium to derive an extended dynamic programming equation and extended Hamilton–Jacobi–Bellman–Isaacs equations for characterizing the robust dynamically optimal control of the problem. We also prove a verification theorem to theoretically support our construction of robust control. To illustrate the tractability of the proposed framework, we study an example of robust dynamic mean–variance portfolio selection under two cases: 1. constant risk aversion; and 2. state-dependent risk aversion.

Iterative computational approach to the solution of the Hamilton-Jacobi-Bellman-Isaacs equation in nonlinear optimal control

In this paper, iterative or successive approximation methods for the Hamilton-Jacobi-Bellman-Isaacs equations (HJBIEs) arising in both deterministic and stochastic optimal control for affine nonlinear systems are developed. Convergence of the methods are established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the methods. However, the results presented in the paper are preliminary, and do not yet imply in anyway that the solutions computed will be stabilizing. More improvements and experimentation will be required before a satisfactory algorithm is developed.

An iterative relaxation approach to the solution of the Hamilton-Jacobi-Bellman-Isaacs equation in nonlinear optimal control

In this paper, we propose an iterative relaxation method for solving the Hamilton-Jacobi-Bellman-Isaacs equation (HJBIE) arising in deterministic optimal control of affine nonlinear systems. Local convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the method. An extension of the approach to Lyapunov equations is also discussed. The preliminary results presented are promising, and it is hoped that the approach will ultimately develop into an efficient computational tool for solving the HJBIEs.