Viscosity Solution Approach Research Articles

Viscosity solutions approach to finite-horizon continuous-time Markov decision process

This paper investigates the optimal control problems for the finite-horizon continuous-time Markov decision processes with delay-dependent control policies. We develop compactification methods in decision processes and show that the existence of optimal policies. Subsequently, through the dynamic programming principle of the delay-dependent control policies, the differential-difference Hamilton-Jacobi-Bellman (HJB) equation in the setting of discrete space is established. Under certain conditions, we give the comparison principle and further prove that the value function is the unique viscosity solution to this HJB equation. Based on this, we show that among the class of delay-dependent control policies, there is an optimal one which is Markovian.

Controlled measure-valued martingales: A viscosity solution approach

Controlled measure-valued martingales: A viscosity solution approach

Deterministic differential games in infinite horizon involving continuous and impulse controls

ABSTRACT We study a new class of two-player, zero-sum, deterministic differential games where each player uses both continuous and impulse controls in an infinite horizon with discounted payoff. We assume that the form and cost of impulses depend on nonlinear functions and the state of the system, respectively. We use Bellman's dynamic programming principle (DPP) and viscosity solutions approach to show, for this class of games, the existence and uniqueness of a solution for the associated Hamilton–Jacobi–Bellman–Isaacs (HJBI) partial differential equations (PDEs). We then, under Isaacs' condition, deduce that the lower and upper value functions coincide, and we give a computational procedure with a numerical test for the game.

A viscosity solution approach to regularity properties of the optimal value function

We analyze the optimal value function v associated with a general parametric optimization problem via the theory of viscosity solutions. The novelty is that we obtain regularity properties of v by showing that it is a viscosity solution to a set of first-order equations. As a consequence, in Banach spaces, we provide sufficient conditions for local and global Lipschitz properties of v. We also derive, in finite dimensions, conditions for optimality through a comparison principle. Finally, we study the relationship between viscosity and Clarke generalized solutions to get further differentiability properties of v in Euclidean spaces.

A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation

<p style='text-indent:20px;'>In this paper, we present a feedback design for numerical solution to optimal control problems, which is based on solving the corresponding Hamilton-Jacobi-Bellman (HJB) equation. An upwind finite-difference scheme is adopted to solve the HJB equation under the framework of the dynamic programming viscosity solution (DPVS) approach. Different from the usual existing algorithms, the numerical control function is interpolated in turn to gain the approximation of optimal feedback control-trajectory pair. Five simulations are executed and both of them, without exception, output the accurate numerical results. The design can avoid solving the HJB equation repeatedly, thus efficaciously promote the computation efficiency and save memory.</p>

Infinite-Horizon Optimal Switching Regions for a Pair-Trading Strategy with Quadratic Risk Aversion Considering Simultaneous Multiple Switchings: A Viscosity Solution Approach

Very few studies have explored the structure of optimal switching regimes. We extend the existing research on the infinite-horizon multiple-regime switching problem with an arbitrary number of switch options by replacing the linear running reward function with a quadratic function in the objective function. To make our analysis more rigorous, we establish the theoretical basis for the application of the simultaneous multiple-regime switches to the problem with the extended objective function, and provide the sufficient condition under which each certain separated region in the space includes, at most, one single connected optimal switching region, which determines the structure of the optimal switching regions, and we identify the structure of the optimal switching regions for the particular problem.

Regime-switching constrained viscosity solutions approach for controlling dam–reservoir systems

A new stochastic control problem of a dam–reservoir system installed in a river is analyzed both mathematically and numerically. Water balance dynamics of the reservoir are piece-wise deterministic and are driven by a stochastic regime-switching inflow process. The system is controlled to balance among the operation purpose and the internal and downstream environmental conditions. Finding the optimal operation policy of the system reduces to solving an optimality equation with a discontinuous Hamiltonian, which is a system of nonlinear degenerate parabolic (or hyperbolic) equations. We show that the optimality equation has at most one constrained viscosity solution and find the solution explicitly under certain conditions. The model is applied to numerical computation of the operation policy of an existing dam–reservoir system using a high-order finite difference scheme. The computational results can suggest how the operation policy should be adapted according to the environmental concerns of the river.

Fishery management in a regime switching environment: Utility theory approach

In this paper, we study the problem of optimal fishing for regime switching in the growth dynamics of a given fish species which is described by the differential stochastic logistic model with two states: prior or during floods and after. The resulting dynamic programming principle leads to a system of two variational inequalities. By using viscosity solutions approach, we prove the existence and uniqueness of the value functions. Then numerical approximation of the obtained system is used to determine the optimal fishing effort for a sustainable fishery.

Solving Elliptic Hamilton-Jacobi-Bellman Equations in a Value Space

In this letter, we study the numerical solutions of a class of elliptic type of Hamilton-Jacobi-Bellman (HJB) equations, which arise from stochastic control and multiplayer stochastic game problems with random terminal time. Two closely related algorithms are proposed. One of them is to use value iteration on approximating Markov decision processes, and the other is to use a deep-learning approach solving Bellman equations. The convergence is shown by using a viscosity solution approach.

Stochastic impulse control Problem with state and time dependent cost functions

<p style='text-indent:20px;'>We consider stochastic impulse control problems when the impulses cost functions depend on <inline-formula><tex-math id="M1">\begin{document}$ t $\end{document}</tex-math></inline-formula> and <inline-formula><tex-math id="M2">\begin{document}$ x $\end{document}</tex-math></inline-formula>. We use the approximation scheme and viscosity solutions approach to show that the value function is a unique viscosity solution for the associated Hamilton-Jacobi-Bellman equation (HJB) partial differential equation (PDE) of stochastic impulse control problems.

On mean curvature flow with forcing

This paper investigates geometric properties and well-posedness of a mean curvature flow with volume-dependent forcing. With the class of forcing which bounds the volume of the evolving set away from zero and infinity, we show that a strong version of star-shapedness is preserved over time. More precisely, it is shown that the flow preserves the ρ-reflection property, which corresponds to a quantitative Lipschitz property of the set with respect to the nearest ball. Based on this property we show that the problem is well-posed and its solutions starting with ρ-reflection property become instantly smooth. Lastly, for a model problem, we will discuss the flow’s exponential convergence to the unique equilibrium in Hausdorff topology. For the analysis, we adopt the approach developed by Feldman-Kim to combine viscosity solutions approach and variational method. The main challenge lies in the lack of comparison principle, which accompanies forcing terms that penalize small volume.

Hamilton–Jacobi equations for optimal control on junctions with unbounded running cost functions

ABSTRACT This paper focuses on the viscosity solution approach to optimal control problems on junctions. Compared to Achdou et al. [Hamilton–Jacobi equations for optimal control on junctions and networks: ESAIM Control Optim. ESAIM Control Optim Calc Var. 2015;21(3):876–899] and Khang [Hamilton–Jacobi equations for optimal control on networks with entry or exit costs. ESAIM Control Optim Calc Var. 2018. (In press). EDP Sciences. <hal-01548133v2>], we work on a less restrictive set of assumptions. We show that the value function is a unique viscosity solution of an associated Hamilton–Jacobi equation, and present some further properties of it. In addition, the viscosity solution method is used to establish a necessary and sufficient condition for an optimal control in a class of optimal control problems.

On Uniform Large Deviations Principle for Multi-valued SDEs via the Viscosity Solution Approach

This paper deals with the uniform large deviations for multivalued stochastic differential equations (MSDEs for short) by applying a stability result of the viscosity solutions of second order Hamilton-Jacobi-Belleman equations with multivalued operators. Moreover, the large deviation principle is uniform in time and in starting point.

The viscosity solutions approach to swing options pricing under a regime-switching mean-reverting model

In this paper, we study the valuation of swing options on electricity markets with local volume and refraction time constraints, under the setting that the dynamic of the underlying spot price is a 2-state regime-switching mean-reverting process. We derive the corresponding optimal multiple stopping problem, reduce it to a sequence of optimal single stopping problems, and further find that those value functions satisfy HJB variational inequalities subject to suitable conditions. Then after a prior estimation for the value functions, the viscosity solutions approach is adopted to get existence and uniqueness results in viscosity sense.

The Valuation of American Passport Options: A Viscosity Solution Approach

The passport option, introduced and marketed by Bankers Trust, is a call option on the balance of a trading account. This paper concerns the American passport option. We rigorously establish the mathematical foundation for pricing the American passport option. We derive the pricing equation, using the dynamic programming principle, and prove that the option value is a viscosity solution of variational inequality, which is a fully nonlinear equation. We also establish the comparison principle, which yields uniqueness of the viscosity solution. Moreover, we prove convexity-preserving property for the viscosity solution. In addition, we obtain further properties of the optimal exercise boundary. Finally, we give several numerical examples and financial analysis.

Zero-Sum Stochastic Differential Game in Finite Horizon Involving Impulse Controls

This paper considers the problem of two-player zero-sum stochastic differential game with both players adopting impulse controls in finite horizon under rather weak assumptions on the cost functions ($c$ and $\chi$ not decreasing in time). We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equation (PDE) of the game. We prove that the upper and lower value functions coincide.

Near-Optimal Control of Stochastic Recursive Systems Via Viscosity Solution

In this paper, we study the near-optimal control for systems governed by forward–backward stochastic differential equations via dynamic programming principle. Since the nonsmoothness is inherent in this field, the viscosity solution approach is employed to investigate the relationships among the value function, the adjoint equations along near-optimal trajectories. Unlike the classical case, the definition of viscosity solution contains a perturbation factor, through which the illusory differentiability conditions on the value function are dispensed properly. Moreover, we establish new relationships between variational equations and adjoint equations. As an application, a kind of stochastic recursive near-optimal control problem is given to illustrate our theoretical results.

Uncertain saddle point equilibrium differential games with non-anticipating strategies

In this paper, we investigate uncertain saddle point equilibrium differential games under uncertain environment. We propose an optimistic value game model and define the value function of the game by introducing the concept of non-anticipating strategy. We prove the continuity and dynamic programming property of the value function. Then we derive the uncertain Hamilton–Jacobi–Isaacs equation by the viscosity solution approach.

Uniform convergence for the incompressible limit of a tumor growth model

We study a model introduced by Perthame and Vauchelet [19] that describes the growth of a tumor governed by Brinkman's Law, which takes into account friction between the tumor cells. We adopt the viscosity solution approach to establish an optimal uniform convergence result of the tumor density as well as the pressure in the incompressible limit. The system lacks standard maximum principle, and thus modification of the usual approach is necessary.

Liquidity management with decreasing returns to scale and secured credit line

This paper examines the dividend and investment policies of a cash constrained firm, assuming a decreasing-returns-to-scale technology and adjustment costs. We extend the literature by allowing the firm to draw on a secured credit line both to hedge against cash-flow shortfalls and to invest/disinvest in a productive asset. We formulate this problem as a two-dimensional singular control problem and use both a viscosity solution approach and a verification technique to get qualitative properties of the value function. We further solve quasi-explicitly the control problem in two special cases.