Hamilton-Jacobi-Bellman Quasi-variational Inequalities Research Articles

A monotone numerical integration method for mean–variance portfolio optimization under jump-diffusion models

We develop a highly efficient, easy-to-implement, and strictly monotone numerical integration method for Mean-Variance (MV) portfolio optimization. This method proves very efficient in realistic contexts, which involve jump-diffusion dynamics of the underlying controlled processes, discrete rebalancing, and the application of investment constraints, namely no-bankruptcy and leverage.A crucial element of the MV portfolio optimization formulation over each rebalancing interval is a convolution integral, which involves a conditional density of the logarithm of the amount invested in the risky asset. Using a known closed-form expression for the Fourier transform of this conditional density, we derive an infinite series representation for the conditional density where each term is strictly positive and explicitly computable. As a result, the convolution integral can be readily approximated through a monotone integration scheme, such as a composite quadrature rule typically available in most programming languages. The computational complexity of our proposed method is an order of magnitude lower than that of existing monotone finite difference methods. To further enhance efficiency, we propose an implementation of this monotone integration scheme via Fast Fourier Transforms, exploiting the Toeplitz matrix structure.The proposed monotone numerical integration scheme is proven to be both ℓ∞-stable and pointwise consistent, and we rigorously establish its pointwise convergence to the unique solution of the MV portfolio optimization problem. We also intuitively demonstrate that, as the rebalancing time interval approaches zero, the proposed scheme converges to a continuously observed impulse control formulation for MV optimization expressed as a Hamilton–Jacobi–Bellman Quasi-Variational Inequality. Numerical results show remarkable agreement with benchmark solutions obtained through finite differences and Monte Carlo simulation, underscoring the effectiveness of our approach.

Error Estimates of Penalty Schemes for Quasi-Variational Inequalities Arising from Impulse Control Problems

This paper proposes penalty schemes for a class of weakly coupled systems of Hamilton--Jacobi--Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. We show that the solutions of the penalized equations converge monotonically to those of the HJBQVIs. We further establish that the scheme is half-order accurate for HJBQVIs with Lipschitz coefficients and first-order accurate for equations with more regular coefficients. Moreover, we construct the action regions and optimal impulse controls based on the error estimates and the penalized solutions. The penalty schemes and convergence results are then extended to HJBQVIs with possibly negative impulse costs. We also demonstrate the convergence of monotone discretizations of the penalized equations and establish that policy iteration applied to the discrete equation is monotonically convergent with an arbitrary initial guess in an infinite-dimensional setting. Numerical examples for infinite-horizon optimal switching problems are presented to illustrate the effectiveness of the penalty schemes over the conventional direct control scheme.

An optimal switching approach toward cost‐effective control of a stand‐alone photovoltaic panel system under stochastic environment

AbstractThe operation of a stand‐alone photovoltaic (PV) system ultimately aims for the optimization of its energy storage. We present a mathematical model for cost‐effective control of a stand‐alone system based on a PV panel equipped with an angle adjustment device. The model is based on viscosity solutions to partial differential equations, which serve as a new and mathematically rigorous tool for modeling, analyzing, and controlling PV systems. We formulate a stochastic optimal switching problem of the panel angle, which is here a binary variable to be dynamically controlled under stochastic weather condition. The stochasticity comes from cloud cover dynamics, which is modeled with a nonlinear stochastic differential equation. In finding the optimal control policy of the panel angle, switching the angle is subject to impulsive cost and reduces to solving a system of Hamilton‐Jacobi‐Bellman quasi‐variational inequalities (HJBQVIs). We show that the stochastic differential equation is well posed and that the HJBQVIs admit a unique viscosity solution. In addition, a finite‐difference scheme is proposed for the numerical discretization of HJBQVIs. A demonstrative computational example of the HJBQVIs, with emphasis on a stand‐alone experimental system, is finally presented with practical implications for its cost‐effective operation.

An exact viscosity solution to a Hamilton\u2013Jacobi\u2013Bellman quasi-variational inequality for animal population management

We formulate a stochastic impulse control model for animal population management and a candidate of exact solutions to a Hamilton–Jacobi–Bellman quasi-variational inequality. This model has a qualitatively different functional form of the performance index from the existing monotone ones. So far, optimality and unique solvability of the Hamilton–Jacobi–Bellman quasi-variational inequality has not been investigated, which are thus addressed in this paper. We present a candidate of exact solutions to the Hamilton–Jacobi–Bellman quasi-variational inequality and prove its optimality and unique solvability within a certain class of solutions in a viscosity sense. We also present and examine a dynamical system-based numerical method for computing coefficients in the exact solutions.

Market making under a weakly consistent limit order book model

AbstractWe develop a new market‐making model, from the ground up, which is tailored toward high‐frequency trading under a limit order book (LOB), based on the well‐known classification of order types in market microstructure. Our flexible framework allows arbitrary order volume, price jump, and bid‐ask spread distributions as well as the use of market orders. It also honors the consistency of price movements upon arrivals of different order types. For example, it is apparent that prices should never go down on buy market orders. In addition, it respects the price‐time priority of LOB. In contrast to the approach of regular control on diffusion as in the classical Avellaneda and Stoikov (Quantitative Finance, 8, 217, 2008) market‐making framework, we exploit the techniques of optimal switching and impulse control on marked point processes, which have proven to be very effective in modeling the order book features. The Hamilton‐Jacobi‐Bellman quasi‐variational inequality (HJBQVI) associated with the control problem can be solved numerically via finite‐difference method. We illustrate our optimal trading strategy with a full numerical analysis, calibrated to the order book statistics of a popular exchanged‐traded fund (ETF). Our simulation shows that the profit of market‐making can be severely overstated under LOBs with inconsistent price movements.

A stochastic differential game approach toward animal migration.

A stochastic differential game model for animal migration between two habitats under uncertain environment, a new population dynamics model, is formulated. Its novelty is the use of an impulse control formalism to naturally describe migrations with different timings and magnitudes that the conventional models could not handle. Uncertainty of the environment that the population faces with is formulated in the context of the multiplier robust control. The optimal migration strategy to give the maximized minimal profit is found through a Hamilton-Jacobi-Bellman quasi-variational inequality (HJBQVI). A key message from HJBQVI is that its free boundary determines the optimal migration strategy. Solving the HJBQVI is carried out with a specialized stable and convergent finite difference scheme. This paper theoretically suggests that the sub-additivity of the performance index, the index to be optimized through the migration, critically affects the resulting strategy. The computational results with the established scheme are consistent with the theoretical predictions and support importance of the sub-additivity property. Social interaction to reduce the net mortality rate is also quantified, suggesting a linkage between the present and existing population dynamics models.

Hamilton–Jacobi–Bellman Quasi-Variational Inequality arising in an environmental problem and its numerical discretization

A Hamilton–Jacobi–Bellman Quasi-Variational Inequality (HJBQVI) for a river environmental restoration problem with wise-use of sediment is formulated and its mathematical properties are analyzed. A finite difference scheme with a penalization technique is then established for solving the HJBQVI. The scheme is free from any iterative solvers and is unconditionally stable and convergent in the viscosity sense under certain conditions. A demonstrative application example of the HJBQVI is finally presented.

Convergence of Implicit Schemes for Hamilton--Jacobi--Bellman Quasi-Variational Inequalities

In [Azimzadeh, P., and P. A. Forsyth. "Weakly chained matrices, policy iteration, and impulse control." SIAM J. Num. Anal. 54.3 (2016): 1341-1364], we outlined the theory and implementation of computational methods for implicit schemes for Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs). No convergence proofs were given therein. This work closes the gap by giving rigorous proofs of convergence. We do so by introducing the notion of nonlocal consistency and appealing to a Barles-Souganidis type analysis. Our results rely only on a well-known comparison principle and are independent of the specific form of the intervention operator.

Optimal Hedge Tracking Portfolios in a Limit Order Book

Derivative hedging under transaction costs has attracted considerable attention over the past three decades. Yet comparatively little effort has been made towards integrating this problem in the context of trading through a limit order book. In this paper, we propose a simple model for a wealth-optimizing option seller, who hedges his position using a combination of limit and market orders, while facing certain constraints as to how far he can deviate from a targeted (Bachelierian) delta strategy. By translating the control problem into a three-dimensional Hamilton–Jacobi–Bellman quasi-variational inequality (HJB QVI) and solving numerically, we are able to deduce optimal limit order quotes alongside the regions surrounding the targeted delta surface in which the option seller must place limit orders vis-à-vis the more aggressive market orders. Our scheme is shown to be monotone, stable, and consistent and thence, modulo a comparison principle, convergent in the viscosity sense.

Weakly Chained Matrices, Policy Iteration, and Impulse Control

This work is motivated by numerical solutions to Hamilton--Jacobi--Bellman quasi-variational inequalities (HJBQVIs) associated with combined stochastic and impulse control problems. In particular, we consider (i) direct control, (ii) penalized, and (iii) semi-Lagrangian discretization schemes applied to the HJBQVI problem. Scheme (i) takes the form of a Bellman problem involving an operator which is not necessarily contractive. We consider the well-posedness of the Bellman problem and give sufficient conditions for convergence of the corresponding policy iteration. To do so, we use weakly chained diagonally dominant matrices, which give a graph-theoretic characterization of weakly diagonally dominant M-matrices. We compare schemes (i)--(iii) under the following examples: (a) optimal control of the exchange rate, (b) optimal consumption with fixed and proportional transaction costs, and (c) pricing guaranteed minimum withdrawal benefits in variable annuities. We find that one should abstain from using scheme (i). (An erratum is attached.)

A dynamic optimal execution strategy under stochastic price recovery

In the present paper, we study the optimal execution problem under stochastic price recovery based on limit order book dynamics. We model price recovery after execution of a large order by accelerating the arrival of the refilling order, which is defined as a Cox process whose intensity increases by the degree of the market impact. We include not only the market order, but also the limit order in our strategy in a restricted fashion. We formulate the problem as a combined stochastic control problem over a finite time horizon. The corresponding Hamilton–Jacobi–Bellman quasi-variational inequality is solved numerically. The optimal strategy obtained consists of three components: (i) the initial large trade; (ii) the unscheduled small trades during the period; (iii) the terminal large trade. The size and timing of the trade is governed by the tolerance for market impact depending on the state at each time step, and hence the strategy behaves dynamically. We also provide competitive results due to inclusion of the limit order, even though a limit order is allowed under conservative evaluation of the execution price.

An implicit method for the finite time horizon Hamilton–Jacobi–Bellman quasi-variational inequalities

We propose a new numerical method for solving the Hamilton–Jacobi–Bellman quasi-variational inequality associated with the combined impulse and stochastic optimal control problem over a finite time horizon. Our method corresponds to an implicit method in the field of numerical methods for partial differential equations, and thus it is advantageous in the sense that the stability condition is independent of the discretization parameters. We apply our method to the finite time horizon optimal forest harvesting problem, which considers exiting from the forestry business at a finite time. We show that the behavior of the obtained optimal harvesting strategy of the extended problem coincides with our intuition.

Existence and uniqueness of viscosity solutions for QVI associated with impulse control of jump-diffusions

General theorems for existence and uniqueness of viscosity solutions for Hamilton–Jacobi–Bellman quasi-variational inequalities (HJBQVI) with integral term are established. Such nonlinear partial integro-differential equations (PIDE) arise in the study of combined impulse and stochastic control for jump-diffusion processes. The HJBQVI consists of an HJB part (for stochastic control) combined with a nonlocal impulse intervention term. Existence results are proved via stochastic means, whereas our uniqueness (comparison) results adapt techniques from viscosity solution theory. This paper, to our knowledge is the first treating rigorously impulse control for jump-diffusion processes in a general viscosity solution framework; the jump part may have infinite activity. In the proofs, no prior continuity of the value function is assumed, quadratic costs are allowed, and elliptic and parabolic results are presented for solutions possibly unbounded at infinity.