Isaacs Equation Research Articles

Stochastic Control of Optimized Certainty Equivalents

Optimized certainty equivalents (OCEs) are a family of risk measures widely used by both practitioners and academics. This is mostly due to its tractability and the fact that it encompasses important examples, including entropic risk measures and average value-at-risk. In this work we consider stochastic optimal control problems where the objective criterion is given by an OCE risk measure or, in other words, a risk minimization problem for controlled diffusions. A major difficulty arises since OCEs are often time-inconsistent. Nevertheless, via an enlargement of state space we achieve a substitute of sorts for time-consistency in fair generality. This allows us to derive a dynamic programming principle and thus recover central results of (risk-neutral) stochastic control theory. In particular, we show that the value of our risk minimization problem can be characterized as a viscosity solution of a Hamilton--Jacobi--Bellman--Isaacs equation. We further establish a comparison principle and uniqueness of the latter under suitable technical conditions.

Optimal sizing of the sediment replenishment capacity based on robust ergodic control of subordinator-driven dynamics

A countermeasure against sediment depletion in rivers is sand replenishment. The cost-efficient sizing of its capacity is critical for sustainable environmental management. However, only temporally discrete replenishment is possible, and sediment transport is intermittent and uncertain. In this paper, we approach this new design problem using a robust ergodic control. First, a stochastic differential equation driven by a (tempered) stable subordinator describes the sediment storage dynamics. Sand replenishment is considered an impulse control, wherein observation/replenishment chances arrive at Poisson jump times. The subordinator is ambiguous since the decision-maker does not precisely know its Lévy measure. Using a dynamic risk measure that balances replenishment cost, disutility triggered by sediment depletion, and penalization of ambiguity, we derived the Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation to determine the optimal storage capacity. We obtained that the HJBI equation has an analytical solution when a stable subordinator is used. Furthermore, we prove it admits a unique continuous viscosity solution and that it is optimal and applied the proposed model to real data.

State and Control Path-Dependent Stochastic Zero-Sum Differential Games: Viscosity Solutions of Path-Dependent Hamilton–Jacobi–Isaacs Equations

In this paper, we consider the two-player state and control path-dependent stochastic zero-sum differential game. In our problem setup, the state process, which is controlled by the players, is dependent on (current and past) paths of state and control processes of the players. Furthermore, the running cost of the objective functional depends on both state and control paths of the players. We use the notion of non-anticipative strategies to define lower and upper value functionals of the game, where unlike the existing literature, these value functions are dependent on the initial states and control paths of the players. In the first main result of this paper, we prove that the (lower and upper) value functionals satisfy the dynamic programming principle (DPP), for which unlike the existing literature, the Skorohod metric is necessary to maintain the separability of càdlàg (state and control) spaces. We introduce the lower and upper Hamilton–Jacobi–Isaacs (HJI) equations from the DPP, which correspond to the state and control path-dependent nonlinear second-order partial differential equations. In the second main result of this paper, we show that by using the functional Itô calculus, the lower and upper value functionals are viscosity solutions of (lower and upper) state and control path-dependent HJI equations, where the notion of viscosity solutions is defined on a compact κ-Hölder space to use several important estimates and to guarantee the existence of minimum and maximum points between the (lower and upper) value functionals and the test functions. Based on these two main results, we also show that the Isaacs condition and the uniqueness of viscosity solutions imply the existence of the game value. Finally, we prove the uniqueness of classical solutions for the (state path-dependent) HJI equations in the state path-dependent case, where its proof requires establishing an equivalent classical solution structure as well as an appropriate contradiction argument.

The method of limitation of dynamic loads of nonlinear electromechanical systems under state vector robust control

Aim. Development of the method of limitation of dynamic loads of nonlinear electromechanical systems under state vector robust control. Methodology. Limitation of dynamic loads of nonlinear electromechanical systems is carried out using the minimum selector of choosing the minimum value of the control vector from formed with the help of local controllers and with the vector of maximum control values. Calculation of the gain coefficients of nonlinear robust controllers and observers are based on solutions of the Hamilton–Jacob–Isaacs equations. Results. The results of computer simulation of transitional processes of main roll drives of the rolling mill 950 of the Zaporozhye plant «Dnіprospetsstal» with limitation of dynamic loads are given. Originality. For the first time the method of limitation of dynamic loads of nonlinear electromechanical systems under state vector robust control based on minimum selector and nonlinear robust control of state variables which is needed limitation is developed. Practical value. Examples of transitional processes of main roll drives of the rolling mill 950 of the Zaporozhye plant «Dnіprospetsstal» with limitation of dynamic loads are given.

Finite Element Methods for Isotropic Isaacs Equations with Viscosity and Strong Dirichlet Boundary Conditions

We study monotone P1 finite element methods on unstructured meshes for fully non-linear, degenerately parabolic Isaacs equations with isotropic diffusions arising from stochastic game theory and optimal control and show uniform convergence to the viscosity solution. Elliptic projections are used to manage singular behaviour at the boundary and to treat a violation of the consistency conditions from the framework by Barles and Souganidis by the numerical operators. Boundary conditions may be imposed in the viscosity or in the strong sense, or in a combination thereof. The presented monotone numerical method has well-posed finite dimensional systems, which can be solved efficiently with Howard’s method.

Matching-based capture strategies for 3D heterogeneous multiplayer reach-avoid differential games

This paper studies a 3D multiplayer reach-avoid differential game with a goal region and a play region. Multiple pursuers defend the goal region by consecutively capturing multiple evaders in the play region. The players have heterogeneous moving speeds and the pursuers have heterogeneous capture radii. Since this game is hard to analyze directly, we decompose the whole game as many subgames which involve multiple pursuers and only one evader. Then, these subgames are used as a building block for the pursuer–evader matching. First, for multiple pursuers and one evader, we introduce an evasion space (ES) method characterized by a potential function to construct a guaranteed pursuer winning strategy. Then, based on this strategy, we develop conditions to determine whether a pursuit team can guard the goal region against one evader. It is shown that in 3D, if a pursuit team is able to defend the goal region against an evader, then at most three pursuers in the team are necessarily needed. We also compute the value function of the Hamilton–Jacobi–Isaacs (HJI) equation for a special subgame of degree. To capture the maximum number of evaders in the open-loop sense, we formulate a maximum bipartite matching problem with conflict graph (MBMC). We show that the MBMC is NP-hard and design a polynomial-time constant-factor approximation algorithm to solve it. Finally, we propose a receding horizon strategy for the pursuit team where in each horizon an MBMC is solved and the strategies of the pursuers are given. We also extend our results to the case of a bounded convex play region where the evaders escape through an exit. Two numerical examples are provided to demonstrate the results.

On dynamic pricing under model uncertainty

AbstractWe formulate and solve a robust dynamic pricing problem for an ambiguity‐averse agent who faces an uncertain probabilistic law governing the realized demand for a single product. Specifically, the pricing problem is framed as a stochastic game that involves a maximizing player (the “agent”) and a minimizing player (“nature”) who promotes robustness by distorting the agent's beliefs within prescribed limits. Our methodology builds on the commonly used entropic approach in the literature but can be utilized to generate a much more versatile class of uncertainty sets. We derive the optimal pricing strategy and the corresponding value function by applying stochastic dynamic programming and solving a version of the Bellman–Isaacs equation. The usefulness of our framework is illustrated by two special cases. Finally, a carefully designed numerical example exposes the value of model robustness.

Hamilton–Jacobi–Bellman–Isaacs equation for rational inattention in the long-run management of river environments under uncertainty

A new stochastic control model for the long-run environmental management of rivers is mathematically and numerically analyzed, focusing on a modern sediment replenishment problem with unique nonsmooth and nonlinear properties. Rational inattention as a novel adaptive strategy to collect information and intervene against the target system is modeled using Erlangization. The system dynamics containing the river discharge following a continuous-state branching with an immigration-type process and the controlled sediment storage dynamics lead to a nonsmooth and nonlocal infinitesimal generator. Modeling uncertainty, which is ubiquitous in certain applications, is considered in a robust control framework in which deviations between the benchmark and distorted models are penalized through relative entropy. The partial integro-differential Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation as an optimality equation is derived, and its uniqueness, existence, and optimality are discussed. A monotone finite difference scheme guaranteeing the boundedness and uniqueness of numerical solutions is proposed to discretize the HJBI equation and is verified based on manufactured solutions. Model applications are also conducted with the parameter values identified from the available data and physical formulae. The computational results suggest that environmental management should be rationally inattentive in a state-dependent and adaptive manner.

Discontinuous Galerkin and C0-IP finite element approximation of periodic Hamilton–Jacobi–Bellman–Isaacs problems with application to numerical homogenization

In the first part of the paper, we study the discontinuous Galerkin (DG) and C0 interior penalty (C0-IP) finite element approximation of the periodic strong solution to the fully nonlinear second-order Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation with coefficients satisfying the Cordes condition. We prove well-posedness and perform abstract a posteriori and a priori analyses which apply to a wide family of numerical schemes. These periodic problems arise as the corrector problems in the homogenization of HJBI equations. The second part of the paper focuses on the numerical approximation to the effective Hamiltonian of ergodic HJBI operators via DG/C0-IP finite element approximations to approximate corrector problems. Finally, we provide numerical experiments demonstrating the performance of the numerical schemes.

Geometric regularity theory for a time-dependent Isaacs equation

The purpose of this work is to produce a regularity theory for a class of parabolic Isaacs equations. Our techniques are based on approximation methods which allow us to connect our problem with a Bellman parabolic model. An approximation regime for the coefficients, combined with a smallness condition on the source term unlocks new regularity results in Sobolev and Hölder spaces.

Fractional McKean–Vlasov and Hamilton–Jacobi–Bellman–Isaacs Equations

Fractional McKean–Vlasov and Hamilton–Jacobi–Bellman–Isaacs Equations

Abstract McKean–Vlasov and Hamilton–Jacobi–Bellman Equations, Their Fractional Versions and Related Forward–Backward Systems on Riemannian Manifolds

We introduce a class of abstract nonlinear fractional pseudo-differential equations in Banach spaces that includes both the Mc-Kean-Vlasov-type equations describing nonlinear Markov processes and the Hamilton-Jacobi-Bellman(HJB)-Isaacs equation of stochastic control and games thus allowing for a unified analysis of these equations. This leads to an effective theory of coupled forward-backward systems (forward McKean-Vlasov evolution and backward HJB-Isaacs evolution) that are central to the modern theory of mean-field games.

Guaranteed Deterministic Approach to Superhedging: the Semicontinuity and Continuity Properties of Solutions of the Bellman–Isaacs Equations

Guaranteed Deterministic Approach to Superhedging: the Semicontinuity and Continuity Properties of Solutions of the Bellman–Isaacs Equations

Optimal event‐triggered control for C‐T system with asymmetric constraints based on dual heuristic dynamic programing structure

AbstractIn this article, a method that applies event‐triggered (ET) mechanism control to continuous‐time (C‐T) nonlinear systems with asymmetric constraints based on dual heuristic dynamic programing (DHP) structure is proposed. At first, we derive ET mechanism from traditional time‐triggered and give the Hamilton–Jacobi–Isaacs (HJI) equation. Second, we give the triggering condition and prove the stability of system under ET mechanism. Then, two neural networks (NNs) are introduced, one of which is the critic network, which is designed to approximate the partial derivatives of value function with respect to inputs, and approximate disturbance policy. The other is the action network, which is used to acquire the estimation optimal control policy. Furthermore, we choose a suitable Lyapunov candidate function to prove that the system and NNs weight estimation errors are uniformly ultimately bounded (UUB). Besides, it is important that we prove that Zeno phenomenon can be avoided. Finally, simulation results are shown that the proposed method is feasible.

Robust control in a rough environment

This paper studies robust decision problems in a rough environment that is described by Volterra processes. Various alternative dominated models are introduced to reflect decision makers' concerns regarding model uncertainty. Using a functional Itô calculus approach, we characterize the robust optimal strategy by a path-dependent Hamilton–Jacobi–Bellman–Isaacs equation. Explicit strategies are derived for robust power and exponential utility maximization with the Volterra Heston model and an example from a robust linear-quadratic control problem. Numerical study shows that it becomes harder to distinguish two probability measures in a rougher environment. Robust optimal investment strategies can reduce losses from ignoring volatility roughness and model uncertainty and can improve the stability of portfolio performance.

Observer-based event-triggered control for zero-sum games of input constrained multi-player nonlinear systems

In this paper, an event-triggered control (ETC) method is investigated to solve zero-sum game (ZSG) problems of unknown multi-player continuous-time nonlinear systems with input constraints by using adaptive dynamic programming (ADP). To relax the requirement of system dynamics, a neural network (NN) observer is constructed to identify the dynamics of multi-player system via the input and output data. Then, the event-triggered Hamilton–Jacobi–Isaacs (HJI) equation of the ZSG can be solved by constructing a critic NN, and the approximated optimal control law and the worst disturbance law can be obtained directly. A triggering scheme which determines the updating time instants of the control law and the disturbance law is developed. Thus, the proposed ADP-based ETC method cannot only reduce the computational burden, but also save communication resource and bandwidths. Furthermore, we prove that the signals of the closed-loop system and the approximate errors of the critic NN weights are uniformly ultimately bounded by using Lyapunov’s direct method, and the Zeno behavior is excluded. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed ETC scheme.

Robust consumption portfolio optimization with stochastic differential utility

This paper examines a continuous time intertemporal consumption and portfolio choice problem with a stochastic differential utility preference of Epstein–Zin type for a robust investor, who worries about model misspecification and seeks robust decision rules. We provide a verification theorem which formulates the Hamilton–Jacobi–Bellman–Isaacs equation under a non-Lipschitz condition. Then, with the verification theorem, the explicit closed-form optimal robust consumption and portfolio solutions to a Heston model are given. Also we compare our robust solutions with the non-robust ones, and the comparisons shown in a few figures coincide with our common sense.

Optimal game theoretic solution of the pursuit‐evasion intercept problem using on‐policy reinforcement learning

AbstractThis article presents a rigorous formulation for the pursuit‐evasion (PE) game when velocity constraints are imposed on agents of the game or players. The game is formulated as an infinite‐horizon problem using a non‐quadratic functional, then sufficient conditions are derived to prove capture in a finite‐time. A novel tracking Hamilton–Jacobi–Isaacs (HJI) equation associated with the non‐quadratic value function is employed, which is solved for Nash equilibrium velocity policies for each agent with arbitrary nonlinear dynamics. In contrast to the existing remedies for proof of capture in PE game, the proposed method does not assume players are moving with their maximum velocities and considers the velocity constraints a priori. Attaining the optimal actions requires the solution of HJI equations online and in real‐time. We overcome this problem by presenting the on‐policy iteration of integral reinforcement learning (IRL) technique. The persistence of excitation for IRL to work is satisfied inherently until capture occurs, at which time the game ends. Furthermore, a nonlinear backstepping control method is proposed to track desired optimal velocity trajectories for players with generalized Newtonian dynamics. Simulation results are provided to show the validity of the proposed methods.

Differential Games for Fractional-Order Systems: Hamilton–Jacobi–Bellman–Isaacs Equation and Optimal Feedback Strategies

The paper deals with a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order α∈(0,1) and a Bolza-type cost functional. A relationship between the differential game and the Cauchy problem for the corresponding Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant derivatives of the order α and the natural boundary condition is established. An emphasis is given to construction of optimal positional (feedback) strategies of the players. First, a smooth case is studied when the considered Cauchy problem is assumed to have a sufficiently smooth solution. After that, to cope with a general non-smooth case, a generalized minimax solution of this problem is involved.

Guaranteed Deterministic Approach to Superhedging: Case of Binary European Option

For the superreplication problem with discrete time, a guaranteed deterministic formulation is considered: the problem is to guarantee coverage of the contingent liability on sold option under all admissible scenarios. These scenarios are defined by means of a priori defined compacts dependent on price prehistory: the price increments at each point in time must lie in the corresponding compacts. In a general case, we consider a market with trading constraints and assume the absence of transaction costs. The formulation of the problem is game theoretic and leads to the Bellman–Isaacs equations. This paper analyses the solution to these equations for a specific pricing problem, i.e., for a binary option of the European type, within a multiplicative market model, with no trading constraints. A number of solution properties and an algorithm for the numerical solution of the Bellman equations are derived. The interest in this problem, from a mathematical prospective, is related to the discontinuity of the option payoff function.