Payoff suboptimality and errors in value induced by approximation of the Hamiltonian

Abstract

Dynamic programming reduces the solution of optimal control problems to solution of the corresponding Hamilton-Jacobi-Bellman partial differential equations (HJB PDEs). In the case of nonlinear deterministic systems, the HJB PDEs are fully nonlinear, first-order PDEs. Standard, grid-based techniques to the solution of such PDEs are subject to the curse-of-dimensionality, where the computational costs grow exponentially with state-space dimension. Among the recently developed max-plus methods for solution of such PDEs, there is a curse-of-dimensionality-free algorithm. Such an algorithm can be applied in the case where the Hamiltonian takes the form of a pointwise maximum of a finite number of quadratic forms. In order to take advantage of this curse-of-dimensionality-free algorithm for more general HJB PDEs, we need to approximate the general Hamiltonian by a maximum of these quadratic forms. In doing so, one introduces some errors. In this work, we obtain a bound on the difference in solution of two HJB PDEs, as a function of a bound on the difference in the two Hamiltonians. Further, we obtain a bound on the suboptimality of the controller obtained from the solution of the approximate HJB PDE rather than from the original.