Approximate value iteration for a class of deterministic optimal control problems with infinite state and input alphabets

Abstract

We consider discrete-time deterministic optimal control problems in which termination at some finite, but not predetermined nor a-priori bounded time is mandatory. We characterize the value function as the maximal fixed point of a suitable dynamic programming operator and establish the convergence of both exact and approximate value iteration under very general assumptions, which cover the classical deterministic shortest path problem and its extension to hypergraphs as well as reachability and minimum-time problems for sampled versions of continuous control systems under constraints. In particular, the state and input alphabets are infinite sets or metric spaces, and the plant dynamics may be nonlinear and subject to disturbances and constraints. The optimization is in the minimax (or maximin) sense, the additive, extended real-valued running and terminal costs are undiscounted and may be unbounded and of arbitrary signs, the value function is typically discontinuous, and our results do apply to the maximization of non-negative rewards under hard constraints.