Multitime dynamic programming for multiple integral actions

Abstract

This paper introduces a new type of dynamic programming PDE for optimal control problems with performance criteria involving multiple integrals. The main novel feature of the multitime dynamic programming PDE, relative to the standard Hamilton-Jacobi-Bellman PDE, is that it is connected to the multitime maximum principle and is of divergence type. Introducing a generating vector field for the maximum value function, we present an interesting and useful connection between the multitime maximum principle and the multitime dynamic programming, characterizing the optimal control by means of a multitime Hamilton-Jacobi-Bellman (divergence) PDE that may be viewed as a feedback law. Section 1 recalls the multitime maximum principle. Section 2 shows how a multitime control dynamics determines the multitime Hamilton-Jacobi-Bellman PDE via a generating vector field of the value function. Section 3 gives an example of two-time dynamics with nine velocities proving that our theory works well. Section 4 shows that the Hamilton PDEs are characteristic PDEs of multitime Hamilton-Jacobi PDE and that the costates in the multitime maximum principle are in fact gradients of the components of the generating vector field.