The Value Function as a Decision Support Tool in Unmanned Vehicle Operations

Abstract

General problems of optimal trajectory generation and of optimal space-time rendezvous for autonomous underwater vehicles affected by time-varying fluid flows are formulated and solved in the framework of dynamic programming. The optimal solutions include optimal trajectories, as well as departure times and positions.The approach consists in using the principle of optimality (PO) to embed, for example, an optimal time to reach a target problem from some fixed position and time into a more general problem of finding the optimal time to reach a target from any point and time. The solution of this general problem is given by the value function, the solution of a Hamilton-Jacobi-Bellman equation (HJBE) which expresses the PO in an infinitesimal form. The HJBE is solved using an efficient parallel numerical solver.The problems of interest are solved either by minimizing the value function over one or more variables (e.g., time) or by using level sets of the value function to coordinate departure times for multiple vehicles to rendezvous at a given target.The paper presents a description and an illustration of the approach and briefly discusses how value-function-based calculations provide a very effective way to solve complex motion planning and coordination problems. The discussion is aided by examples modeling real operational scenarios using current velocity forecasts from a state-of-the-art model of the Sado river estuary in Portugal.