Min-Max Differential Dynamic Programming: Continuous and Discrete Time Formulations

Abstract

In this work, the first min-max Game-Theoretic Differential Dynamic Programming (GT-DDP) algorithm in continuous time is derived. A set of backward differential equations for the value function is provided, along with its first- and second-order derivatives without assuming proximity of the initial nominal controls to the optimal controls. The corresponding update laws for both the minimizing and maximizing controls are also derived. For comparison, the derivation of the GT-DDP algorithm in discrete time is presented in order to better elucidate the differences between the continuous and discrete time formulations. The effect of the game-theoretic formulation in the feed-forward and feedback parts of the optimal control policies is analyzed, and the discrete and continuous time GT-DDP algorithms are compared through numerical examples. Experimental results using a quadrotor demonstrate the superiority of GT-DDP in handling model uncertainties and external disturbances over the standard DDP algorithm.