Dynamic programming and the calculus of variations

Abstract

Problems in the Calculus of Variations can be viewed as multistage decision problems of a continuous type. It follows that their solutions can be characterized by the functional equation technique of dynamic programming [1]. In this paper, it will be shown that the functional equation approach yields, in simple and intuitive fashion, formal derivations of such classical necessary conditions of the Calculus of Variations as the Euler-Lagrange equations, the Weierstrass and Legendre conditions, natural boundary conditions, a transversality condition and the Erdmann corner conditions. The more general “problem of Bolza” in which the final time is defined implicitly and in which the expression to be extremized is the sum of an integral and a function evaluated at the end point is also considered. The principal necessary condition, usually called the “multiplier rule,” is deduced. We shall also derive necessary conditions for the case where the decision variables are restricted by inequality constraints. Finally, it is shown that the functional equation characterization readily yields the Hamilton-Jacobi partial differential equation of classical mechanics.