Dynamic programming and the Hamilton-Jacobi method of classical mechanics

Abstract

The conventional dynamic programming method for analytically solving a variational problem requires the determination of a particular solution, the optimal value function or return function, of the fundamental partial differential equation. Associated with it is another function, the optimal policy function. At each point, this function yields the value of the slope of the optimal curve to that point (or from that point, depending on the method of solution). The optimal curve itself can then be found by integration. In this paper, dynamic programming concepts and principles are used to develop two alternatives to the conventional method of solution. In the first method, a particular solution of two simultaneous partial differential equations is used to generate optimal curves by differentiations and solution of simultaneous equations. In the second method, any solution of the fundamental equation containing an appropriate number of arbitrary constants is sought. It is shown how such a function yields directly, by differentiations and solution of simultaneous equations, the optimal curve for a given problem. While the derivations to follow are new, the results are equivalent to those of a method due to Hamilton and its modification due to Jacobi.