Enhancement of the Global Convergence of Using Iterative Dynamic Programming To Solve Optimal Control Problems

Abstract

Although iterative dynamic programming (IDP) has been well recognized as a powerful method for finding the true optimal solution to a nonlinear dynamic optimization problem, the global optimality of the IDP solution is still not guaranteed completely. This paper explores enhancing the global convergence of using iterative dynamic programming to solve optimal control problems. Two approaches are employed to enhance the possibility of obtaining the true optimum while reducing the required computation efforts. One approach employs Sobol's quasi-random sequence generator to generate allowable controls and the other utilizes multipass computation. Numerical examples show that the use of multipass IDP computation with small numbers of state grid points and allowable control values can indeed enhance the possibility of obtaining a true optimum. This is particularly true when the allowable control values are generated by using Sobol's quasi-random sequence generator.