On the Convergence Rate of the Leake-Liu Algorithm for Solving Hamilton-Jacobi-Bellman Equation

Abstract

AbstractAlthough many iterative algorithms have been proposed for solving Hamilton-Jacobi-Bellman equation arising from nonlinear optimal control, it remains open how fast those algorithms can converge. The convergence rate of those algorithms is of great importance in concluding whether they offer practical benefit. This paper presents a study on how fast the well-known Leake-Liu algorithm in Leake and Liu (1967) can converge. The relationship between the sequence of approximate solutions to the HLB equation and the corresponding sequence of control laws is first established. Based on this relation, several results are provided on the convergence rate of the Leake-Liu algorithm. These results demonstrate that the convergence rate of the Leake-Liu algorithm can be quadratic in the domain of interest under favorable conditions. Further, they include the well known quadratic convergence results in Kleinman (1968) and in Reid (1972) as special cases, which have been established for linear time-invariant and time-varying systems (with quadratic performance index) respectively. Under weaker conditions, the convergence rate of the Leake-Liu algorithm is shown to be quadratic locally i.e. in the neighborhood of the origin.