Implementation of Dynamic Programming for Optimal Control Problems With Continuous States

Abstract

Dynamic programming is a numerical method to solve a dynamic optimal control problem. Due to its numerical framework, it is very suitable to describe discrete dynamics, nonlinear characteristics, and nonconvex constraints. The implementation of continuous states in the discrete framework, however, may lead to optimization inaccuracies. This brief addresses implementation methods with fundamentally different utilizations of the nodes in the quantized time-state space. A new implementation method is presented, which combines the advantages of numerical and analytical optimization techniques to substantially improve the optimization accuracy for a given quantization of the continuous state. If desired, the computation time can be substantially reduced for a given accuracy by lowering the quantization resolution. As a case study, the optimal energy controller is computed for a mechanical hybrid powertrain, which is characterized by switched dynamics, active state constraints, and nonconvex control constraints. Results show that the optimization accuracy of the new method is superior to that of the conventional method based on nearest neighbor rounding. For a given desired accuracy, the computation time is reduced by an order of magnitude.