Continuous-time finite-horizon ADP for automated vehicle controller design with high efficiency

Abstract

Automated vehicle controller's design can be formulated into a general optimal control problem. Existing control methods can not meet the millisecond-level time requirements of onboard standard controllers, especially for nonlinear dynamics with non-affine and saturated controller. This paper presents a continuous-time (CT) finite-horizon approximate dynamic programming (ADP) method, which can synthesis off-line near optimal control policy with analytical vehicle dynamics. Firstly, we develop a finite-horizon variant of the CT Hamilton-Jacobi-Bellman (HJB) equation. To find the nearly optimal solution of the finite-horizon HJB, i.e., the nearly optimal policy, both value function and policy are approximated by neural networks to map the system states to cost-to-go and control inputs respectively. Then the CT finite-horizon ADP algorithm is proposed based on the generalized policy iteration framework to gradually approach the optimal solution of the finite-horizon HJB. The proposed algorithm can converge to the nearly optimal policy and value function. Finally, we apply the proposed algorithm to the automated vehicle trajectory tracking control in both linear and nonlinear cases. The simulation results demonstrate that the proposed finite-horizon ADP algorithm can obtain the nearly optimal control policy and high online operation efficiency (500 times faster than the nonlinear ipopt solver).