Optimal Train Control by Approximate Dynamic Programming: Comparison of Three Value Function Approximation Methods

Abstract

Due to the exponential growth of states and variables, traditional exact dynamic programming suffers from the curse of dimensionality in computing the optimal train control strategy. To address this problem, this paper first proposes a complete discrete model for depicting train control process, and the optimal train control problem is reformulated into a Markov decision process through defining state variables with three dimensionalities. To enhance the computational efficiency of dynamic programming, we design three value function approximation methods to estimate the optimal value functions, which are rollout algorithm, interpolation method and neural network with back propagation, respectively. In particular, the rollout algorithm uses one step forward prediction structure to generate the optimal train control law, while the interpolation method employs a lattice partitioning process for every stage in dynamic programming. The simulation experiments on Beijing Subway show that, 1) rollout algorithm could achieve the best performance compared with the other two algorithms in computing the approximate optimal control strategies, and 2) a simple neural network approximation can not always achieve a solid performance compared with other algorithms.