Intelligent Train Operation Algorithms for Subway by Expert System and Reinforcement Learning

Abstract

Current research in automatic train operation concentrates on optimizing an energy-efficient speed profile and designing control algorithms to track the speed profile, which may reduce the comfort of passengers and impair the intelligence of train operation. Different from previous studies, this paper presents two intelligent train operation (ITO) algorithms without using precise train model information and offline optimized speed profiles. The first algorithm, i.e., ITOe, is based on an expert system that contains expert rules and a heuristic expert inference method. Then, in order to minimize the energy consumption of train operation online, an ITOr algorithm based on reinforcement learning (RL) is developed via designing an RL policy, reward, and value function. In addition, from the field data in the Yizhuang Line of the Beijing Subway, we choose the manual driving data with the best performance as ITOm. Finally, we present some numerical examples to test the ITO algorithms on the simulation platform established with actual data. The results indicate that, compared with ITOm, both ITOe and ITOr can improve punctuality and reduce energy consumption on the basis of ensuring passenger comfort. Moreover, ITOr can save about 10% energy consumption more than ITOe. In addition, ITOr is capable of adjusting the trip time dynamically, even in the case of accidents.