Rescheduling of high-speed trains: a reinforcement learning approach

Abstract

With the construction of high-speed railway networks in China, the traffic density is constantly increasing, and the complexity and difficulty of train rescheduling sharply increase when trains are delayed due to emergencies. How to dynamically adjust the train group operation to reduce the delay and improve the punctuality rate is the core of operation adjustments. In this paper, a model-free reinforcement learning method is proposed for the operation adjustment of high-speed trains in emergency situations. Firstly, the operation adjustment of multiple trains in multiple stations and blocks is modeled as a sequential multi-stage decision-making process of constrained resource occupation and allocation, and a dynamic spatio-temporal topological matrix is proposed to model the stations and blocks. In view of the strong spatio-temporal correlation of high-speed railway trains, a reinforcement learning state space, action space and reward function containing spatio-temporal distribution information such as vehicle positions and network resources are proposed for the first time, and an effective reward feedback mechanism is constructed. Then, aiming at the difficulty of huge search space in high-speed railway operation systems, this paper proposes an adaptive generation method of heuristic action subspace. This method uses some explicit static constraints to construct heuristic rules to reduce the search space, which can effectively reduce the trial-and-error times of model-free reinforcement learning, improve the efficiency of solution, and retain the advantages of model-free generality. Finally, the simulation analysis of the case based on the actual data of Beijing-Guangzhou high-speed railway shows that the proposed algorithm can converge well and significantly reduce the delay propagation within the train group under multiple delays caused by the high wind speed limit in different space and time ranges. Compared with MILP, ACO, and FCFS algorithms, the proposed method can reduce the average delay time of the train group by 2%–20%.