A Deep Reinforcement Learning Approach for the Energy-Aimed Train Timetable Rescheduling Problem Under Disturbances

Abstract

At metro stations with very large passenger volumes and severe congestion, random disturbances often occur, resulting in the original offline optimal timetable no longer applicable. Deep reinforcement learning has the advantages of self-learning and online learning, making it possible to solve the energy-aimed train timetable rescheduling (ETTR) problem under random disturbances. In this article, a deep reinforcement learning approach (DRLA) has been proposed and applied to the ETTR problem to reschedule the train timetable in order to achieve the online optimal timetable for the minimum energy usage under disturbances. The proposed DRLA has the advantages of a real-time performance, high learning efficiency, and better energy-saving effect compared with the traditional heuristic algorithms such as genetic algorithm (GA), deep learning methods such as the combination of improved GA and long short-term memory (IGSA-LSTM), or other reinforcement learning algorithms such as deep deterministic policy gradient (DDPG) algorithm. Various experiments are conducted to compare the performance of DRLA and other algorithms. The experimental results indicate that in the two-train metro system and the five-train metro system, DRLA can save an average of energy by 5.11% and 7.29% with an average reaction time of only 0.0013 and 0.0019 s against disturbances, respectively.