Deep Reinforcement Learning based dynamic optimization of bus timetable

Abstract

Bus timetable optimization is a key issue to reduce operational cost of bus company and improve the transit service quality. Existing methods optimize the timetable offline. However, in practice, the short-term passenger flow may change dramatically from time to time. Timetables generated offline cannot be adjusted in real time to handle the changed passenger flow. In this paper, we propose a Deep Reinforcement Learning based bus Timetable dynamic Optimization method (DRL-TO). In DRL-TO, the problem of bus timetable optimization is formulated as a Markov Decision Process (MDP). A Deep Q-Network (DQN) is applied as the agent to decide whether a bus departs at each minute during the service period. Therefore, departure intervals of bus services are determined in real time in accordance with passenger demand. We identify several new and useful state features for the DQN agent, including the load factor, the carrying capacity utilization rate, passengers’ waiting time and the number of stranded passengers. Considering the interests of both the bus company and passengers, a reward function is designed, which includes metrics of full load rate, empty load rate, passengers’ waiting time, and the number of stranded passengers. Experiments demonstrate that, in comparison to the timetable generated by offline optimization approaches and the manual method, DRL-TO can dynamically determine the departure intervals based on the real-time passenger flow, and generate a timetable with less departure time points (i.e., operational cost) and shorter passengers’ waiting time (i.e., higher quality of service).