Q_EDQ: Efficient path planning in multimodal travel scenarios based on reinforcement learning

Abstract

Recently, Mobility as a Service (MaaS) has garnered increasing attention by integrating various modes of transportation to provide users with a unified travel solution. However, In multimodal transportation planning, we primarily face three challenges: Firstly, a multimodal travel network is constructed that covers multiple travel modes and is highly scalable. Secondly, the routing algorithm fully considers the dynamic and real-time nature of the multimodal travel process. Finally, a generalized travel cost objective function is constructed that considers the psychological burden of transfers on passengers in multimodal travel scenarios. In this study, we firstly constructed an integrated multimodal transport network based on graph theory, which covers four transport modes, namely, the metro, the bus, the car-sharing and the walking. Subsequently, by introducing a double-Q learning mechanism and an optimized dynamic exploration strategy, we propose a new algorithm, Q_EDQ, the algorithm aims to learn the globally optimal path as efficiently as possible, with faster convergence speed and improved stability. Experiments utilizing real bus and metro data from Xi’an, Shaanxi Province, were conducted to compare the Q_EDQ algorithm with traditional genetic algorithms. In the conducted four experiments, compared to the optimal paths planned by traditional genetic algorithms, the improved Q-algorithm achieved a minimum efficiency increase of 12.52% and a maximum of 35%. These results demonstrate the enhanced capability of the improved Q-algorithm to learn globally optimal paths in complex multimodal transportation networks. Compared to the classical Q algorithm, the algorithmic model in this study shows an average performance improvement of 10% to 30% in global optimal path search, as well as convergence performance including loss and reward values.