Reliable Navigation Learning in Stochastic Transportation Networks with Uncertain Topology

Abstract

This paper studies the reliable navigation problem in stochastic transportation networks with uncertain topology. The objective is to find a dynamic routing policy, which navigates the ego vehicle to the destination with the minimal linear combination of mean and standard deviation (mean-std) of travel time. Different from almost all other stochastic routing problems, which focus on the stochasticity of travel time and assume a fixed topology of the underlying transportation network, the reliable navigation problem considered in this paper involves an additional layer of stochasticity, namely the uncertain network topology. To reach reliable navigation, we advance the canonical reinforcement learning (RL) method, which could originally find a routing policy with the minimal mean travel time, to simultaneously estimate and minimize the linear combination of the mean and standard deviation of a routing policy’s travel time. Additionally, to cope with the uncertain network topology problem, we enhance RL’s underlying Markov decision process (MDP) with a variational action set as well as a novel decision-list policy that outputs a list of ranked actions instead of a single one. We evaluate and compare the proposed method with state-of-the-art mean-std shortest path solutions in a range of transportation networks, and demonstrate that the proposed method achieves superior performance with the lowest response time.