Hierarchical Sarsa Learning Based Route Guidance Algorithm

Feng Wen,Xingqiao Wang,Xiaowei Xu

doi:10.1155/2019/1019078

Abstract

In modern society, route guidance problems can be found everywhere. Reinforcement learning models can be normally used to solve such kind of problems; particularly, Sarsa Learning is suitable for tackling with dynamic route guidance problem. But how to solve the large state space of digital road network is a challenge for Sarsa Learning, which is very common due to the large scale of modern road network. In this study, the hierarchical Sarsa learning based route guidance algorithm (HSLRG) is proposed to guide vehicles in the large scale road network, in which, by decomposing the route guidance task, the state space of route guidance system can be reduced. In this method, Multilevel Network method is introduced, and Differential Evolution based clustering method is adopted to optimize the multilevel road network structure. The proposed algorithm was simulated with several different scale road networks; the experiment results show that, in the large scale road networks, the proposed method can greatly enhance the efficiency of the dynamic route guidance system.

Highlights

In the recent decades, more and more people own their private vehicles, and the traffic pressure in the city increased rapidly
The results show that, compared with traditional methods, the proposed Sarsa learning based route guidance algorithm (SLRGA) and Sarsa learning with Boltzmann distribution algorithm (SLWBD) can strongly reduce the travelling time and relieve traffic congestion
(1)Dijkstra algorithm (DA): DA is adopted to represent the static shortest route method, and it calculates the routes begin //Initializing Q-value of BT󸀠 in each subnetwork for each d ∈ BT󸀠 do Initialize Qd According to Eq (27) in the corresponding subnetwork end for //Initializing Q-value of D in the higher level network for each d ∈ D do Initialize Qd According to Eq (27) in the Gh󸀠 igh end for end Algorithm 3: Procedure of Initialization

Summary

Introduction

More and more people own their private vehicles, and the traffic pressure in the city increased rapidly. Reinforcement learning strategy has been widely used in the dynamic environment [10,11,12,13], because it can reduce the computational time and make full use of real-time information. With these characters, reinforcement learning strategy has been used in the dynamic route guidance system. The results show that, compared with traditional methods, the proposed Sarsa learning based route guidance algorithm (SLRGA) and Sarsa learning with Boltzmann distribution algorithm (SLWBD) can strongly reduce the travelling time and relieve traffic congestion

Methods

Results

Conclusion