RSU-Assisted Traffic-Aware Routing Based on Reinforcement Learning for Urban Vanets

Jinqiao Wu,Min Fang,Haikun Li,Xiao Li

doi:10.1109/access.2020.2963850

Jinqiao Wu, Min Fang + Show 2 more

Open Access

https://doi.org/10.1109/access.2020.2963850

Copy DOI

Abstract

In urban vehicular ad hoc networks (VANETs), the high mobility of vehicles along street roads poses daunting challenges to routing protocols and has a great impact on network performance. In addition, the frequent network partition caused by an uneven distribution of vehicles in an urban environment further places higher requirements on the routing protocols in VANETs. More importantly, the high vehicle density during the traffic peak hours and a variety of natural obstacles, such as tall buildings, other vehicles and trees, greatly increase the difficulty of protocol design for high quality communications. Considering these issues, in this paper, we introduce a novel routing protocol for urban VANETs called RSU-assisted Q-learning-based Traffic-Aware Routing (QTAR). Combining the advantages of geographic routing with the static road map information, QTAR learns the road segment traffic information based on the Q-learning algorithm. In QTAR, a routing path consists of multiple dynamically selected high reliability connection road segments that enable packets to reach their destination effectively. For packet forwarding within a road segment, distributed V2V Q-learning (Q-learning occurs between vehicles) integrated with QGGF (Q-greedy geographical forwarding) is adopted to reduce delivery delay and the effect of fast vehicle movements on path sensitivity, while distributed R2R Q-learning (Q-learning occurs between RSU units) is designed for packet forwarding at each intermediate intersection. In the case of a local optimum occurring in QGGF, SCF (store-carry-forward) is used to reduce the possibility of packet loss. Detailed simulation experimental results demonstrate that QTAR outperforms the existing traffic-aware routing protocols, in terms of 7.9% and 16.38% higher average packet delivery ratios than those of reliable traffic-aware routing (RTAR) and greedy traffic-aware routing (GyTAR) in high vehicular density scenarios and 30.96% and 46.19% lower average end-to-end delays with respect to RTAR and GyTAR in low vehicular density scenarios, respectively.

Highlights

With the rapid development of wireless communication technology, vehicular ad hoc networks (VANETs) have emerged as one of the most prospective solutions to enhance road traffic efficiency and decrease road traffic accidents in an intelligent transportation system (ITS)
In this paper, we propose a novel RSU-assisted Q-learning-based Traffic Aware Routing (QTAR) protocol designed for urban VANETs to enhance the awareness of road traffic conditions and reduce the impact of the rapid mobility of vehicles on the network performance by providing an efficient packet forwarding mechanism for a variety of applications in scalable urban VANETs
We present the main functionality of Q-learning-based Traffic-Aware Routing (QTAR), which mainly consists of the following components: first, deciding the first intersection to which packets are forwarded from the source vehicle Vs; second, packet forwarding at each intermediate intersection to the adjacent intersection until reaching the last intersection that connects the road segment on which the destination vehicle Vd is moving; and packet forwarding within the road segment from the last intersection to Vd

Summary

INTRODUCTION

With the rapid development of wireless communication technology, vehicular ad hoc networks (VANETs) have emerged as one of the most prospective solutions to enhance road traffic efficiency and decrease road traffic accidents in an intelligent transportation system (ITS). Q-learning-based routing encounters scalability limitations for large highly dynamic networks because of the slow convergence of the learning algorithm; the forwarding decisions cannot keep up with the road traffic and network topology changes To this end, in this paper, we propose a novel RSU-assisted Q-learning-based Traffic Aware Routing (QTAR) protocol designed for urban VANETs to enhance the awareness of road traffic conditions and reduce the impact of the rapid mobility of vehicles on the network performance by providing an efficient packet forwarding mechanism for a variety of applications in scalable urban VANETs. The high-rate but short-range V2V communications within the road segments through the V2V channel are guided by low-rate but longrange R2R communications through the R2R channel. We first provide an elaborated description of QTAR and present comprehensive experimental results compared with other existing related protocols

THE PROPOSED PROTOCOL

HELLO PACKET FORMAT FOR V2V AND R2R Q-LEARNING

R2R Q-LEARNING FORWARDING AT INTERSECTIONS

EXPERIMENTAL RESULTS

PERFORMANCE FOR VARYING α AND γ

CONCLUSION AND FUTURE WORK