Vehicular intelligent collaborative intersection driving decision algorithm in Internet of Vehicles

Abstract

Connected and Automated Vehicles (CAV) are regarded as the developing trend of future transportation due to the advantages in terms of increasing traffic throughput, safety, and reducing energy consumption. One of the challenging CAV research problems is intelligent collaborative driving decision in dynamically changing traffic scenarios. The sixth-generation mobile communication technology (6G) plays an important role in attaining CAV intelligent driving in Internet of Vehicles (IoV) by provide significantly higher system performance, higher spectral efficiency, higher reliability, and higher security. This paper formulates the collaborative decision problem of CAVs at unsignalized intersections as a multi-agent reinforcement learning (MARL) problem, where CAVs entering an intersection safely cross the intersection and minimize their traffic time through collaborating to learn a strategy. An efficient and scalable MARL algorithm based on Proximal Policy Optimization (PPO) is applied to the dynamic intersection scenarios, where the parameter-sharing mechanism is used to improve the PPO algorithm to be a multi-agent version. Instead of the global reward, the local reward is used to promote cooperation between agents and achieve scalability. In addition, the action masking mechanism is adopted to improve the learning efficiency by filtering out invalid actions at each step. A joint simulation platform is built to verify the performance of the Parameter-Sharing PPO algorithm (PS-PPO). The simulation results show that the PS-PPO algorithm not only guarantees a low collision rate but also greatly reduces the vehicular travel time. This algorithm promotes cooperation among CAVs and performs well in the task of collaborative traffic flow at unsignalized intersections, especially in high-traffic volume scenarios. It is helpful to improve the safety and efficiency of traffic flow at unsignalized intersections.