Providing real-time en-route suggestions to CAVs for congestion mitigation: A two-way deep reinforcement learning approach

Abstract

This research investigates the effectiveness of information provision for congestion reduction in Connected Autonomous Vehicle (CAV) systems. The inherent advantages of CAVs, such as vehicle-to-everything communication, advanced vehicle autonomy, and reduced human involvement, make them conducive to achieving Correlated Equilibrium (CE). Leveraging these advantages, this research proposes a reinforcement learning framework involving CAVs and an information provider, where CAVs conduct real-time learning to minimize their individual travel time, while the information provider offers real-time route suggestions aiming to minimize the system’s total travel time. The en-route routing problem of the CAVs is formulated as a Markov game and the information provision problem is formulated as a single-agent Markov decision process. Then, this research develops a customized two-way deep reinforcement learning approach to solve the interrelated problems, accounting for their unique characteristics. Moreover, CE has been formulated within the proposed framework. Theoretical analysis rigorously proves the realization of CE and that the proposed framework can effectively mitigate congestion without compromising individual user optimality. Numerical results demonstrate the effectiveness of this approach. Our research contributes to the advancement of congestion reduction strategies in CAV systems with the mitigation of the conflict between system-level and individual-level goals using CE as a theoretical foundation. The results highlight the potential of information provision in fostering coordination and correlation among CAVs, thereby enhancing traffic efficiency and achieving system-level goals in smart transportation.