Cooperative Learning of a Driving Strategy to Suppress Phantom Traffic Jams

Abstract

In this paper, we propose a cooperative learning method to acquire the intelligent self-driving strategy. The intelligent self-driving vehicle can control its velocity by itself to maintain an optimal traffic flow avoiding a phantom traffic jam. We focus on the traffic simulation where both intelligent and non-intelligent vehicles exists, i.e., manually driven by a human. In related previous researches, because driving strategy has been designed empirically for a particular traffic situation, the driving strategy has to be redesigned as the traffic situation changes. In contrast, our approach, which is based on a reinforcement learning, considers the traffic situation and utilizes factors such as velocity and interval of other vehicle to learn the optimal control strategy in the current situation. Furthermore, we focused on the cooperation of the self-driving vehicles among themselves in traffic, where they share the observation of each surrounding state and acquire policies from each other for common driving strategy. To show the effectiveness of our approach, we simulated a road with a sag section and comprising manual driving vehicles with velocity perturbations, which is the reason for the phantom traffic jams. As a result, we found that intelligent vehicles that can share information of the current state suppresses the phantom traffic jams when the intelligent vehicles represented more than 10% of the total traffic. We also found that by sharing information when learning the driving strategy, the intelligent vehicle's learning behavior converges sooner than without sharing.