Social Learning In Markov Games: Empowering Autonomous Driving

Abstract

In a multi-agent system (MAS), a social learning scheme allows independent agents to learn through interactions with agents randomly selected from a pool. Such a scheme is important for autonomous vehicles (AV) to navigate complex traffic environments consisting of many road users. In this paper, we apply the social learning scheme to Markov games and leverage deep reinforcement learning (DRL) to investigate how individual AVs learn policies and form social norms in traffic scenarios. To capture agents’ different attitudes toward traffic environments, a heterogeneous agent pool with cooperative and defective AVs is introduced to the social learning scheme. To solve social norms formed by AVs, we propose a DRL algorithm, and apply them to traffic scenarios: unsignalized intersection and highway platoon. We find that compared to defective AVs, cooperative AVs can easily conform to expected social norms. In addition, cooperative AVs would lead to lower crash rates. We also find that prioritized roads/lanes can make AVs conform to expected social norms.