Common belief multi-agent reinforcement learning based on variational recurrent models

Abstract

The tacit cooperation among human teams benefits from the fact that consensus can be reached on a task through common belief. Similar to human social groups, agents in distributed learning systems can also rely on common belief to achieve cooperation under the condition of limited communication. In this paper, we show the role of common belief among agents in completing cooperative tasks, by proposing the Common Belief Multi-Agent (CBMA) reinforcement learning method. CBMA is a novel value-based method that infers the belief between agents with a variational model and models the environment with a variational recurrent neural network. We validate CBMA on two grid-world games as well as the StarCraft II micromanagement benchmark. Experimental results show that the learned common belief by CBMA can improve performance in both discrete and continuous state settings.