Recursive Reasoning for Bounded Rationality in Multi-Agent Non-Equilibrium Play Learning Systems

Abstract

In this work, and inspired by the theory of bounded rationality and recursive reasoning, we propose two frameworks for modeling players’ behaviors and for choosing their policies in multi-agent dynamic stochastic game settings. In particular, we define multiple levels of rationality for each player, where at each level a player may reason about everyone else in two different ways; first, they may assume that the rest of the players have a cognitive level that is immediately lower than theirs, which is known as level-k thinking; second, they may assume that the rest of the players’ cognitive level follows a Poisson distribution, which is known as cognitive hierarchy. We construct algorithms for estimating the players’ policies at each level of rationality, both in a level-recursive as well as in a level-paralleled manner, and we study these algorithms’ convergence properties. Simulations on a grid world are provided to illustrate the efficacy of the proposed models.