Modeling Human Recursive Reasoning Using Empirically Informed Interactive Partially Observable Markov Decision Processes

Abstract

Recursive reasoning of the form what do I think that you think that I think (and so on) arises often while acting in multiagent settings. Previously, multiple experiments studied the level of recursive reasoning generally displayed by humans while playing sequential general-sum and fixed-sum, two-player games. The results show that subjects experiencing a general-sum strategic game display first or second level of recursive thinking with the first level being more prominent. However, if the game is made simpler and more competitive with fixed-sum payoffs, subjects predominantly attributed first-level recursive thinking to opponents thereby acting using second level. In this article, we model the behavioral data obtained from the studies using the interactive partially observable Markov decision process, appropriately simplified and augmented with well-known models simulating human learning and decision. We experiment with data collected at different points in the study to learn the models parameters. Accuracy of the predictions by our models is evaluated by comparing them with the observed study data, and the significance of the fit is demonstrated by comparing the mean squared error of our model predictions with those of a random hypothesis. Accuracy of the predictions by the models suggest that these could be viable ways for computationally modeling strategic behavioral data in a general way. While we do not claim the cognitive plausibility of the models in the absence of more evidence, they represent promising steps toward understanding and computationally simulating strategic human behavior.