Reinforcement Learning of Heterogeneous Private Agents in a Macroeconomic Policy Game

Abstract

A repeated inflation-unemployment game within the linear-quadratic framework of Barro and Gordon is studied assuming that the government would like to cheat optimally and the finite heterogeneous population of private agents attempts to learn the government’s targets using a reinforcement learning algorithm. Private agents are heterogeneous in their initial expectations of inflation rate but are assumed to utilize an identical anticipatory reinforcement learning process, namely Q-learning. In our heterogeneous setting, the only way for the private agents to achieve a zero value for their loss function, is for all of them to correctly anticipate the Nash equilibrium. It is of particular significance that such a solution requires a convergence of expectations across an initially heterogeneous population. Computer simulations have been conducted using different tuning parameters to investigate the convergence of our proposed model of learning process to Nash equilibrium.