Incorporating reputation into reinforcement learning can promote cooperation on hypergraphs

Abstract

Most previous studies that investigate the evolutionary dynamics of cooperation in the multi-agent social dilemma only consider the pairwise interactions. However, pairwise interactions alone cannot adequately characterize the higher-order interaction relationships that are commonly present in real-world social systems. In this study, we propose a spatial public goods game on uniform random hypergraphs with reputation mechanism and self-regarding Q-learning algorithm, where agents take an action based on the Q-value and update their reputation by the first-order evaluation norm. We define the reward as a function of the agent’s payoff and their reputation influenced by the social attention degree. Simulation results indicate that an increase in the social attention degree can effectively facilitate cooperation, as social reputation incentivizes agents to adopt cooperative actions to obtain optimal rewards. Additionally, we find that the discount factor and the learning rate both significantly impact the evolution of cooperation, with the influence of the learning rate on the evolution of cooperation being dependent on the discount factor. Finally, the robustness of this model is demonstrated based on the results of theoretical analysis.