Universal effect of dynamical reinforcement learning mechanism in spatial evolutionarygames

Abstract

One of the prototypical mechanisms in understanding the ubiquitous cooperation in socialdilemma situations is the win–stay, lose–shift rule. In this work, a generalized win–stay,lose–shift learning model—a reinforcement learning model with dynamic aspirationlevel—is proposed to describe how humans adapt their social behaviors based on theirsocial experiences. In the model, the players incorporate the information of the outcomes inprevious rounds with time-dependent aspiration payoffs to regulate the probability ofchoosing cooperation. By investigating such a reinforcement learning rule in the spatialprisoner’s dilemma game and public goods game, a most noteworthy viewpoint is thatmoderate greediness (i.e. moderate aspiration level) favors best the development andorganization of collective cooperation. The generality of this observation is tested againstdifferent regulation strengths and different types of network of interaction as well. We alsomake comparisons with two recently proposed models to highlight the importance of themechanism of adaptive aspiration level in supporting cooperation in structuredpopulations.