Q-learning boosts the evolution of cooperation in structured population by involving extortion

Abstract

Extortion strategies can guarantee that one player’s own surplus exceeds the co-player’s surplus by a fixed percentage. Although extortion is unstable in the well-mixed population, recent studies have found that extortion can act as a catalyst to promote cooperation in the spatial prisoner’s dilemma game, especially the strategy updating is ruled by replicator-like dynamics and innovation mechanisms, such as myopic best response or aspiration-driven dynamics. Q-learning is a typical self-reinforcement learning algorithm. Importantly, it cannot promote cooperation in the classic two-strategy prisoner’s dilemma game. Here, we explore the effect of Q-learning on cooperation by involving extortion. Results reveal Q-learning significantly boosts the evolution of cooperation, which is robust to population structures (regular lattice, small world network and scale-free network) and extortion strength. The reason for that is the extortioner provides cooperators a better opportunity to survive and cooperators act as catalysts to promote the coexistence of the three strategies. In particular, Q-learning is more significant in promoting cooperation than replicator-like dynamics and myopic best response. When the temptation to defect is not too large, Q-learning performs better than aspiration-driven dynamics, on the contrary, aspiration-driven dynamics performs better. This study reveals the important role of reinforcement learning in the evolution of cooperation.