A Q-learning approach to generating behavior of emotional persuasion with adaptive time belief in decision-making of agent-based negotiation

Abstract

There is an increasing research interest in agent-based automated negotiation. However, how to design an agent that can adaptively compete with its human counterpart is still in its early stage. This research makes efforts toward this end by modeling two essential elements of human-led negotiation: emotion and timing, to adapt to negotiation dynamics. In multi-issue-multi-round bilateral negotiation, the Web-Fechner law is applied to model an agent’s emotion generation and the emotion decay law is used to realize a decaying property of emotion. Meanwhile, rather than a fixed time belief, agents are endowed with time beliefs that can vary with selected negotiation tactics and be adjusted by agents’ emotions. A proposal updating algorithm integrating those two elements is consequently proposed based on Q-learning. Then, 135 numerical experiments were conducted with comparative analyses and statistical tests. The results show that the proposed model can significantly increase an individual agent’s utility, the joint utility, and the probability of successful negotiations by an average of 76.14%, 172.22%, and 66.10%, respectively, whereas reducing the utility difference and negotiation rounds by an average of 97.39% and 29.57%, respectively. These results imply that the proposed negotiation model has advantages in improving negotiation utility, efficiency, and outcome fairness.