Multi-issue negotiation with deep reinforcement learning

Abstract

Negotiation is a process where agents work through disputes and maximize surplus. This paper investigates the use of deep reinforcement learning in the domain of negotiation, evaluating its ability to exploit, adapt, and cooperate. Two actor–critic networks were trained for the bidding and acceptance strategy, against time-based agents, behavior-based agents, and through self-play. Results reveal four key findings. First, neural agents learn to exploit time-based agents, achieving clear transitions in decision values. The primary barriers are the change in marginal utility (second derivative) and cliff-walking resulting from negotiation deadlines. Second, the Cauchy distribution emerges as suitable for sampling offers, due to its peaky center and heavy tails. Third, neural agents demonstrate adaptive behavior against behavior-based agents. Fourth, neural agents learn to cooperate during self-play. Agents learn non-credible threats, which resemble reputation-based strategies in the evolutionary game theory literature.