Warmth and Competence in Human-Agent Cooperation

Abstract

**Read paper on the following link:** https://ifaamas.org/Proceedings/aamas2022/pdfs/p898.pdf **Abstract:** Cooperation is an overarching aspiration of artificial intelligence (AI) research. Recent studies demonstrate that AI agents trained with deep reinforcement learning are capable of interacting and collaborating with humans. These studies have primarily evaluated human compatibility and intelligence through "objective" metrics like task performance, obscuring potential variation in the levels of trust and subjective preference that different agents generate. To better understand the factors shaping subjective preferences in human-agent cooperation, we train deep reinforcement learning agents in Coins, a two-player social dilemma. We recruit participants for a human-agent cooperation study, and measure their social perception of the agents they encounter. Participants' perceptions of warmth and competence predict their preferences for different agents, above and beyond objective performance metrics. Drawing inspiration from social science and biology research, we subsequently implement a new "partner choice" framework to elicit revealed preferences: after playing an episode with an agent, participants are asked whether they would like to play the next round with the same agent or to play alone. As with stated preferences, social perception better predicts participants' revealed preferences than does objective performance. Given these findings, we recommend human-agent interaction researchers routinely incorporate the measurement of social perception and subjective preferences into their studies.