Is chatting with a sophisticated chatbot as good as chatting online or FTF with a stranger?

Abstract

Emotionally-responsive chatbots are marketed as agents with which one can form emotional connections. They can also become weak ties in the outer layers of one's acquaintance network and available for social support. In this experiment, which was designed to study the acquaintance process, we randomly assigned 417 participants into three conditions: face-to-face (FTF) chat with a human, online chat with a human, and online chat with a commercially-available, emotionally-responsive chatbot, Replika. After a 20-min getting-acquainted chat, participants reported their affective state and relational evaluations of the chat. Additionally, all chats were recorded and text analyzed using the Linguistic Inquiry and Word Count (LIWC) program. In all conditions, participants reported moderate levels of positive emotions and low levels of negative emotions. Those who chatted FTF with a human reported significantly more negative emotions than those who chatted with a bot. However, those who chatted with a human also reported more homophily with and liking of their chat partner and that their partner was more responsive. Meanwhile, participants had fewest conversational concerns with the chatbot. These findings have implications for future computer-mediated interaction studies: conversations with chatbots appear to have different affordances and effects on chatter enjoyment and conversational concerns in getting-acquainted contexts. These results may help designers improve reception and marketability for chatbots in consumer markets.