How humans obtain information from AI: Categorizing user messages in human-AI collaborative conversations

Abstract

Although there is an increasingly number of research about the design and use of conversational agents, it is still difficult for conversational agents to completely replace human service. Therefore, more and more companies have adopted human-AI collaborative systems to deliver customer service. It is important to understand how people obtain information from human-AI collaborative conversations. While the existing work relies on self-reported methods to elicit qualitative feedback from users, we have concluded a categorization system for user messages in human-AI collaborative conversations after a thorough examination of a real-world customer service log, which could objectively reflect the user's information needs. We categorize user messages into five categories and 15 specific types related to three high-level intentions. Two annotators independently classified the same set of 1,478 user messages from 300 conversations and reached a moderate consistency. We summarize and report the characteristics of different message types and compare their usage in sessions with only human, AI, or both representatives. Our results show that different message types vary significantly in usage frequency, length, and text similarities with other messages in a session. Also, the frequency of using different message types in our dataset seems consistent over sessions with different types of representatives. But we also observed some significant differences in a few specific message types across the sessions with different representatives. Our results are used to suggest some areas for improvement and future work in human-AI collaborative conversational systems.