Estimating User’s Conversational Engagement Based on Gaze Behaviors

Abstract

In face-to-face conversations, speakers are continuously checking whether the listener is engaged in the conversation. When the listener is not fully engaged in the conversation, the speaker changes the conversational contents or strategies. With the goal of building a conversational agent that can control conversations with the user in such an adaptive way, this study analyzes the user's gaze behaviors and proposes a method for predicting whether the user is engaged in the conversation based on gaze transition 3-Gram patterns. First, we conducted a Wizard-of-Oz experiment to collect the user's gaze behaviors as well as the user's subjective reports and an observer's judgment concerning the user's interest in the conversation. Next, we proposed an engagement estimation algorithm that estimates the user's degree of engagement from gaze transition patterns. This method takes account of individual differences in gaze patterns. The algorithm is implemented as a real-time engagement-judgment mechanism, and the results of our evaluation experiment showed that our method can predict the user's conversational engagement quite well.