Evaluating the Impact of Embodied Conversational Agents (ECAs) Attentional Behaviors on User Retention of Cultural Content in a Simulated Mobile Environment

Abstract

The paper presents an evaluation study of the impact of an ECA's attentional behaviors using a custom research method that combines facial expression analysis, eye-tracking and a retention test. The method provides additional channels to EEG-based methods (e.g., [8]) for the study of user attention and emotions. In order to validate the proposed approach, two tour guide applications were created with an embodied conversational agent (ECA) that presents cultural content about a real-tourist attraction. The agent simulates two attention-grabbing mechanisms - humorous and serious to attract the users' attention. A formal study was conducted to compare two tour guide applications in the lab. The data collected from the facial expression analysis and eye-tracking helped to explain particularly good and bad performances in retention tests. In terms of the study results, strong quantitative and qualitative evidence was found that an ECA should not attract more attention to itself than necessary, to avoid becoming a distraction from the flow of the content. It was also found that the ECA had an inverse effect on the retention performance of participants with different gender and their use on computer interfaces is not a good idea for elderly users.