How Do You Like Your Virtual Agent?: Human-Agent Interaction Experience through Nonverbal Features and Personality Traits

Abstract

Recent studies suggest that human interaction experience with virtual agents can be, to a very large degree, described by people’s personality traits. Moreover, the nonverbal behavior of a person has been known to indicate several social constructs in different settings. In this study, we analyze human-agent interaction from the perspective of the personality of the human and the nonverbal behaviors he/she displays during the interaction. Based on existing work in psychology, we designed and recorded an experiment on human-agent interactions, in which a human communicates with two different virtual agents. Human-agent interactions are described with three self-reported measures: quality, rapport and likeness of the agent. We investigate the use of self-reported personality traits and extracted audio-visual nonverbal features as descriptors of these measures. Our results on a correlation analysis show significant correlations between the interaction measures and several of the personality traits and nonverbal features, which are supported by both psychology and human-agent interaction literature. We further use traits and nonverbal cues as features to build regression models for predicting measures of interaction experience. Our results show that the best results are obtained when nonverbal cues and personality traits are used together.