"Pretending to be Okay in a Sad Voice": Social Robot’s Usage of Verbal and Nonverbal Cue Combination and its Effect on Human Empathy and Behavior Inducement

Abstract

Inducing a user’s behavior through social interaction is a goal that a social robot aims to achieve. It has been argued that empathy has a strong effect on behavior inducement. In human-human interaction, it has been verified that the influence of a nonverbal cue on empathy outweighs that of a verbal cue when those are used in a combined way. The objectives of this study are to explore if such outweighing effect of nonverbal cues is maintained in human-robot interaction (HRI) and to investigate the mechanism of communication cues’ effects by analyzing the mediation structure with the following mediator variables: perceived emotion, perceived intentionality, perceived malfunction, and empathy inducement. To this end, we conducted 2 (verbal type: positive verbal cue vs. negative verbal cue) × 2 (nonverbal type: positive nonverbal cue vs. negative nonverbal cue) within-participant experiment (N = 48). The experiment created a situation in which the social robot was harshly criticized during a conversation. The analysis of experiment results showed the outweighing effect of a nonverbal cue was maintained. When a nonverbal cue conveyed a negative, situation-accordant emotion, it had a decisive effect on perceived emotion, empathy, and behavior inducement. In contrast, a verbal cue induced participants’ empathy and behavior when it conveyed a positive, situation-discordant emotion. This inconsistency between verbal cue and nonverbal cue made the combination of positive verbal cue and negative nonverbal cue have the strongest effect. It implies that participants had different expectations for each of the two communication cues, just as they did in social interactions with human beings. It implies that participants might have applied normative expectations for social interaction with human beings to the social robot.