Video communication system with speech-driven embodied entrainment audience characters with partner's face

Abstract

We previously proposed an embodied video communication system called E-VChat, in which a computer-generated (CG) avatar is superimposed on the other talker's video image in a face-to-face scene, so that they can better comprehend their mutual interactions during remote communication. In communications using a video chat in strain scenes, such as an interview, the effect that eases tensions is shown by carrying out superposition composition of the character, which becomes a partner image with a self-character, and four audience characters, which perform communicative motions, such as a nod based on speech input. In this paper, we describe a new video communication system that includes audience characters showing a partner's face. This system provides a more interactive space that makes it easy to communicate by clearly distinguishing the simple audience characters and the functional substitutes of the partner. We performed a communication experiment to confirm the effectiveness of the E-VChat system for 12 pairs of subjects using three communication modes: only the talker's avatar, the talker's avatar and audience character, and the talker's avatar and audience characters with the partner's face.