Automatic Generation System of Virtual Agent's Motion using Natural Language

Abstract

A virtual agent in a dialogue system should express appropriate body motions according to utterances and effectively communicate with a user. We previously proposed a generation model of whole body motions such as head direction, nodding, facial expressions, hand gestures, and upper-body posture accompanying utterances at appropriate times similar to humans by using various types of natural-language-analysis information obtained from spoken language. As an attempt to promote this model, we constructed an API that can easily generate motions by using the generation model and constructed a demonstration system that can automatically control a virtual agent from only the spoken language. When inputting an arbitrary utterance language, synthesized sound and motion information are acquired from the speech synthesizer and motion-generation API, and the vocalization of the virtual agent and animated motion are generated. A dialog agent that is more attractive and can communicate smoothly by automatically generating natural motions is expected.