Speech-driven Embodied Entrainment Character Research Articles

A speech-driven embodied entrainment character system with a delayed voice back-channel based on negative emotional expression utterances

A speech-driven embodied entrainment character system with a delayed voice back-channel based on negative emotional expression utterances

E-VChat : 頭部動作連動型音声駆動身体引き込みキャラクタを対面合成した実映像対話システム

E-VChat : 頭部動作連動型音声駆動身体引き込みキャラクタを対面合成した実映像対話システム

Evaluation of back-channel feedback timing in the speech-driven embodied entrainment character system

Evaluation of back-channel feedback timing in the speech-driven embodied entrainment character system

Speech-driven embodied entrainment character system with a voice back-channel

Speech-driven embodied entrainment character system with a voice back-channel

発話内単語の感情極性に基づき反応動作を行う身体的引き込みキャラクタシステム

The prior research includes the development of a speech-driven embodied entrainment computer-generated character called InterActor, which automatically generates communicative motions and actions such as nods for entrained interaction from voice rhythm based on only speech input. Because the conventional InterActor character generates only positive actions from the verbal content but no negative actions, it is possible to aggravate a speaker's negative emotions by performing positive gestures in response to negative verbal content. In this paper, we develop an advanced speech-driven embodied entrainment character system that can respond to and improve the speaker's emotional state using speech recognition. In this system, the speaker's words are converted to text by speech recognition; their emotions are then estimated from character strings in the converted text. The system uses a database that quantifies each word and estimates the emotion associated with it. Then, the system automatically generates negative/positive motions based on the semantic orientations of words in utterance as well as entrained motions. Furthermore, we demonstrate the effectiveness of the system through three experiments: two role-play experiments for one user involving positive/negative scenarios, and a communication experiment for two remote users using the developed system.

Speech-driven embodied entrainment character system with pupillary response

Speech-driven embodied entrainment character system with pupillary response

場の盛り上がり推定モデルを用いた視線インタラクションを支援する音声駆動型身体引き込みキャラクタシステム

Line-of-sight such as gaze and eye-contract plays an important role to enhance the embodied interaction and communication through avatars. In addition, many eyeball movement models and communication systems using avatars with the line-of-sight have been proposed and developed. However, these eyeball movements by generating the above-mentioned models were not considered to enhance the embodied interaction such as activated communication, because the models stochastically generate the eyeball movements based on the human gaze behavior. Therefore, in order to enhance and promote the embodied interaction from the view point of line-of-sight, it is desired to design a line-of-sight interaction which relates to the interaction-activated communication. In this study, we analyze the interaction between human gaze behavior and the interaction-activated communication by using an embodied communication system with line-of-sight measurement devices. Then, we propose a line-of-sight model based on the above-mentioned analysis, and develop an advanced embodied communication system by applying the proposed model to the speech-driven embodied entrainment character. This system generates the eyeball movements such as gaze and looking away based on the proposed model, and provides a communication environment wherein the embodied interaction is promoted. The effectiveness of the system is demonstrated by means of sensory evaluations of 15 pairs of subjects involved in avatar-mediated communication.

1110 音声駆動型身体引き込みキャラクタに視線モデルを付与した身体的コミュニケーションシステムの開発(OS9-1 ヒューマンインタフェース・ユーザビリティ1)

We have already analyzed enhanced interaction through avatars using an embodied virtual communication system and proposed an interaction-activated communication model based on the heat conduction equation. In this paper, we analyzed the interaction between the talker's gaze and the degree of interaction-activated communication in face-to-face communication, by using an embodied communication system with a line-of-sight measurement device. On the basis of this analysis, we proposed a line-of-sight model, which consists of an eyeball delay movement model and a look away model. Then, we developed an advanced communication system in which the proposed model is applied to speech-driven embodied entrainment character for enhancing embodied interaction and communication. This system generates the eyeball movement on the basis of the proposed model by using the speech input alone to realize smooth and enjoyable interaction and communication.

1111 発話感情表現に基づき反応動作を行う音声駆動型身体的引き込みキャラクタシステムの開発(OS9-1 ヒューマンインタフェース・ユーザビリティ1)

1111 発話感情表現に基づき反応動作を行う音声駆動型身体的引き込みキャラクタシステムの開発(OS9-1 ヒューマンインタフェース・ユーザビリティ1)

2207 音声認識による動作・情動表現機能を有する音声駆動型身体的引き込みキャラクタシステムの評価(OS11 ヒューマンインタフェース・ユーザビリティII)

2207 音声認識による動作・情動表現機能を有する音声駆動型身体的引き込みキャラクタシステムの評価(OS11 ヒューマンインタフェース・ユーザビリティII)

C15 音声駆動型身体引き込みキャラクタに眼球動作モデルを付与したコミュニケーションシステム(ロボティクス・メカトロニクス)

C15 音声駆動型身体引き込みキャラクタに眼球動作モデルを付与したコミュニケーションシステム(ロボティクス・メカトロニクス)

Development of a Speech-Driven Embodied Color-Changed Entrainment System

InterActor is the speech-driven embodied entrainment character for activating human interaction and communication by embodied rhythms between speech and body motions such as nodding in human face-to-face conversation. In this paper, to clarify the visual effect of color change of entrainment on interaction support, a speech-driven entrainment color-changed entrainment system is developed. The system changes color on an object with a visualized response equivalent to nodding in the same timing as InterActor to speech input. By using the system, sensory evaluation by paired comparison demonstrates the effect of color change from the result that the pattern that the color changes from the top of the object is more desirable than the pattern that the whole color changes at a moment.

Effects of Speech-Driven Embodied Entrainment Character "InterActor" on Talker's Speech

A speech-driven embodied entrainment character called “InterActor” with both functions of speaker and listener has been developed for activating human interaction by generating the whole body motions such as nodding, blinking and the actions of head, arms and waist coherently related to speech input. This paper demonstrates the effects of InterActor on interaction in human speech input by the analysis of speech temporal parameters. It is found that speech duration increases and speech activity decreases because of the increase of total silence duration and the number of silence parts in human-to-InterActor speech input, which is the advantage of real time processing for speech recognition and so forth. Interaction with InterActor is also highly evaluated in smoothness, enjoyment and preference by sensory evaluation.

C211 3Dオブジェクトを用いた音声駆動型身体的引き込みシステム

InterActor is the speech-driven embodied entrainment character with both functions of speaker and listener for activating human interaction and communication. To clarify the essential interactive motion for embodied entrainment, a cylindrical InterActor is developed for the analysis by synthesis by changing the shape from human to cylinder as a primitive object. Sensory evaluation by using the system demonstrates the effectiveness of the motion that the cylinder leans forward to a user. On the basis of the result, a speech-driven embodied entrainment system EntrainmentField with InterTree/InterFlower of a tree/flower character with branches and leaves that have the same function of cylindrical InterActor is developed for a new embodied interaction and communication support. The application of EntrainmentField to a car navigation system is also proposed.