On Appropriateness and Estimation of the Emotion of Synthesized Response Speech in a Spoken Dialogue System

Abstract

Paralinguistic features such as emotion of an utterance is as important as its linguistic content for generating better response utterances in spoken dialog systems. In this research, we carried out an experiment to reveal the effect of emotional speech synthesis in a dialogue system, and investigated what method was effective for giving emotion to the synthetic speech. Firstly, we carried out an experiment where an agent with various emotional speech talked to the user, and the appropriateness of the emotion was evaluated. As expected, users had better impression on the agent when we added emotion appropriately. Next, we examined methods of automatic estimation of emotion for the system’s response, and we found that the best method was to give the same emotion as the user’s previous utterance regardless of the content of the system’s utterance.