DNN-based Speech Synthesis using Dialogue-Act Information and Its Evaluation with Respect to Illocutionary Act Naturalness

Abstract

This paper aims at improving naturalness of synthesized speech generated by a text-to-speech (TTS) systemwithin a spoken dialogue system with respect to “how natural the system’s intention is perceived via the synthesizedspeech”. We call this measure “illocutionary act naturalness” in this paper. To achieve this aim, we propose toutilize dialogue-act (DA) information as an auxiliary feature for a deep neural network (DNN)-based speech synthesissystem. First, we construct a speech database with DA tags. Second, we build the proposed DNN-based speechsynthesis system based on the database. Then, we evaluate the proposed method by comparing its performance withtwo conventional hidden Markov model (HMM)-based speech synthesis systems, namely, the style-mixed modelingmethod and the style adaptation method. The objective evaluation results show that the proposed method overwhelmsthe style-mixed modeling method in the accuracy of reproduction of global prosodic characteristics of dialogue-acts.They also reveal that the proposed method overwhelms the style adaptation method in the accuracy of reproduction of sentence final tone characteristics of dialogue-acts. The subjective evaluation results also show that the proposed method improves the illocutionary act naturalness compared with the two conventional methods.