Exploring features from natural language generation for prosody modeling

Abstract

Prosody modeling is critical in developing a Concept-to-Speech (CTS) system where both Natural Language Generation (NLG) and Speech Synthesis are used to automatically generate natural, coherent speech. In this paper, we empirically verify the usefulness of various natural language features in prosody modeling. Three groups of features are investigated: semantic, syntactic, and surface features produced by SURGE, a general-purpose surface natural language generator for English, deep semantic, and discourse features that are available during the domain modeling and content planning phases of generation, and information-based measures statistically derived from text. Our experiments identify which of this large set of features are effective in prosody modeling. This work represents an important step towards building a comprehensive prosody model for CTS systems that employ general NLG. This investigation is conducted in the context of MAGIC, a medical application that involves automatic speech and graphics generation.