Auditory assessment of synthesized speech in application scenarios: Two case studies

Abstract

For assessing the synthesized speech output component in a complex application system, application-oriented evaluation methods and methodologies are needed which are not supplied by standardized test batteries so far. Many standardized tests analyze synthetic speech mainly with regard to its form (surface structure), and only to a less degree with regard to the meaning that is assigned to it (deep structure). In turn, in order to obtain a valid assessment focus for an application system, the functional aspect of speech (which depends on its deep structure) has to be taken into account. In the paper two case studies are presented which focus on the acceptability of the synthesis component and its constituent dimensions in different application scenarios. In the first one synthetic speech in a car navigation and traffic information system is assessed. The second study relates to synthetic speech in a dialogue system. The assessment is limited to laboratory experiments and avoids costly field tests. It turns out that different dimensions contribute to a variable degree to overall acceptability, differently depending on the application scenario. Application-oriented testing is thus required to identify the application-specific dimensions. It is discussed which characteristics of the application have to be modeled in the assessment, and examples are given for both applications.