TTS Voices Research Articles

Comparing human and machine's use of coarticulatory vowel nasalization for linguistic classificationa).

Anticipatory coarticulation is a highly informative cue to upcoming linguistic information: listeners can identify that the word is ben and not bed by hearing the vowel alone. The present study compares the relative performances of human listeners and a self-supervised pre-trained speech model (wav2vec 2.0) in the use of nasal coarticulation to classify vowels. Stimuli consisted of nasalized (from CVN words) and non-nasalized (from CVCs) American English vowels produced by 60 humans and generated in 36 TTS voices. wav2vec 2.0 performance is similar to human listener performance, in aggregate. Broken down by vowel type: both wav2vec 2.0 and listeners perform higher for non-nasalized vowels produced naturally by humans. However, wav2vec 2.0 shows higher correct classification performance for nasalized vowels, than for non-nasalized vowels, for TTS voices. Speaker-level patterns reveal that listeners' use of coarticulation is highly variable across talkers. wav2vec 2.0 also shows cross-talker variability in performance. Analyses also reveal differences in the use of multiple acoustic cues in nasalized vowel classifications across listeners and the wav2vec 2.0. Findings have implications for understanding how coarticulatory variation is used in speech perception. Results also can provide insight into how neural systems learn to attend to the unique acoustic features of coarticulation.

Multilingual Support for Accessibility in PowerPoint STEM Contents.

Many online educational materials for sighted students such as presentation slides, PDF materials, MP4 videos, etc. are produced with Microsoft PowerPoint (PPT). In terms of non-technical contents, accessible PPT contents can be produced in a certain level; however, as far as STEM contents are concerned, there is still a serious difficulty. Our new add-on for PPT allows users to make efficiently/easily PPT STEM contents accessible. By making use of it, alt text/aloud reading by a TTS voice can be added to any technical part such as mathematical expressions included in PPT slides. An accessible MP4 video for STEM education also can be produced efficiently. By making use of multilingual support in Infty software, this add-on has been recently improved so that users can use it for PPT contents in various local languages other than Japanese or English.

Comparing alignment toward American, British, and Indian English text-to-speech (TTS) voices: influence of social attitudes and talker guise

Text-to-speech (TTS) voices, which vary in their apparent native language and dialect, are increasingly widespread. In this paper, we test how speakers perceive and align toward TTS voices that represent American, British, and Indian dialects of English and the extent that social attitudes shape patterns of convergence and divergence. We also test whether top-down knowledge of the talker, manipulated as a “human” or “device” guise, mediates these attitudes and accommodation. Forty-six American English-speaking participants completed identical interactions with 6 talkers (2 from each dialect) and rated each talker on a variety of social factors. Accommodation was assessed with AXB perceptual similarity by a separate group of raters. Results show that speakers had the strongest positive social attitudes toward the Indian English voices and converged toward them more. Conversely, speakers rate the American English voices as less human-like and diverge from them. Finally, speakers overall show more accommodation toward TTS voices that were presented in a “human” guise. We discuss these results through the lens of the Communication Accommodation Theory (CAT).

Comparing intelligibility and recognition memory of human and text-to-speech voices

Clear speech can be more intelligible and better remembered than casual speech. Intelligibility and memory are assumed to be related—under the effortfulness hypothesis, the intelligibility-enhancing properties of clear speech (e.g., slower speech rate) facilitate processing, which frees up working memory resources and confers a memory advantage. However, prior work has only investigated naturally produced clear speech. How does clear speech influence memory for computer voices? The current study examines intelligibility and recognition memory of clear and casual speech for a text-to-speech (TTS) voice and a human voice, and compares older and younger adult listeners. In a speech-perception-in-noise task, clear speech was more intelligible than casual speech for both voices, the human was more intelligible than the TTS voice, and the younger adults performed better than the older adults. In an old-new recognition task, memory was better for the human voice relative to the TTS voice, consistent with the intelligibility results and the effortfulness hypothesis. However, inconsistent with the effortfulness hypothesis, there was no clear speech memory benefit, and the older adults performed better than the younger adults. The current findings are relevant for our understanding of human-computer interaction and theories about the relationship between speaking style and memory.

German Word-Final Devoicing in Naturally-Produced and TTS Speech

This study explores the production and perception of word-final devoicing in German across text-to-speech (from technology used in common voice-AI “smart” speaker devices—specifically, voices from Apple and Amazon) and naturally produced utterances. First, the phonetic realization of word-final devoicing in German across text-to-speech (TTS) and naturally produced word productions was compared. Acoustic analyses reveal that the presence of cues to a word-final voicing contrast varied across speech types. Naturally produced words with phonologically voiced codas contain partial voicing, as well as longer vowels than words with voiceless codas. However, these distinctions are not present in TTS speech. Next, German listeners completed a forced-choice identification task, in which they heard the words and made coda consonant categorizations, in order to examine the intelligibility consequences of the word-final devoicing patterns across speech types. Intended coda identifications are higher for the naturally produced productions than for TTS. Moreover, listeners systematically misidentified voiced codas as voiceless in TTS words. Overall, this study extends previous literature on speech intelligibility at the intersection of speech synthesis and contrast neutralization. TTS voices tend to neutralize salient phonetic cues present in natural speech. Subsequently, listeners are less able to identify phonological distinctions in TTS. We also discuss how investigating which cues are more salient in natural speech can be beneficial in synthetic speech generation to make them more natural and also easier to perceive.

Partial compensation for coarticulatory vowel nasalization across concatenative and neural text-to-speech.

This study investigates the perception of coarticulatory vowel nasality generated using different text-to-speech (TTS) methods in American English. Experiment 1 compared concatenative and neural TTS using a 4IAX task, where listeners discriminated between a word pair containing either both oral or nasalized vowels and a word pair containing one oral and one nasalized vowel. Vowels occurred either in identical or alternating consonant contexts across pairs to reveal perceptual sensitivity and compensatory behavior, respectively. For identical contexts, listeners were better at discriminating between oral and nasalized vowels in neural than in concatenative TTS for nasalized same-vowel trials, but better discrimination for concatenative TTS was observed for oral same-vowel trials. Meanwhile, listeners displayed less compensation for coarticulation in neural than in concatenative TTS. To determine whether apparent roboticity of the TTS voice shapes vowel discrimination and compensation patterns, a "roboticized" version of neural TTS was generated (monotonized f0 and addition of an echo), holding phonetic nasality constant; a ratings study (experiment 2) confirmed that the manipulation resulted in different apparent roboticity. Experiment 3 compared the discrimination of unmodified neural TTS and roboticized neural TTS: listeners displayed lower accuracy in identical contexts for roboticized relative to unmodified neural TTS, yet the performances in alternating contexts were similar.

Trust influences perceptions of virtual humans, but not necessarily learning

Researchers have investigated the extent to which features of virtual humans (e.g., voice) influence learning. However, only recently have researchers began to question how trust influences perceptions of virtual humans and learning with a virtual human. In this study, we use unsupervised machine learning (k-means clustering) to examine the extent to which learners trust a virtual human with different voices. General trust attitudes toward a virtual human were measured using a validated questionnaire after participants watched videos of the virtual human presenting educational information. The results indicated that a virtual human with a low-quality text-to-speech (TTS) voice had the most variation in perceived trust, followed by a high-quality TTS voice and a recorded human voice. In addition, trust significantly influenced perceptions of the virtual human persona in the TTS conditions, whereas it had little influence on measures of persona in the recorded human voice condition. Finally, trust had little influence on learning outcomes in any of the conditions. While previous findings have shown that trust can vary between voice conditions, the results presented here show that different types of voices led to different trust responses even within voice conditions, raising important implications for instructional designers.

Building Statistical Parametric Multi-speaker Synthesis for Bangladeshi Bangla

We present a text-to-speech (TTS) system designed for the dialect of Bengali spoken in Bangladesh. This work is part of an ongoing effort to address the needs of new under-resourced languages. We propose a process for streamlining the bootstrapping of TTS systems for under-resourced languages. First, we use crowdsourcing to collect the data from multiple ordinary speakers, each speaker recording small amount of sentences. Second, we leverage an existing text normalization system for a related language (Hindi) to bootstrap a linguistic front-end for Bangla. Third, we employ statistical techniques to construct multi-speaker acoustic models using Long Short-term Memory Recurrent Neural Network (LSTM-RNN) and Hidden Markov Model (HMM) approaches. We then describe our experiments that show that the resulting TTS voices score well in terms of their perceived quality as measured by Mean Opinion Score (MOS) evaluations.

안드로이드 플랫폼 기반 시각장애인용 음성도서 서비스 모델 구축 및 평가

This paper describes the process and methodology followed in developing the Android-based LG Sangnam Audiobook service and an evaluation of its usefulness to the public. The methods included a survey of user needs, analysis of usage statistics, and user interviews. The study found that visually impaired users: 1) were greatly interested and willing to use smartphones if there were no barrier in cost and access; 2) preferred downloads to streaming services; 3) did not mind performance differences between real and TTS (text-to-speech) voices; 4) showed marked differences in book preferences according to age, 5) made about 14,000 downloads in 2014; and 6) indicated bookmarking and moving between pages and tables of content as the most important functions in using audiobooks.

Designing interfaces for multiple-goal environments

Designing computer-human interfaces for multiple-goal environments is challenging because people pursue multiple goals with conflicting priorities. Safety-critical environments, such as driving, aggravate the need for a more nuanced understanding of interfaces that may reconcile conflicting tasks. Speech interfaces are prime examples of such interfaces. In this article, we investigate how design variations of an in-vehicle speech interface influence performance of a primary task (driving safely) and a secondary task (e-mailing). In a controlled experiment, we test the performance implications of using single computer-generated Text-To-Speech (TTS) voice and multiple matching TTS voices while users respond to e-mails with varying levels of complexity during driving. Our results indicate that the number of voices used has a significant effect on both driving performance and handling e-mail--related activities. We discuss potentially unintended consequences of making the interface too naturalistic and too engaging for the driver and conclude with theoretical and practical implications.

SYSTEM AND METHOD FOR BLENDING SYNTHETIC VOICES

A system and method for generating a synthetic text-to-speech TTS voice are disclosed. A user is presented with at least one TTS voice and at least one voice characteristic. A new synthetic TTS voice is generated by blending a plurality of existing TTS voices according to the selected voice characteristics. The blending of voices involves interpolating segmented parameters of each TTS voice. Segmented parameters may be, for example, prosodic characteristics of the speech such as pitch, volume, phone durations, accents, stress, mis-pronunciations and emotion.

On the intelligibility of fast synthesized speech for people who are blind: A cross-system comparison.

People who are blind increasingly use synthesized speech as a primary output modality when interacting with computers, mobile devices, and web-based services. They typically prefer to listen to the synthesized speech at speeds multiple times real time, and consequently develop strong preferences for particular text-to-speech (TTS) engines and voices. These usage-based preferences lead to potentially incorrect assumptions about which TTS approaches are “best” for fast speech. We report on a cross-system comparison of the intelligibility of fast synthesized speech for users who are blind from birth. We used male and female voices from multiple TTS engines representing the main approaches to TTS: diphone synthesis, unit selection synthesis, HMM-based synthesis, and formant synthesis. We recruited participants from organizations that work with the blind. Each participant listened to and transcribed semantically underspecified sentences from a single TTS engine and voice, spoken at speeds ranging from 300 to 550 words/min. We used TTS engine, algorithm for speeding up synthesized speech, and speaker sex as independent variables, and transcription accuracy as the primary dependent variable. We report differences by TTS approach, speed up method, TTS engine type, and speaker sex.

Using TTS voices to develop audio materials for listening comprehension: A digital approach

AbstractThis paper reports a series of experiments with text‐to‐speech (TTS) voices. These experiments have been conducted to develop audio materials for listening comprehension as an alternative technology to traditionally used audio equipment like the compact cassette. The new generation of TTS voices based on unit selection synthesis provides impressive naturalness and intelligibility. In the experiments, TTS‐generated audio files have been put to use in the classroom procedure and listening tests. So far, these endeavours have yielded positive results. The paper also evaluates the synthesised speech involved in the experiments and discusses problems that arise in the application.

Effect of Speaker and Sampling Rate on Mos-X Ratings of Concatenative TTS Voices

The MOS-X is a recently-developed questionnaire used to evaluate the quality of artificial speech. In this experiment, participants listened to audio files produced by concatenative text-to-speech voices for the purpose of assessing the effect of Speaker and Sampling Rate on MOS-X ratings. The concatenative voices were developed from recordings of three different human speakers (code named AF, AM, and B) and produced using two different sampling rates (8 kHz and 22 kHz). Six independent groups of raters participated, one group for each combination of speaker and sampling rate. Analyses of variance indicated a significant main effect of Voice, but no significant main effect of Sampling Rate and no significant Voice by Sampling Rate interaction. The results indicate that independent groups of raters are sensitive to speaker differences in concatenative text-to-speech (TTS) voices, but not to differences in these sampling rates.

Acceptability of variations in question intonation in natural and synthesized American English

In a previous study exploring American English question intonation, we found that some speakers deviated considerably from expected question prosody. In this study, we focus on listener-rated acceptability of the various prosodic patterns observed for yes/no and wh questions. A variety of intonational patterns realized in both question utterances recorded from five female and three male professional speakers and in questions synthesized from several TTS voices of both genders was presented to listeners. Subjects judged the acceptability of each utterance in the context of a dialogue between a travel agent and customer. We hypothesized that question utterances with the expected intonational features (phrase-final fall in wh questions, phrase-final rise in yes/no questions) would be rated as more acceptable than question utterances with deviating intonational features, and that this result would hold for both natural and synthetic speech conditions. In addition, following our previous results, we hypothesized that the unexpected intonation pattern of phrase-final falls for yes/no questions would be more acceptable for lower-pitched than for higher-pitched voices. We also varied the prominence of the interrogative pronouns in synthetic wh questions in order to see whether simulating their high intonational prominence in natural wh questions improved the acceptability of synthetic wh questions.

Voice selection for speech synthesis

A TTS voice quality experiment was conducted to select a speaker and to evaluate synthesis techniques. Small-scale TTS diphone inventories using six professional female speakers who were pre-selected in an audition were recorded. Two types of inventories were recorded for each speaker: a series of nonsense words and a series of English sentences. Using these 12 inventories, two synthesis methods were compared: PSOLA [Charpentier and Moulines, Eurospeech ’89] and Harmonic Plus Noise (HNM) [Stylianou et al., Eurospeech ’97]. Synthetic prosody closely modeled naturally spoken versions of the target utterances. Three fully synthetic (TTS) and two hybrid (i.e., partly recorded from the human speaker and partly synthesized) sentences formed the experimental stimuli for subjective testing. For references, two MNRU versions of the naturally spoken sentences were used: (a) Q10 (resembling low-end commercial 16-kbps encoded speech) and (b) Q35 (resembling high-quality telephone speech). Forty-one subjects rated intelligibility [I], naturalness [N], and pleasantness [P] on five-point MOS scales. A total of 936 ratings were collected from each subject. Repeated measures of analyses of variance (ANOVAs) were performed on the data. There were significant main effects of speaker, synthesis method, and inventory, plus interactions. It was found that (1) the best speaker consistently outperformed the others on all three rating scales; for the optimal combination of parameters, TTS ratings ranged (across speakers) as follows: [I] 3.64–2.94, [N] 3.36–2.7, [P] 3.34–2.53. (2) HNM outperformed PSOLA (consistently 0.25 points higher for [I], [N], [P] scores), and (3) the diphone inventory extracted from sentences was preferred over that extracted from nonsense words (with a significantly smaller difference of 0.10 for HNM than 0.19 for PSOLA).

Text-To-Speech Technology for Dual Party Relay Services

We evaluated several human factors issues involved in applying AT&T Text-to-Speech (TTS) technology to Dual Party Relay (DPR) services for hearing- or speech-impaired end-users. Covered in this paper are: 1) lexical studies that were conducted to gather abbreviations and typographical errors generated by users of the Telecommunications Device for the Deaf (TDD); and 2) a laboratory trial that was performed to evaluate the feasibility of the TTS application and to examine associated user interface issues. Information gathered from the lexical studies was used for the laboratory trial and for constructing the field trial system. The laboratory trial consisted of several studies, including a comparison between the comprehension of TTS and human voices, identification of a desirable TTS presentation rate, examination of the effects of pauses, and observation of the difficulties related to TTS relay calls. Recommendations were made based on the lab trial results. We concluded that applying TTS to DPR service is feasible. That is, hearing and hearing-impaired subjects can carry on DPR conversations with the use of TTS technology. A field trial was subsequently conducted in California.