Quality Of Synthetic Speech Research Articles

A new speech synthesis algorithm for high quality TTS systems: a mixed phase vocoder

AbstractCurrently, time domain pitch synchronous overlap‐add (TD‐PSOLA) is the most popular synthesis algorithm for text‐to‐speech (TTS) systems. The algorithm produces very high quality synthetic speech, particularly when a pitch modification factor is small. However, as the pitch modification factor becomes larger, the quality degradation due to a slight pitch epoch detection error becomes severe. On the other hand, the vocoder framework has very flexible prosody manipulation. It can obtain a uniform voice quality over a wide range of prosody modification. Unfortunately, the synthesized speech quality from the vocoder is far from natural human speech, often showing buzzy quality. To remedy buzzy quality from the vocoder and make more natural synthetic speech, we propose a new speech synthesis algorithm for high quality TTS systems that is based on the homomorphic vocoder framework. The impulse response of vocal tract is obtained by mixing the minimum phase in lower frequency band and original phase in higher frequency band—thus, the name is a mixed phase vocoder. Informal subjective listening tests reveal that the mixed phase vocoder is a good candidate for TTS synthesis with high intelligibility and naturalness. Copyright © 2004 AEI.

Text-to-speech from concatenation of articulatory units derived from natural speech

It has been conjectured that articulatory synthesis possesses the greatest potential for generating high quality synthetic speech. However, for text-to-speech (TTS), waveform concatenation techniques have proven more practical due in part to the challenge of generating appropriate trajectories of articulatory parameters. A waveform generation method for TTS that combines the practical success of concatenative methods with the quality potential of articulatory synthesis is under development. The system concatenates articulatory units derived from natural speech using an articulatory voice mimic. The mimic estimates articulatory parameters by minimizing a cost function that includes a spectral distance between natural and synthetic speech and a geometric distance that penalizes rapid or discontinuous changes in articulator positions. A database of articulatory trajectories representing phonetic units is constructed from the estimated parameters. For TTS, phonetic units generated by text analysis are used to select the corresponding articulatory units from the database. Duration modification, concatenation, and smoothing across units are performed in the articulatory domain resulting in a single articulatory trajectory for the complete utterance. Speech is synthesized from the trajectory using a two mass model for voicing, achieving a high degree of acoustic continuity across unit boundaries while also allowing for source–tract interaction.

Evaluating the Quality of an Integrated Model of German Prosody

The perceived quality of synthetic speech strongly depends on its prosodic naturalness. Departing from earlier works by Mixdorff on a linguistically motivated model of German intonation based on the Fujisaki model, an integrated approach to predicting F0 along with syllable duration and energy was developed. The current paper first presents some statistical results concerning the relationship between linguistic and phonetic information underlying an utterance and its prosodic features. These results were employed for training the MFN-based integrated prosodic model predicting syllable duration and energy along with syllable-aligned Fujisaki control parameters. The paper then focusses on the method of perceptual evaluation developed, comparing resynthesis stimuli created by controlled prosodic degrading of natural speech with stimuli created using the integrated model. The results indicate that the integrated model generally receives better ratings than degraded stimuli with comparable durational and F0 deviations from the original. An important outcome is the observation that the accuracy of the predicted syllable durations appears to be a stronger factor with respect to the perceived quality than the accuracy of the predicted F0 contour.

A Text-to-Speech Platform for Variable Length Optimal Unit Searching Using Perception Based Cost Functions

In concatenative Text-to-Speech, the size of the speech corpus is closely related to synthetic speech quality. In this paper, we describe our work on a new corpus-based Bell Labs' TTS system. This encompasses large acoustic inventories with a rich set of annotations, models and data structures for representing and managing such inventories, and an optimal unit selection algorithm that accommodates a broad range of possible cost criteria. We also propose a new method for setting weights in the cost functions based on a perceptual preference test. Our results show that this approach can successfully predict human preference patterns. Synthetic speech using weights determined in this manner consistently demonstrates smoother transitions and higher voice quality than speech using manually set weights.

Intelligibility tests for synthetic speech subjective evaluation: The semantically unpredictable sentences approach for European Portuguese

In this paper a test proposal for European Portuguese (EP) synthetic speech quality evaluation as well as some of its application are presented. For this purpose we have built a semantically unpredictable sentences (SUS) corpus of 25 sentences with different lengths that were chosen according to the standard EP phonological, syllabic and prosodic specific features. The goal is to create a standard test linguistically focused that constitutes a base for a more accurate comparison and ranking of different synthesis techniques. The tests were conducted in two distinct conditions: one performed in a controlled environment with listeners ranging from 16 to 60 years old; the other performed through the web by a large group of listeners of different ages, each one in their own environment. The listeners had only one chance of listening to each sentence. Of course in the second case this could not be controlled. The evaluation parameters considered were the number of (1) totally well identified sentences; (2) nonidentified sentences; (3) misunderstood words in each sentence; (4) misunderstood words in the whole corpus; (5) misunderstood words according to the length of the sentence; (6) errors in the perception of a word according to its position in the sentence.

Challenges and Rewards in Using Parametric or Concatenative Speech Synthesis

Highest quality synthetic voices remain scarce in both parametric synthesis systems and in concatenative ones. Much synthetic speech lacks naturalness, pleasantness and flexibility. While great strides have been made over the past few years in the quality of synthetic speech, there is still much work that needs to be done. Now the major challenges facing developers are how to provide optimal size, performance, extensibility, and flexibility, together with developing improved signal processing techniques. This paper focuses on issues of performance and flexibility against a background containing a brief evolution of speech synthesis; some acoustic, phonetic and linguistic issues; and the merits and demerits of two commonly used synthesis techniques: parametric and concatenative. Shortcomings of both techniques are reviewed. Methodological developments in the variable size, selection and specification of the speech units used in concatenative systems are explored and shown to provide a more positive outlook for more natural, bearable synthetic speech. Differentiating considerations in making and improving concatenative systems are explored and evaluated. Acoustic and sociophonetic criteria are reviewed for the improvement of variable synthetic voices, and a ranking of their relative importance is suggested. Future rewards are weighed against current technical and developmental challenges. The conclusion indicates some of the current and future applications of TTS.

Evaluation of synthetic speech quality: A comparative study of several computer-based speech synthesizers

This paper examines some versions of Lithuanian and Russian synthetic speech intelligibility and Lithuanian, Russian, Hungarian and Italian synthetic speech acceptability. The speech of both Russian and Lithuanian speaker is more intelligible than Russian or Lithuanian synthesis. Previous version of Russian synthesis is worse than Lithuanian and improved Russian synthesis (IRS). Study of characteristics of IRS sounds shows two opposite tendencies - according to the general quantity of mistake reduction this version is tending towards the natural speech, but according to the homogeneity of mistakes, it moves away. As the first tendency is clearly dominant, the general resultant in the new version shows a tend to improve.Correlation between intelligibility and acceptability of IRS deals possibility of small progress towards the natural speech. The IRS is more acceptable to subjects than previous version. The old synthesis is viewed as a rather decent instance of a robot's speech, while the IRS - as a poor variant of human speech.Acceptability studies showed natural speech more enjoyed by Hungarian listeners and more critical by Italian. All versions of synthetic speech were judged as less acceptable than natural but after improvement most of listeners changed their mind.

Unit Generation Based on Phrase Break Strength and Pruning for Corpus-Based Text-to-Speech

This paper discusses two important issues of corpus-based synthesis: synthesis unit generation based on phrase break strength information and pruning redundant synthesis unit instances. First, the new sentence set for recording was designed to make an efficient synthesis database, reflecting the characteristics of the Korean language. To obtain prosodic context sensitive units, we graded major prosodic phrases into 5 distinctive levels according to pause length and then discriminated intra-word triphones using the levels. Using the synthesis unit with phrase break strength information, synthetic speech was generated and evaluated subjectively. Second, a new pruning method based on weighted vector quantization (WVQ) was proposed to eliminate redundant synthesis unit instances from the synthesis database. WVQ takes the relative importance of each instance into account when clustering similar instances using vector quantization (VQ) technique. The proposed method was compared with two conventional pruning methods through objective and subjective evaluations of synthetic speech quality: one to simply limit the maximum number of instances, and the other based on normal VQ-based clustering. For the same reduction rate of instance number, the proposed method showed the best performance. The synthetic speech with reduction rate 45% had almost no perceptible degradation as compared to the synthetic speech without instance reduction.

A dynamical system model for generating fundamental frequency for speech synthesis

Higher quality speech synthesis is required for widespread use of text to-speech (TTS) technology, and prosody is one component of synthesis technology with the greatest need for improvement. This paper describes a new approach to generation of two important cues to prosodic patterns-fundamental frequency (F/sub 0/) and energy contours-given symbolic prosodic labels and text. Specifically, the approach represents vectors of F/sub 0/ and energy with a dynamical system model, which allows automatic estimation of parameters from labeled speech. Parameters at different time scales in the model are structured to capture segment, syllable, phrase and discourse level effects based on linguistic research. F/sub 0/ generation experiments with the dynamical system model show improved synthetic speech quality over the hybrid target/filter approach.

Comparing the naturalness of several approaches for generating F0 contours in German text-to-speech systems

Generating near-to-natural F0 contours is an important issue in text-to-speech synthesis and contributes vastly to the quality of synthetic speech. In earlier studies by the authors, a model of German intonation was developed that is based on the quantitative Fujisaki model. A typical F0 contour is described as a sequence of major rises and falls, which are modeled by onsets and offsets of accent commands connected to accented syllables. The current paper addresses perception experiments comparing the intonational naturalness of a Fujisaki-model-based TTS and four other German TTS systems with comparably high segmental quality. Natural speech samples were used as a reference. Three of the TTS systems had PSOLA, and one LPC segmentals. Two types of experiments were conducted with 20 subjects: (1) a pair comparison of 15 isolated sentences, (2) a ranking test based on a news passage of about 15 sec produced with each of the systems. Preliminary results from experiment (1) show, that on a naturalness scale from 0 to 5, the natural speech samples reach a maximum score of 4.5, with values of 2.8 for the best synthesis, the LPC-based one. The system with Fujisaki-model-based intonation leads the group of PSOLA systems, which is closely clustered at a mean of 2.4.

Automatic creation of CV templates for formant type speech synthesis based on HMM-based segmentation and syllable boundary detection

A formant type speech synthesis method has an enormous advantage in that it allows one to generate speech with various voice quality variations and talker individualities. But it has suffered from unnatural speech quality, not because of theoretical limitations but because of an incomplete set of rules for the synthesis. Any insufficient approximation to the acoustics causes degradation of the perceived quality of synthetic speech. A novel formant type speech synthesizer in Japanese based on concatenation of CV (consonant-vowel) formant-source templates obtained from natural utterances has been investigated, in which multiple sets of formant and voice source parameter values are used for each of the CV syllables. This paper describes an automatic method to create the CV formant-source templates from speech corpus. The ARX (autoregressive with exogenous input) analysis method is first used to automatically extract formant and voice source parameters and then an HMM based segmentation is performed to locate the CV segments. The segments are further analyzed to detect a starting point of the syllable. A distance measure is used to decide the number of templates needed for each of the syllables. The method is proved to be useful in creating CV templates by experiments performed on 503 Japanese sentences.

Quality enhancement of sinusoidal transform vocoders

The authors present quality enhancement of sinusoidal transform coders (STCs) via the development of parametric models. The benefits of the Bark spectrum are explored for use in the design of perceptual coding of the sine-wave amplitudes. According to the results, the proposed approach provides a uniform perceptual fit across the spectrum. To enhance the accuracy of phase representation, noncausal all-pole modelling of the vocal system is also discussed. Experimental results indicate that the use of the developed new parametric models allows the STC to improve the phase accuracy as well as the synthetic speech quality.

TFW-PM reference for subjectively assessing the quality of synthetic speech

This paper proposes a set of speech samples generated by time and frequency warping based on phase modulation (TFW-PM) for use as references in subjective assessments of synthetic speech, and describes its effectiveness. Their qualities depend on three variable parameters: the number of PARCOR synthesis coefficients, the modulation factor, and the frame length. Repeated assessment tests for four kinds of synthetic speech using the TFW-PM reference generated by combining those parameters showed that the modulation factor (Δθ) had the largest effect on the TFW-PM speech quality. Each optimum value corresponding to the other two parameters was determined based on the minimum deviation criterion of the equivalent Δθ converted from the mean opinion score (MOS) of the synthetic speech via the relationship between Δθ and MOS of the reference. In addition, TFW-PM speech samples and modulated noise reference (MNR) were examined to see whether or not synthetic speech qualities in terms of equivalent Δθ were stable from three viewpoints: repeated testing, subjects divided into groups familiar with and unfamiliar with synthetic speech, and listening environments. The results show that the equivalent Δθ is more stable for all three criteria than the customary equivalent Q. © 1998 Scripta Technica. Electron Comm Jpn Pt 1, 81(1): 59–67, 1998

Voiced speech excitation synthesis using a sinusoidal model

A modification to the ‘shape-invariant’ sinusoidal speech model is proposed, whereby the phases of the component sinewaves used for the excitation are made to add coherently at each glottal closure. Applied to pitch and time-scale modification, higher quality synthetic speech is produced when large changes are required.

Prosodic Phrasing and Comprehension

From previous research we know that prosodic features are perceptually effective in marking boundaries and that a suitable implementation of these features improves the quality of synthetic speech in terms of acceptability. It can further be assumed that listeners use the perceived prosodic information to compute the meaning of the input speech. This paper, therefore, investigates and determines whether a well-phrased utterance, (that is, an utterance with prosodic boundaries in appropriate positions and with appropriate realizations), is easier to comprehend than a poorly-phrased one. To measure this, we designed a method in which a kind of verification task is combined with a question-answering task (“monitoring for the answer”). The stimulus set consisted of structurally ambiguous sentences. The expectation was that when listeners hear a question followed by an appropriately phrased utterance, they will react more rapidly than when the question is followed by an utterance with neutral phrasing. Also, it was expected that in the latter situation reaction times (RTs) will be shorter than if an inappropriately phrased utterance is presented. The results confirmed the expectations: an appropriately phrased utterance always produced the fastest RTs.

Measuring the naturalness of synthetic speech

Even the highest quality synthetic speech generated by rule sounds unlike human speech. As the intelligibility of rule-based synthetic speech improves, and the number of applications for synthetic speech increases, the naturalness of synthetic speech will become an important factor in determining its use. In order to improve this aspect of the quality of synthetic speech it is necessary to have diagnostic tests that can measure naturalness. Currently, all of the available metrics for evaluating the acceptability of synthetic speech do not distinguish sufficiently between measuring overall acceptability (including naturalness) and simply measuring the ability of listeners to extract intelligible information from the signal. In this paper we propose a new methodology for measuring the naturalness of particular aspects of synthesized speech, independent of the intelligibility of the speech. Although naturalness is a multidimensional, subjective quality of speech, this methodology makes it possible to assess the separate contributions of prosodic, segmental, and source characteristics of the utterance. In two experiments, listeners reliably differentiated the naturalness of speech produced by two male talkers and two text-to-speech systems. Furthermore, they reliably differentiated between the two text-to-speech systems. The results of these experiments demonstrate that perception of naturalness is affected by information contained within the smallest part of speech, the glottal pulse, and by information contained within the prosodic structure of a syllable. These results show that this new methodology does provide a solid basis for measuring and diagnosing the naturalness of synthetic speech.

Speech analysis/synthesis and modification using an analysis-by-synthesis/overlap-add sinusoidal model

Sinusoidal modeling has been successfully applied to a broad range of speech processing problems, and offers advantages over linear predictive modeling and the short-time Fourier transform for speech analysis/synthesis and modification. This paper presents a novel speech analysis/synthesis system based on the combination of an overlap-add sinusoidal model with an analysis-by-synthesis technique to determine the model parameters. It describes this analysis procedure in detail, and introduces an equivalent frequency-domain algorithm that takes advantage of the computational efficiency of the fast Fourier transform (FFT). In addition, a refined overlap-add sinusoidal model capable of shape-invariant speech modification is derived, and a pitch-scale modification algorithm is defined that preserves speech bandwidth and eliminates noise migration effects. Analysis-by-synthesis achieves very high synthetic speech quality by accurately estimating the component frequencies, eliminating sidelobe interference effects, and effectively dealing with nonstationary speech events. The refined overlap-add synthesis model correlates well with analysis-by-synthesis, and modifies speech without objectionable artifacts by explicitly controlling shape invariance and phase coherence. The proposed analysis-by-synthesis/overlap-add (ABS/OLA) system allows for both fixed and time-varying time-, frequency-, and pitch-scale modifications, and computational shortcuts using the FFT algorithm make its implementation feasible using currently available hardware.

Good prosody facilitates comprehension

Previous research showed that prosodic features are perceptually conspicuous in marking boundaries and that a suitable implementation of these features improves the quality of synthetic speech in terms of acceptability. Assumably, listeners use prosody to compute the informational structure of the input speech. The hypothesis is that differently phrased utterances may lead to differences in cognitive load during comprehension. A method was designed in which a kind of verification task is combined with a question-answering task. The stimulus set consisted of structurally ambiguous sentences. The expectations were as follows: (a) when listeners receive a question followed by an appropriately phrased utterance, they will react more rapidly than when it is followed by an utterance without phrasing; (b) in the latter situation reaction times (RTs) will still be shorter than if an inappropriately phrased utterance is presented. The results confirmed these expectations for the situation where the prosodically intended reading corresponded to the most likely interpretation of the ambiguous sentences. However, when the least likely interpretation was the prosodically suggested one, it appeared that a question followed by an inappropriately phrased utterance gave the same RTs as one without phrasing. But an appropriately phrased utterance still produced the fastest RTs.

A neuronal formant synthesizer

Speech synthesis by rule has made considerable advances and it is being used today in numerous text-to-speech synthesis systems. Current systems are able to synthesise pleasant-sounding voices at high intelligibility levels. However, because their synthetic speech quality is still inferior to that of fluently produced human speech it has not found wide acceptance and instead it has been restricted mainly in useful applications for the handicapped or for restricted tasks in telecommunications. The problems with automatic speech synthesis are related to the methods of controlling speech synthesizer models in order to mimic the varying properties of the human speech production system during discourse. In this paper, artificial neural networks are developed for the control of a formant synthesizer. A set of common words comprising of larynx-produced phonemes were analysed and used to train a neural network cluster. The system was able to produce intelligible speech for certain phonemic combinations of new and unfamiliar words.

Analysis of quality factors in synthetic speech produced by rules

This paper investigates how various factors affect the quality of synthetic speech produced by rules. Using rules to synthesize speech will be an important technique for providing various telecommunication services in future intelligent networks. The quality of synthetic speech is generally measured by subjectively evaluating the speech from the viewpoint of intelligibility, or by comparing it with the quality of other types of synthetic speech. However, the development of a practical speech synthesis method for use in telecommunication networks requires an overall quality evaluation, including intelligibility and naturalness. The quality should be compared with that of natural telephone speech. To establish an overall quality evaluation method, the effects of several factors on the overall quality (expressed by MOS) of speech synthesized by several Japanese text-to-speech systems are quantitatively compared with the effects of using additive speech-correlated white noise as a natural speech material. Experimental results show that such factors as subject, listening experience, average pitch frequency, and text affect synthetic speech more than natural speech. Quality evaluation characteristics due to these factors are discussed and an overall quality evaluation method for synthetic speech is proposed.