Klatt Synthesizer Research Articles

Re-testing theories of vowel inherent spectral change

Three basic models of vowel inherent spectral change have been proposed: onset + offset, onset + direction, and onset + slope (or spectral rate of change). Morrison (2013) presents three theoretical stimuli whose identification rates relative to each other should lend weight to one of these hypotheses: (i) canonical duration and canonical onset-to-offset trajectory, (ii) canonical duration with double trajectory length, and (iii) canonical trajectory with half duration. The bulk of previous work supports onset + offset. In the present study, eight English monophthongs were synthesized with the Klatt synthesizer of the Berkeley Phonetics Machine. Three versions of each vowel were synthesized, corresponding to the theoretical stimuli of Morrison (2013). These 24 stimuli were used in two studies (n= 18 for each). First was a vowel identification task, where the choices were [hVd] words with the chosen vowels ( heed, hid, etc.). Second was a goodness rating task with a 7-point Likert scale, where participants were played a vowel stimulus while being shown the orthographic [hVd] word containing that vowel. The results do not fully support any one theory, but they align most closely with the onset + direction hypothesis, with onset + offset also being supported.

Genetic algorithm to estimate the input parameters of Klatt and HLSyn formant-based speech synthesizers

Voice imitation basically consists in estimating a synthesizer's input parameters to mimic a target speech signal. This is a difficult inverse problem because the mapping is time-varying, non-linear and from many to one. It typically requires considerable amount of time to be done manually. This work presents the evolution of a system based on a genetic algorithm (GA) to automatically estimate the input parameters of the Klatt and HLSyn formant synthesizers using an analysis-by-synthesis process. Results are presented for natural (human-generated) speech for three male speakers. The results obtained with the GA-based system outperform those obtained with the baseline Winsnoori with respect to four objective figures of merit and a subjective test. The GA with Klatt synthesizer generated similar voices to the target and the subjective tests indicate an improvement in the quality of the synthetic voices when compared to the ones produced by the baseline.

Utterance copy through analysis-by-synthesis using genetic algorithm

Utterance copy consists in estimating the input parameters to reconstruct a speech signal using a speech synthesizer. This process is distinct from the more traditional text-to-speech but yet used in many areas, especially in linguistics and health. Utterance copy is a difficult inverse problem because the mapping is non-linear and from many to one. It requires considerable amount of time to manually perform utterance copy and automatic methods, such as the one proposed here, are of interest. This work presents our system based on genetic algorithm (GA) to automatically estimate the input parameters of the Klatt synthesizer using an analysis-by-synthesis process. Results are presented for synthetic (computer-generated) and natural (human-generated) speech, for male and female speakers. These results are compared with the ones obtained with WinSnoori, the only currently available software that performs the same task. The experiments showed that the proposed newGASpeech system is an effective alternative to the laborious manual process of estimating the input parameters of a Klatt synthesizer. And it outperforms the baseline by a large margin with respect to five objective figures of merit. For example, in average, the mean squared error is reduced to approximately 60.4 % and 75.2 % when natural target voices from male and female speakers are used, respectively.

Statistical Formant Speech Synthesis for Arabic

This work constructs a hybrid system that integrates formant synthesis and context-dependent Hidden Semi-Markov Models (HSMM). HSMM parameters comprise of formants, fundamental frequency, voicing/frication amplitude, and duration. For HSMM training, formants, fundamental frequency, and voicing/frication amplitude are extracted from waveforms using the Snack toolbox and a decomposition algorithm, and duration is calculated using HMM modeled by multivariate Gaussian distribution. The acoustic features are then generated from the trained HSMM models and combined with default values of complementary acoustic features such as glottal waveform parameters to produce speech waveforms utilizing the Klatt synthesizer. We construct the text processor for phonetic transcription required at the training and synthesis phases by utilizing phonemic pronunciation algorithms. A perceptual test reveals that the statistical formant speech text-to-speech system produces good-quality speech while utilizing features that are small in dimension and close to speech perception cues.

Measurement of formants in synthetic vowels

The measurement of formant frequencies of vowels is among the most common measurements in speech studies, but is known to be biased by the particular fundamental frequency (F0) exciting the formants, and to be inaccurate for formants close together or for speakers using a high F0. To allow a comparison across multiple measurement techniques, vowels were synthesized using the Klatt synthesizer with known formant values. The synthetic vowels were constructed with five different F1 values and nine different F0 values; formant bandwidths, and higher formant frequencies, were constant. The F0s varied in such a way that the most intense harmonic in F1 or F2 either matched the center frequency or deviated in the range of 3–87 Hz. Manual measurements by four subjects were compared to automatic measures using the LPC Burg algorithm, LP closed-phase covariance, and spectra smoothed cepstrally or by averaging repeated DFT's. Formants were also measured from pruned reassigned spectrograms. Error patterns differ among the methods, but most tracked the frequency of the most intense harmonic; the smallest errors occur with closed-phase covariance and reassigned spectrogram. Implications for such measures on vowels in isolated words of real speech are discussed. [Work supported by NIH-NIDCD grant DC-002717.]

The influence of amplitude modulation depth on perceived roughness of vowels

Previous research has shown that roughness perception for vowels is influenced by the waveform amplitude modulation. It was observed that sinusoidal modulation (constant modulation depth of 100%) between 20 Hz and 70 Hz has the greatest impact on perceived roughness of vowels [Shrivastav & Eddins, 2011; JASA, 129, 2661]. The present experiment examined the effects of modulation depth on the perception of roughness for two of 10 vowels used by Shrivastav & Eddins (2011). Synthetic copies of two vowel stimuli selected from the Sataloff/Heman-Ackah database were generated using a Klatt synthesizer. Sinusoidal amplitude modulation was superimposed on each vowel using three modulation frequencies (20 Hz, 30 Hz, and 45 Hz) and five modulation depths (0, -5, -10, -15, and -20 dB). Listeners judged the perceived roughness of each of the 30 speech stimuli (2 talkers x 3 modulation frequencies X 5 modulation depths) by matching it to a comparison sound. The comparison sound consisted of a sawtooth wave, mixed with speech-shaped noise, and amplitude modulated with a raised (power of 4) cosine wave. Results will show how depth of amplitude modulation affects perceived roughness and will help to develop predictive models to quantify roughness. Research funded by NIH.

Developing a Single Comparison Stimulus for Matching Breathy Voice Quality

In this experiment, a single comparison stimulus was developed as a reference in a perceptual matching task for the quantification of breathy voice quality. Perceptual judgments of a set of synthetic voice samples were compared to previous data obtained using multiple comparison stimuli "customized" for different voices (Patel, Shrivastav, & Eddins, 2010). Five male and 5 female samples of the vowel /a/ were selected from the Kay Elemetrics Disordered Voice Database and resynthesized using a Klatt synthesizer. Eleven samples were created for each base voice by manipulating the aspiration noise level. Five samples from each continuum were evaluated in a perceptual matching task in which a single sawtooth and noise comparison stimulus was used to obtain breathiness judgments. Linear regression was used to compare measurements obtained using the new comparison stimulus against the customized comparison stimuli. Results indicated that the noncustomized sawtooth comparison provides reliability and perceptual distances between stimuli similar to those obtained using customized comparison stimuli. A single-variable matching task using a single comparison stimulus can be used to obtain perceptual estimates of breathiness across voices and experiments in a laboratory setting. This technique will help develop models of voice-quality perception.

The influence of amplitude modulation frequency on perceived roughness of vowels.

The goal of the current experiment is to determine how perceived roughness of vowels is influenced by the frequency of inherent amplitude modulation. A set of 10 vowel stimuli was synthesized using a Klatt synthesizer and were modeled after ten talkers selected from the Satloff/Heman‐Ackah disordered voice database. These vowels were selected using stratified sampling procedures to represent a wide range of roughness. The vowel f0 and the first three formants were determined and used to generate a synthetic copy of these vowels. The aspiration noise level was adjusted to subjectively match the overall quality of the talkers. To each of the ten prototypical vowels, sinusoidal amplitude modulation was superimposed with a modulation depth of −6 dB and modulation frequencies ranging from 10 to 70 Hz. Listeners judged roughness in a matching task where the modulation depth of a comparison sound was adjusted to match the perceived roughness of each of the 70 speech stimuli (10 talkers × 7 modulation frequencies). The dependence of perceived roughness on amplitude modulation frequency will be compared to the perceived roughness of sinusoidal carriers as reported by Fastl and Zwicker (Springer, New York, 1997).

An effect of phoneme frequency on stop place perception by English-speaking and Japanese-speaking listeners.

The effects of phoneme frequency on stop place perception were examined. In English, [t] is more frequently observed than [k] while the opposite is true in Japanese. If the sound frequency affects the phoneme perception, English listeners would identify more of the ambiguous [t]-[k] stimuli as “t” than do Japanese listeners. In this study, a 4IAX discrimination experiment and a phoneme boundary decision experiment were conducted with English-speaking and Japanese-speaking listeners to test this hypothesis. The stimuli of the two experiments were created by the Klatt synthesizer. [ta], [ka], [to], and [ko] were recorded by a native Japanese speaker and a native English speaker, respectively. Then, [ta], [ka], [to], and [ko] were synthesized based on the parameter values obtained from the measurements of the recorded tokens. They were used as the end-point tokens of a nine-step continuum. The rest of the tokens were created by manipulating frequency parameters of the synthesizer. Four sets of [k]-[t] continua were created (two languages: English and Japanese, and two vowel conditions: [a] and [o]). A preliminary analysis of a subset of data demonstrated that the more frequent sound in each language occupied a larger perceptual region of the [a] vowel continua.

STRAIGHT as a research tool for L2 study: How to manipulate segmental and supra-segmental features

A morphing algorithm based on STRAIGHT speech manipulation system [Kawahara et. al., ICASSP (2003)] has a potential importance in L2 study, since it generates stimulus continuum with quality close to natural speech. The morphing procedure yields an equally spaced continuum in an abstract parameter domain, which is spanned by fundamental frequency, STRAIGHT’s time-frequency representation (a variant of smoothed spectrogram), the periodicity map, and the time-frequency coordinate. Instead of manipulating all these parameters at once, it is useful to manipulate each parameter independently. This individual parameter manipulation can also be generalized to coupled manipulation by introducing new parameters defined using a set of parameters. This extension enables us to manipulate segmental and supra-segmental features instead of manipulating abstract physical parameters. For example, coupling energy envelope, fundamental frequency, and temporal axis stretching factor yield access to a prosodic feature that is basically supra-segmental. Manipulation of spectral landmark positions yields access to a phonological feature that is basically segmental. A preliminary test results using the right-light continuum sets generated using a STRAIGHT-based method and the Klatt synthesizer will also be presented.

Some difference limens for the perception of breathiness

Perception of breathy voice quality appears to be cued by changes in the vowel spectrum. These changes are related to alterations in the intensity of aspiration noise and spectral slope of the harmonic energy [Shrivastav and Sapienza, J. Acoust. Soc. Am., 114 (4), 2217-2224 (2003)]. Ten young-adult listeners with normal hearing were tested using an adaptive listening task to determine the smallest change in signal-to-noise ratio that resulted in a change in breathiness. Six vowel continua, three female and three male, were generated using a Klatt synthesizer and served as stimuli. Results showed that listeners needed as much as 20-dB increase in aspiration noise to perceive a change in breathiness against a relatively normal voice. In contrast, listeners needed approximately an 11-dB increase in aspiration noise to discriminate breathiness against a severely breathy voice. The difference limens for breathiness were observed to vary across the six talkers. Voices having aspiration noise that was predominantly in the high frequencies had smaller difference limens. No significant differences for male and female voice were observed.

Is nonmodal phonation at the end of utterances a cue for speaker recognition by humans?

Many studies of nonmodal phonation report a high rate of interspeaker variation. These experiments also showed that the rate and type of glottalization are characteristic to the speaker. Still, it remains an open question whether human listeners use this information as a cue for recognizing familiar voices. A listening experiment was conducted to investigate this issue, concentrating on utterance-final irregularities. Short utterances produced by four speakers were copy synthesized with the Klatt synthesizer in two conditions: with modal and with nonmodal voice quality in their final regions. Because two of the speakers were reliable ‘‘glottalizers’’ and two seldom ‘‘glottalized,’’ one of the two synthesis conditions reflected the speaker’s usual utterance-final voice quality and the other did not. These pairs of stimuli were used to test the effect of the presence or absence of utterance-final nonmodal phonation on the identification of voices familiar to the subjects. When asked to choose which one of the two variants is the speaker’s voice, listeners preferred the stimuli with the speaker’s typical voice quality at the end. Results will be discussed in comparison with a well-known perceptual cue on speaker identity, the F0 contour. [Work supported by a Fulbright Fellowship.]

Predicting changes in breathiness resulting from variations in aspiration noise and open quotient using a loudness model

Previous studies have reported that increases in high-frequency aspiration noise (AH) and open quotient (OQ) are correlated with an increase in the perception of breathiness in vowels [Klatt and Klatt, J. Acoust. Soc. Am. 87, 820–857 (1990)]. However, a system based on psychoacoustic loudness models has been shown to predict the level of breathiness in vowels to a greater extent than AH and OQ themselves [Shrivastav and Sapienza, J. Acoust. Soc. Am. 114, 2217–2224 (2003)]. The goal of this experiment was to verify the significance of AH and OQ in the perception of breathiness and to determine how each of these influences the output of the loudness model, individually and in combination. Ten vowels, five male and five female, were synthesized using a Klatt synthesizer. Each vowel was used to generate three continua varying in terms of their OQ and/or AH. Ten listeners rated breathiness for each continuum using a magnitude estimation task. Results show that normalized partial loudness as well as the loudness of aspiration noise predicted changes in breathiness resulting from an increase in AH. Increase in OQ resulted in very small changes in perceived breathiness, resulting in poor correlation with loudness measures. [Research supported by NIH/R21DC006690.]

Effects of spectral slope on perceived breathiness in vowels

Previous studies have indicated that breathiness is correlated with changes in spectral slope as well as aspiration noise [Klatt & Klatt, ‘‘Analysis, synthesis, and perception of voice quality variations among female and male talkers,’’ J. Acoust. Soc. Am. 87, 820–857 (1990)]. In a recent study, Shrivastav and Pinero [Shrivastav & Pinero ‘‘Effect of aspiration noise and spectral slope on perceived breathiness in vowels,’’ J. Acoust. Soc. Am. 117, 2622 (2005)] observed that varying spectral slope resulted in minimal changes on the perceived breathiness for synthetic vowels. However, the stimuli tested in this experiment included a relatively narrow range of spectral slope variation. The goal of the present experiment was to verify the role of spectral slope on the perception of breathiness by testing stimuli that had a wider range of variation in spectral slope. Ten voices (5 male and 5 female) were synthesized using a Klatt synthesizer. Each of these was manipulated to generate two continua varying in their spectral slope from −3 to −30 dB/octave. One continuum had a constant cutoff frequency of 500 Hz, while the other continuum had a cutoff frequency between the second harmonic (H2) and the third harmonic (H3). Ten listeners judged the degree of breathiness using a 7-point rating scale. Preliminary results suggest that spectral slope has minimal effect on the perception of breathiness in synthetic vowels. [Research supported by NIH/R21 DC006690.]

Effect of aspiration noise and spectral slope on perceived breathiness in vowels

Breathy voice quality is frequently encountered in both normal and dysphonic voices. Breathiness has been associated with an increase in the intensity of aspiration noise as well as changes in spectral slope [Childers and Ahn, 1995; Fischer-Jorgenson, 1967; Huffman, 1987; Klatt and Klatt, 1990]. Shrivastav and Sapienza (2003) found that subjective ratings of breathiness obtained from a group of listeners were highly correlated with certain measures calculated from the auditory spectrum of the vocal acoustic signal. One of these measures, the partial loudness of the harmonic energy, was related to both the aspiration noise and the spectral slope of the voices. In the present experiment, 10 young adult listeners with normal hearing were presented with voices that varied systematically in terms of their aspiration noise and spectral slope. The stimuli, five male and five female voices, were generated using the Klatt synthesizer and were modeled after naturally occurring voices. Listeners rated breathiness for each of these stimuli using a 7-point rating scale. Results show the relative contribution of spectral slope and aspiration noise to the perception of breathiness.

Co-articulation effects on sound /r/ in cluster words

In this study the coarticulatory effects on the formant frequencies of sound /r/, one of the most context dependent sounds of Hindi, have been investigated, by taking combinations of the cluster sounds of the forms C1r, rC2 and C3C4r. In these cluster combinations C1 is one of the consonants from (/k/,/p/,/g/,/t/,/d/,/m/,/n/,/h/,/bh/,/w/,/b/,/ph/), C2 is one of the consonants from (/y/,/ñ/,/th/,/t/,/∫/,/m/,/dh/,/k/,/w/,/n/,/g/,/j/,/bh/,/t∫/, /dz/,/ph/,/d/,/s/) and C3C4 is one of the combination from (/∫t/,/st/,/nd/,/n:r/, /ndh/). A database of 438 cluster words spoken by three male speakers has been prepared. The sound /r/ has been perceptually segmented from these utterances and the first three formants and their corresponding bandwidths have been measured. For C1r combination, the deviations of the formant frequencies become larger with the order of the formants. The deviations in the formants for rC2 case are also large and it is nasalized if C2 is some nasal sound like /ñ/ and /m/. For the third case (C1C2r), the formant frequencies do not deviate much as in the earlier two cases but the effect of the frication of the combination /∫t/ makes it fricative. These results are being used in improving the quality of Hindi Klatt Synthesizer.

Difference limen for perception of aspiration noise

The relationship between the vocal acoustic signal and the perception of voice quality has been the subject of much research. An increase in the intensity of aspiration noise is associated with a perception of greater breathiness in voices [Klatt and Klatt (1990); Childers (1993)]. Shrivastav (2001) found that subjective ratings of breathiness could be better predicted using measures calculated from an auditory spectrum of the vocal acoustic signal. These measures were found to vary with changes in the spectral slope as well as the intensity of aspiration noise in voice. The aim of the present experiment was to determine the smallest change in the intensity of aspiration noise that could be perceived by listeners. Six voice continua, with increasing intensity of aspiration noise, were generated using the Klatt synthesizer. Five listeners participated in an adaptive listening test, where these stimuli were presented in pairs and listeners were asked to identify the stimuli in each pair as being same or different. Listener responses were used to determine the difference limen for changes in intensity of aspiration noise in voice. These findings will help understand the perception of breathy voice quality and to develop objective measures for its quantification.

Difference limen for aspiration noise in perception of breathy voice quality

Despite much research, the relationship between the vocal acoustic signal and the perception of breathy voice quality is debated. Shrivastav (2001) used an auditory model to study the perception of breathy voice quality and found that listeners’ perception of breathiness was related to the partial loudness of the periodic energy obtained with the aspiration noise acting as a masker. This measure was found to be a better predictor of perceptual judgments than conventionally used acoustic measures of breathiness. The aim of the present experiment was to determine the minimal change in aspiration noise that led to a change in the perception of breathiness. A continuum of voices, identical in all aspects except aspiration noise, was synthesized using a Klatt synthesizer. These stimuli were presented in pairs and listeners identified each pair as being the same or different in terms of their breathiness. Percent-correct scores for each stimulus pair were calculated and compared with differences in the acoustic and auditory spectral measures. Findings of this experiment may be useful in the development of objective tools for the quantification of breathy voice quality.

Dynamic center-of-gravity effects in consonant–vowel transitions

Lublinskaja has reported that dynamic changes in the spectral center-of-gravity (COG) of one formant in selected Russian vowels led to changes in the identification of the vowel. Movement of the COG was effected by simultaneous amplitude modulation of two formants placed at the end points of the desired frequency transition [Iyer et al., J. Acoust. Soc. Am. 109, 2294 (2001)]. The present study explores whether such COG effects will extend into the processing of consonant–vowel transitions (CVs). A [dɑ]–[gɑ] continuum was generated using a Klatt synthesizer with a 50-ms CV transition and a 200-ms vowel. The onset of F3 varied from 2018–2818 Hz in 80-Hz step. The F3 of the vowel remained constant at 2527 Hz. Three different stimulus sets differed in terms of whether the F3 transition was a: (1) formant, (2) FM tone, or (3) virtual frequency (VF) glide. In the VF stimuli, the frequencies of two tones matched the onset or offset frequencies of the F3 transition and modulation of their amplitudes produced a gliding spectral COG. Preliminary results indicate that listener performance is not strongly affected by changing the means by which spectral changes to F3 are made. [Research supported by a grant from OSU’s College of Social and Behavioral Sciences to L. Feth and an INRS award from NIH to R. Fox.]

Phonation types in Suai

Many Mon-Khmer languages have such phonologically distinctive phonation types as modal, breathy, and creaky voice. The Suai language studied here is spoken in northeastern Thailand. Literature exists on these phonation types (‘‘voice registers’’), including some attention to acoustics; however, no perceptual studies appear to have been published. Suai was chosen here because it has just two contrasting categories conventionally labeled modal voice versus breathy voice. The Klatt synthesizer was used to control five of its parameters: F0, amplitude of turbulence, open quotient of the voicing waveform, overall gain, and spectral tilt. All combinations of these parameter settings yielded 51 variants of a basic syllable. These stimuli were identified by native speakers as one word or the other of a minimal pair. The results showed breathiness and pitch to be major cues. The original recordings of our informants, however, appeared only to have a pitch difference, an impression tested by acoustic analysis. A reliable difference was found only in F0 contours, with a lower one on the ‘‘breathy’’ type. There appears to be evidence for an ongoing shift from a system of phonation types to one of tones. [Work supported by NIH Grant DC-02727 and the Thailand Research Fund.]