Vowel Synthesis Research Articles

From "Hello, World!" to Fourier transformations: Teaching linguistics undergraduates to code in ten weeks or less

I used Backward Design to scaffold ten weeks of assignments that taught students how to perform sine wave vowel synthesis and a Fourier transformation approximation using just a few fundamental programming concepts. This strategy gave all students, regardless of their previous programming experience, the opportunity to implement algorithms related to core concepts in phonetics and speech technology. Reflecting on the course, it seems that the coding assignments were generally well-received by students and contributed to students programming something complex and meaningful.

Source and Filter Acoustic Measures of Young, Middle-Aged and Elderly Adults for Application in Vowel Synthesis

IntroductionThe output sound has important changes throughout life due to anatomical and physiological modifications in the larynx and vocal tract. Understanding the young adult to the elderly speech acoustic characteristics may assist in the synthesis of representative voices of men and women of different age groups. ObjectiveTo obtain the fundamental frequency (f0), formant frequencies (F1, F2, F3, F4), and bandwidth (B1, B2, B3, B4) values extracted from the sustained vowel /a/ of young, middle-aged, and elderly adults who are Brazilian Portuguese speakers; to present the application of these parameters in vowel synthesis. Study DesignProspective study. MethodsThe acoustic analysis of tokens of the 162 sustained vowel /a/ produced by vocally healthy adults, men, and women, between 18 and 80 years old, was performed. The adults were divided into three groups: young adults (18 to 44 years old); middle-aged adults (45 to 59 years old) and, elderly adults (60 to 80 years old). The f0, F1, F2, F3, F4, B1, B2, B3, B4 were extracted from the audio signals. Their average values were applied to a source-filter mathematical model to perform vowel synthesis in each age group both men and woman. ResultsYoung women had higher f0 than middle-aged and elderly women. Elderly women had lower F1 than middle-aged women. Young women had higher F2 than elderly women. For the men's output sound, the source-filter acoustic measures were statistically equivalent among the age groups. Average values of the f0, F1, F2, F3, F4, B1, and B2 were higher in women. The sound waves distance in signals, the position of formant frequencies and the dimension of the bandwidths visible in spectra of the synthesized sounds represent the average values extracted from the volunteers' emissions for the sustained vowel /a/ in Brazilian Portuguese. ConclusionSustained vowel /a/ produced by women presented different values of f0,F1 and F2 between age groups, which was not observed for men. In addition to the f0 and the formant frequencies, the bandwidths were also different between women and men. The synthetic vowels available represent the acoustic changes found for each sex as a function of age.

Formant pattern and spectral shape ambiguity in vowel synthesis: The role of fundamental frequency and formant amplitude

When investigating formant pattern and spectral shape ambiguity in Klatt synthesis, an earlier study showed that the perceived vowel quality of Standard German vowel sounds can be changed by varying fundamental frequency only [Maurer et al. (2017). J. Acoust. Soc. Am. 141(5):3469-3470]. In this follow-up study, the previous original synthesis experiment was repeated twice, firstly, with fundamental frequencies (fo) of the corresponding sounds lowered by one octave, and secondly, with different ratios of the first and second formant amplitudes. Here, the role of the fo range and the formant amplitudes for the investigation of formant pattern and spectral shape ambiguity in vowel synthesis was further examined. The same five phonetic expert listeners that participated in the previous experiment also identified all of the newly synthesised sounds in a multiple-choice identification task. Results revealed that the perceived vowel quality only changes for fos above 200 Hz and that, for back vowels, the ratio of the formant amplitudes used in the Klatt synthesis also affects vowel recognition. Thus, the results of the experiments confirm earlier indications of a non-systematic relation between fo or pitch and formant patterns or spectral envelopes for vowel recognition.

“Flat” vowel spectra revisited in vowel synthesis

Some studies of natural and of synthesized vowel sounds indicate “flat” vowel-related spectral envelopes or envelope parts in terms of vowel-related frequency ranges with harmonics equal in amplitude. The present investigation addresses this question in a vowel synthesis experiment in which sounds related to series of harmonics, multiples of 200 Hz in frequency and equal in amplitude, were created. Thereby, for various frequency ranges, the number of harmonics was increased stepwise from a single lower harmonic to an increasingly broader harmonic series, and, inversely, it was also decreased from a broad series of harmonics to a single higher harmonic. The entire frequency range of investigation was 0.2-4 kHz. Vowel recognition was investigated by means of a listening test in which five phonetic expert listeners were asked to assign the synthesized sounds to Standard German vowel qualities. The results of the experiment reveal that synthesized sounds with frequency bands of series of two or more equal-amplitude harmonics allow for a perceptual differentiation of the Standard German vowels /i-y-e-ɛ-a-ɔ-o/. Methodological issues concerning future investigations as well as implications for the acoustics and perception of vowels are discussed.

Vowel synthesis related to equal-amplitude harmonic series in frequency ranges > 1 kHz combined with single harmonics < 1 kHz, and including variation of fundamental frequency

Front vowels can be synthesized on the basis of series of harmonics equal in amplitude, with frequencies only above 1 kHz. In these cases, spectral energy usually attributed to the first formant frequency is lacking. The present paper reports results of an experiment in which sound synthesis was performed on the basis of harmonic series covering higher frequency ranges above 1 kHz, combined with a single lower harmonic < 1 kHz, all harmonics equal in amplitude. Thereby, two or three sounds were synthesized for which the higher frequency range and the frequency of the lower harmonic is identical, but the frequency distance of the higher harmonics differs resulting in different perceived pitches of the sounds. Vowel recognition of all sounds was investigated by means of a listening test in which five phonetic expert listeners were asked to assign the synthesized sounds to Standard German vowel qualities. The results of the experiment reveal that the perceived vowel quality of such types of sound pairs or sound triples differs, confirming earlier indications of the spectral envelope being ambiguous with regard to vowel quality. Implications for the acoustics and perception of vowels are discussed.

Effects of augmentative visual training on audio-motor mapping

The purpose of this study was to determine the effect of augmentative visual feedback training on auditory–motor performance. Thirty-two healthy young participants used facial surface electromyography (sEMG) to control a human–machine interface (HMI) for which the output was vowel synthesis. An auditory-only (AO) group (n=16) trained with auditory feedback alone and an auditory–visual (AV) group (n=16) trained with auditory feedback and progressively-removed visual feedback. Subjects participated in three training sessions and one testing session over 3days. During the testing session they were given novel targets to test auditory–motor generalization. We hypothesized that the auditory–visual group would perform better on the novel set of targets than the group that trained with auditory feedback only. Analysis of variance on the percentage of total targets reached indicated a significant interaction between group and session: individuals in the AV group performed significantly better than those in the AO group during early training sessions (while using visual feedback), but no difference was seen between the two groups during later sessions. Results suggest that augmentative visual feedback during training does not improve auditory–motor performance.

Combined Auditory and Vibrotactile Feedback for Human–Machine-Interface Control

The purpose of this study was to determine the effect of the addition of binary vibrotactile stimulation to continuous auditory feedback (vowel synthesis) for human-machine interface (HMI) control. Sixteen healthy participants controlled facial surface electromyography to achieve 2-D targets (vowels). Eight participants used only real-time auditory feedback to locate targets whereas the other eight participants were additionally alerted to having achieved targets with confirmatory vibrotactile stimulation at the index finger. All participants trained using their assigned feedback modality (auditory alone or combined auditory and vibrotactile) over three sessions on three days and completed a fourth session on the third day using novel targets to assess generalization. Analyses of variance performed on the 1) percentage of targets reached and 2) percentage of trial time at the target revealed a main effect for feedback modality: participants using combined auditory and vibrotactile feedback performed significantly better than those using auditory feedback alone. No effect was found for session or the interaction of feedback modality and session, indicating a successful generalization to novel targets but lack of improvement over training sessions. Future research is necessary to determine the cognitive cost associated with combined auditory and vibrotactile feedback during HMI control.

Vowel synthesis using a vocal tract mapping interface and simulation study of inverse mapping

We have developed a vocal tract mapping interface to produce vowel sounds. The interface provides an easy and effective setting of the vocal tract shape with a simple mouse click on its interface window. A vocal tract shape at the mouse position on the interface window is calculated by interpolation based on the prepared vocal tract shapes. In this paper, the features and advantages of the interface are shown through examples of the generated vowel sounds and vocal tract shapes. In addition, the inverse estimation of the vocal tract shape from formant frequencies is studied as one of its applications. In this method, the vocal tract shape is estimated by searching the point inside the region after determining a possible region that includes the solution. The possible region on the interface window is determined based on the changes in the formant frequencies. The usefulness of the proposed method is shown through simulation results.

Categorical Vowel Perception Enhances the Effectiveness and Generalization of Auditory Feedback in Human-Machine-Interfaces

Human-machine interface (HMI) designs offer the possibility of improving quality of life for patient populations as well as augmenting normal user function. Despite pragmatic benefits, utilizing auditory feedback for HMI control remains underutilized, in part due to observed limitations in effectiveness. The goal of this study was to determine the extent to which categorical speech perception could be used to improve an auditory HMI. Using surface electromyography, 24 healthy speakers of American English participated in 4 sessions to learn to control an HMI using auditory feedback (provided via vowel synthesis). Participants trained on 3 targets in sessions 1–3 and were tested on 3 novel targets in session 4. An “established categories with text cues” group of eight participants were trained and tested on auditory targets corresponding to standard American English vowels using auditory and text target cues. An “established categories without text cues” group of eight participants were trained and tested on the same targets using only auditory cuing of target vowel identity. A “new categories” group of eight participants were trained and tested on targets that corresponded to vowel-like sounds not part of American English. Analyses of user performance revealed significant effects of session and group (established categories groups and the new categories group), and a trend for an interaction between session and group. Results suggest that auditory feedback can be effectively used for HMI operation when paired with established categorical (native vowel) targets with an unambiguous cue.

Audio-visual feedback for electromyographic control of vowel synthesis.

We describe the design and testing of a human machine interface to use surface electromyography (sEMG) collected from a covert location in response audio-visual feedback. Using sEMG collected from the Auricularis Posterior muscle, N=5 healthy participants participated in 6 sessions over multiple days to learn to transition from visual and vowel synthesis feedback to vowel synthesis feedback alone. Results indicate that individuals are able to learn sEMG control of vowel synthesis using auditory feedback alone with an average of 67% accuracy and that this skill can also generalize to new vowel targets. Control of vowel synthesis using covertly-recorded sEMG is a promising step toward more reliable mobile human machine interfaces for communication.

Real-Time Dynamic Articulations in the 2-D Waveguide Mesh Vocal Tract Model

Time domain articulatory vocal tract modeling in one-dimensional (1-D) is well established. Previous studies into two-dimensional (2-D) simulation of wave propagation in the vocal tract have shown it to present accurate static vowel synthesis. However, little has been done to demonstrate how such a model might accommodate the dynamic tract shape changes necessary in modeling speech. Two methods of applying the area function to the 2-D digital waveguide mesh vocal tract model are presented here. First, a method based on mapping the cross-sectional area onto the number of waveguides across the mesh, termed a widthwise mapping approach is detailed. Discontinuity problems associated with the dynamic manipulation of the model are highlighted. Second, a new method is examined that uses a static-shaped rectangular mesh with the area function translated into an impedance map which is then applied to each waveguide. Two approaches for constructing such a map are demonstrated; one using a linear impedance increase to model a constriction to the tract and another using a raised cosine function. Recommendations are made towards the use of the cosine method as it allows for a wider central propagational channel. It is also shown that this impedance mapping approach allows for stable dynamic shape changes and also permits a reduction in sampling frequency leading to real-time interaction with the model

Vowel synthesis tool based on transmission line model as a web application

This paper introduces a web application for vowel synthesis in acoustics education. Synthesis systems that provide users with interactive operation for determining vocal tract configuration and vocal fold control parameters can be effective tools for intuitive understanding of vowel production for students. For example, differences in the size of the vocal tract and vocal folds between male and female speakers can be recognized as differences in formant and pitch frequencies in speech sounds. The realization of such a system as a web application is suitable for platform independent use, considering the recent progress in information and communication technology. In our developed system, interactive operation for determining vocal tract configuration and vocal fold control parameters is achieved on a web browser by using a Java applet. These articulatory parameters are transferred to a server computer via a client/server connection using socket programming, and synthetic speech is produced using calculation programs on the server computer. The speech synthesizer installed on the server is based on the hybrid speech production model proposed by Sondhi and Schroeter. A preliminary user test showed that the system demonstrated a certain applicability as a teaching tool.

Development of vowel synthesis system based on transmission line model as a web application

Development of vowel synthesis system based on transmission line model as a web application

An experimental study of acoustic characteristics of hypopharyngeal cavities using vocal tract solid models

Solid models of the vocal tract with hypopharyngeal cavities were molded with a stereolithographic technique based on MRI data obtained from a male speaker during the production of Japanese vowels /a/ and /o/. A vowel synthesis experiment conducted with the models revealed a relatively good agreement in the second and third formants, as well as in anti-resonance at 4–5 kHz. The elimination of the models’ piriform fossa resulted in the disappearance of the anti-resonance and shifts of the adjacent formants. The modification of the laryngeal cavity into a uniform tube caused spectral changes in the frequency range of 1.5–7.0 kHz. These acoustic effects of hypopharyngeal cavities were dependent on vocal tract shapes.

Surrogate analysis for detecting nonlinear dynamics in normal vowels

Normal vowels are known to have irregularities in the pitch-to-pitch variation which is quite important for speech signals to be perceived as natural human sound. Such pitch-to-pitch variation of vowels is studied in the light of nonlinear dynamics. For the analysis, five normal vowels recorded from three male and two female subjects are exploited, where the vowel signals are shown to have normal levels of the pitch-to-pitch variation. First, by the false nearest-neighbor analysis, nonlinear dynamics of the vowels are shown to be well analyzed by using a relatively low-dimensional reconstructing dimension of 4 < or = d < or = 7. Then, we further studied nonlinear dynamics of the vowels by spike-and-wave surrogate analysis. The results imply that there exists nonlinear dynamical correlation between one pitch-waveform pattern to another in the vowel signals. On the basis of the analysis results, applicability of the nonlinear prediction technique to vowel synthesis is discussed.

A vowel synthesizer based on formant sinusoids modulated by fundamental frequency

A vowel synthesis algorithm was developed resembling parallel formant synthesizers. However, the formants are computed as amplitude-and phase-modulated sinusoids instead of being represented by recursive filters or resonators. The fundamental frequency is imposed on the signal in the following way: Each pitch period exhibits an initial rise of a duration of π/2 of the first formant frequency; at the end of this ramp period all formants start as sinusoids at predefined phase angles and amplitudes, and successively undergo linear attenuation toward zero amplitude at the end of each pitch period. Adjacent pitch periods, thus, do not influence each other. In principle, the signals produced by this algorithm are similar to the output of parallel formant synthesizers [e.g., D. H. Klatt, J. Acoust. Soc. Am. 67, 971–995 (1980)]. However, by incremental sample-by-sample computation of the formants’ phase angles, formant frequencies are allowed to continuously change within single pitch periods. This new algorithm creates vowels as well as formant transitions signaling stop consonant–vowel syllables such as /ba/. The output signals exhibit a homogeneous, stereotypic voice quality. Because of its additive procedure, this method might allow us to implement a variety of further acoustic aspects of the human speaking voice in a well-controled manner.

A vowel synthesizer based on F0-modulated formant sinusoids

A vowel synthesis algorithm was developed resembling parallel formant synthesizers. However, formants are computed as amplitude- and phase-modulated sinusoids instead of being represented by recursive filters or resonators. The fundamental frequency is imposed on the signal in the following way: Each pitch period starts with a short initial rise of a duration of pi/2 of the first formant frequency; at the end of this ramp phase all formants start as sinusoids at pre-defined phase angles and amplitudes, and successively undergo linear attenuation toward zero amplitude at the end of each pitch period. Adjacent pitch periods, thus, do not influence each other. In principle, the signals produced by this algorithm resemble the output of parallel formant synthesizers [e.g., D. H. Klatt, J. Acoust. Soc. Am. 67, 971–995 (1980)]. However, by incremental sample-by-sample computation of the formants’ phase angles, formant frequencies are allowed to continuously change within single pitch periods. This algorithm produces vowels as well as formant transitions signaling stop consonant–vowel syllables such as /ba/, exhibiting a homogeneous, stereotypic voice quality. Because of its additive procedure, this method might allow for the implementation of a variety of further acoustic aspects of the human speaking voice in a well-controlled manner.

Fourier spectra and formant patterns of German vowels produced at F0 of 70–850 Hz

Previous studies of vowel synthesis and of natural vowels have indicated: (1) a correlation between the lower formants and F0 for F0&amp;gt;175 Hz; (2) intelligibility and spectral differences of vowels with high F0&amp;gt;500 Hz; (3) severe methodological problems of formant frequency estimation for F0&amp;gt;300 Hz; (4) formant number alterations (appearance of different numbers of relevant formants for one vowel); (5) formant pattern ambiguity (appearance of the same formant pattern for different vowels). The purpose of this study was to replicate these findings. A large sample of the German vowels /i,y,e,o/,ε,■,o,u/ produced in isolation as well as in CVC context was investigated. There were 18 674 recordings made of 35 men, 44 women, and 20 children. Isolated vowels were produced at different levels of F0, and with different intensities. A listening test was performed. Fourier spectra, LPC spectra and spectral constancy were analyzed. The results strongly support the five indications mentioned. In particular, for isolated vowel sounds the correlation between the formants and F0, the appearance of ‘‘one-formant back vowels’’ (only one apparent formant &amp;lt;2 kHz), and the formant pattern ambiguity for nonadjacent vowels proved to be very pronounced. Consequences for normalization approaches are discussed.

Formant synthesis: Turning cascade into parallel with applications to the Klatt synthesizer

Formant speech synthesis is one of the successful approaches for text-to-speech synthesis. Usually, formants are connected in parallel for the synthesis of consonants and in either cascade or parallel for vowels. The cascade structure requires a fewer number of control parameters and the synthesized spectrum is guaranteed to be correct. It is best suited for vowel synthesis. However, because of the necessary interchange of a parallel and cascade structure, it is not trivial to maintain good continuity in resonance modes. On the other hand, vowels can also be synthesized by the same parallel formant synthesizer used for consonants. This solution avoids the interchange of resonators, but more control parameters are necessitated and the resultant spectrum may be distorted. In this paper, a method is described which mathematically turns a cascade configuration into parallel. No additional parameters, that is, the complex amplitudes of formants, are explicitly required. Instead, these parameters are determined from given formant frequencies, bandwidths, and the higher-pole correction. The method thus maintains continuity in resonance modes without requiring extra parameters. The method has been integrated into the Klatt synthesizer. Synthetic speech generated using the proposed method will be demonstrated. [Work partially supported by ARPA Contract No. DAST 63-93-C-0064.]

FO-dependence, number alteration, and non-systematic behaviour of the formants in German vowels.

In acoustic theory, formant pattern differences within one vowel identify are related to differences in the speaker groups or in the types of vocalization. Yet, studies of vowel synthesis indicate that formant patterns can also vary strongly in relation either to pitch or to formant number alterations. Within this study, natural vocalizations of nine German vowels were investigated with regard to different formant patterns representing the same vowel identity, apart from differences in speaker groups and vocalization types. The results show i) FO-dependence of the lower formants < 1.5-2 kHz; ii) occurrence of natural one-formant back and two-formant front vowels; iii) a non-systematic relationship between FO and the formants, and a non-systematic relationship between formant patterns of different formant numbers. The implications for psychophysics, physiology and perception of speech are discussed.