Articulatory Speech Synthesis Research Articles

Unsupervised neural networks that use a continuity constraint to track articulators

A recent study [Hogden etal., J. Acoust. Soc. Am. 91, 2443 (A) (1992)] showed that articulatory speech synthesizer parameters describing tongue positions can be recovered from synthesized acoustic signals using continuitymapping. Previous results are extended to two neural network implementations of continuity mapping. Both networks are tested on a two-dimensional spatial analog of the articulator tracking problem. The network’s inputs (analogous to acoustic signals in the previous study) are the distances between a randomly moving robot (analogous to the tongue) and its nearest obstacles. The network’s outputs encode the robot’s position. The first implementation uses a frequency-sensitive, competitive learning network (FSCLN) to vector quantize the inputs, followed by a linear layer with one output unit for each degree of freedom of the robot. At each time t the delta rule is used to teach the network to produce an output, O(t), that is similar to O(t−1). The second implementation follows the FSCLN with a Kohonen-like neural layer. The relationship between these simulations and the acoustic-to-articulatory mapping problem in speech will be discussed.

An unsupervised method for learning to track tongue position from an acoustic signal.

A procedure for learning to recover the relative positions of the articulators from speech signals is demonstrated. The algorithm learns without supervision, that is, it does not require information about which articulator configurations created the acoustic signals in the training set. The procedure consists of vector quantizing short time windows of a speech signal, then using multidimensional scaling to represent quantization codes that were temporally close in the encoded speech signal by nearby points in a continuitymap. Since temporally close sounds must have been produced by similar articulator configurations, sounds which were produced by similar articulator positions should be represented close to each other in the continuity map. Using an articulatory speech synthesizer to produce acoustic signals from known articulator positions, relative articulator positions were estimated from synthesized acoustic signals and compared to the synthesizer’s actual articulator positions. High rank-order correlations, ranging from 0.92 to 0.99, were found between the estimated and actual articulator positions. Reasonable estimates of relative articulator positions were made using 32 categories of sound, and the accuracy improved when more sound categories were used.

Perception of Assimilation in Consonants Clusters:A Gestural Model

Abstract The goal of this experiment was to examine the perceptual effects of gestural overlap or coproduction on the first consonant in a cluster. The experiment was conducted using synthetic speech stimuli containing [b#d] and [d#b] consonant clusters in two word phrases. The stimuli were synthesized using an articulatory speech synthesizer according to the linguistic gestural model of Browman and Goldstein. The influence of the relative places of constriction in the cluster and the complexity of the presentation environment are discussed. Two asymmetries with respect to gestural overlap were found. First, assimilation was more readily perceived when C<sub>1</sub> was an alveolar consonant than when C<sub>1</sub> was bilabial. Second, a two-word context encouraged the perception of anticipatory assimilation to a greater degree than an environment in which the second word had been truncated leaving only the acoustic effects of consonantal coproduction on the vowel offset. The results are discussed in terms of the linguistic gestural model with special reference to the application of this framework to the description of assimilation, multiply articulated consonants, and secondary articulations.

Measurements of vocal tract shapes using magnetic resonance imaging

Magnetic resonance imaging is used to determine the vocal tract area functions of a speaker producing five steady vowel sounds. The data is used in an acoustic tube model, formed by concatenating short uniform sections, and the acoustic spectra of the computed output from the model are compared with those of natural speech. For all but one of the vowels good agreement (typically within 120 Hz) is obtained for the first three formant frequencies. To compensate for measurement errors, optimisation procedures are applied, aimed at adjusting the areas to minimise a chosen cost function. A cost function which combines a windowed cepstral error with a Euclidean distance penalty for the areas is effective in finding area functions that gave much closer spectral matches, but which still lie largely within the estimated errors of measurement. These adjusted functions provide suitable root shapes for use in articulatory speech synthesis.

Speech production theory and articulatory speech synthesis

Speech production theory and articulatory speech synthesis

Design and evaluation of optimal cepstral lifters for accessing articulatory codebooks

It is noted that of great importance to the success of the articulatory approach to speech coding is the use of a good distortion measure between a given speech signal and the entries in a stored codebook of impulse responses and corresponding vocal-track shapes (articulatory codebook). One promising distortion measure is the weighted cepstral distortion. Since the impulse responses in the articulatory codebook do not include glottal characteristics, the authors derive optimal weighting functions (cepstral lifters) to reduce the influence of a varying glottal source on the cepstral distortion measure. This is done by examining the ensemble of cepstral coefficients of speech produced by an articulatory speech synthesizer that also includes a vocal-cord model. The obtained cepstral lifters are optimal for the given ensemble of cepstral coefficients and for given constraints on the weighting function. They are different for cepstral coefficients derived from the power spectrum (FFT cepstra) and for those derived from LPC (linear predictive coding) coefficients (LPC cepstra). The performances of the obtained cepstral lifters are compared in an articulatory codebook search. >

Articulatory synthesis: nasal sounds and male and female voices

We describe the implementation of an articulatory speech synthesizer using a discrete circuit model that converts the acoustic equations into linear algebraic equations. The algorithm is fast, flexible and stable. Three experiments were conducted with the synthesizer to determine: (1) the effect of the sinuses on the quality of nasal sounds; (2) the effect of the velopharyngeal port opening on nasal sounds; (3) the effect of fundamental frequency, vocal tract area, vocal fold parameters and excitation on the synthesis of female voices.

A new algorithm for speech synthesis based on vocal tract modeling

A new algorithm for articulatory speech synthesis is described in this paper. The algorithm constitutes two main parts: a detailed frequency domain modeling of the vocal tract and a data transform from the frequency domain to the time domain. A computer model was developed to simulate the vocal tract acoustics. The model incorporates all known important components of the vocal system and computes the transfer function between the lip/nostril output and the acoustic source. By decomposing the obtained transfer function into its numerator and denominator, frequencies and bandwidths of resonances (and of antiresonances, if any) can be determined. The transfer function can next be written as a partial fraction expansion series in terms of calculated residues at the poles and can be approximated by retaining the first few terms in the series. Usually, each of these terms is a second‐order module and corresponds to an elementary formant resonance. Formants are thus connected in parallel. The time domain output ...

Evaluation of cepstral lifters for articulatory codebook search

Of great importance for the success of the articulatory approach to speech coding [Schroeter et al., IEEE‐ICASSP, 308–311 (1987)] is the use of a good distance measure between a given speech signal and the entries in a stored codebook of impulse responses and corresponding vocal tract shapes (articulatory codebook). One promising distance measure is the weighted cepstral distance. Since the impulse responses in the articulatory codebook do not include glottal characteristics, optimal weighting functions (cepstral lifters) are derived to reduce the influence of a varying glottal source on the cepstral distance measure. This is done by examining the ensemble of cepstral coefficients of speech produced by an articulatory speech synthesizer that also includes a vocal‐cord model. The obtained cepstral lifters are optimal for the given ensemble of cepstral coefficients and for given constraints on the weighting function. They are different for cepstral coefficients derived from the power spectrum (FFT cepstra) and those derived from LPC coefficients (LPC cepstra). The performances of the obtained cepstral lifters are compared in an articulatory codebook search.

Vector quantization of the articulatory space

A technique is presented for quantizing the articulatory space, i.e. for replacing the continuum of all possible vocal tract shapes by a finite set of shapes that span the articulatory space. Vocal tract shapes are represented as vectors of parameters that control an articulatory model, so vector quantization are applicable for deriving this codebook of tract shapes. Samples of the articulatory space are generated by interpolating model parameters along paths between selected target configurations. These shapes constitute the training data that are clustered to form the codebook. Such a codebook of shapes has direct application when vocal tract shapes must be estimated from speech. One such application, which motivated the present study, is the use of the codebook to providing starting configurations for an optimizing articulatory speech synthesizer. >

A hybrid time-frequency domain articulatory speech synthesizer

High quality speech at low bit rates (e.g., 2400 bits/s) is one of the important objectives of current speech research. As part of long range activity on this problem, we have developed an efficient computer program that will serve as a tool for investigating whether articulatory speech synthesis may achieve this low bit rate. At a sampling frequency of 8 kHz, the most comprehensive version of the program, including nasality and frication, runs at about twice real time on a Cray-1 computer.

A hybrid domain articulatory speech synthesizer

Articulatory speech synthesizers model the human vocal tract by means of its geometrical and functional properties. It is believed that this approach can be advantageous in speech coding at bit rates below 4800 b/s. Existing articulatory synthesizers work in the time domain. They either solve a system of differential equations for the vocal tract and the glottis, or synthesize speech using wave digital filters. The first approach is computationally very cumbersome. Both approaches have difficulties in incorporating important acoustic parameters, for example, the radiation impedance at the lips, wall vibration, and other losses. So far a realistic glottis model suitable for the wave digital filter approach does not exist. We have combined a nonlinear time‐domain model of the vocal cords [K. Ishizaka and J. L. Flanagan, Bell Syst. Tech. J. 51, 1233–1268 (1972)] with a linear frequency‐domain chain‐matrix model of the vocal tract [e.g., M. M. Sondhi, paper ♯4.5.1, Proc. Int. Congress on Acoustics, Paris, France, 1983, Vol. 4, pp. 167–170]. The interface between these two models consists of convolving the glottal flow in the time domain with impulse responses of the tract obtained by inverse FFT. Examples of synthesized speech using manually generated and measured tract areas will be given.

Adaptive emulation of human speech by an articulatory speech synthesizer

By controlling the parameters of an articulatory synthesizer [C. H. Coker, Proc. IEEE 64, 452–459 (1976)] with an unconstrained optimization technique, the synthesizer can be made to adapt to and emulate natural utterances. For ten simple English vowels in steady state we have observed convergence of the first three formants to within a few hertz of those of human speech in fewer than 25 iterations. In addition, the corresponding positions of the articulators are physiologically reasonable. We are currently studying methods for controlling the articulatory dynamics of the synthesizer in order to imitate temporally varying utterances. Details of the adaptation algorithm and experimental results are given. Finally the relevance of this study to problems in speech recognition, synthesis, and transmission is outlined.

Studies of Nasal Consonants with an Articulatory Speech Synthesizer

An electrical analog of the nasal cavities has been designed for use in conjunction with a dynamically controllable analog of the vocal tract [G. Rosen, J. Acoust. Soc. Am. 30, 201 (1958)]. The acoustical coupling between vocal tract and nasal cavities is electronically variable. Listening tests are conducted to determine the most suitable vocal-tract configurations for producing nasal consonants in consonant-vowel and vowel-consonant syllables. Other listening tests determine the most suitable patterns of temporal variables for these syllables. In general, it is found that more natural-sounding syllables are obtained when the velopharyngeal coupling varies slowly with time compared to changes in vocal-tract configuration. The results compare favorably with recent cineradiological data, and are interpreted to support the view that studies with articulatory synthesizers reveal important details about the production and perception of natural speech. The experiments demonstrate that speech can be synthesized from articulatory information in a straightforward and natural manner.