Articulatory Model Research Articles

Study of quantal theory of vowel production using area function and articulatory models

Quantal theory of Stevens [J. Phonetics 17, 3–45 (1989)] states that there appear to be ranges of the articulatory parameter for which there is very little change in the acoustic parameter and other ranges where the acoustic parameter is more sensitive to changes in articulation. This result has been arrived at using a three- or four-tube model and coupled resonator theory. In this paper, the above experiment is replicated using a more realistic frequency domain method for calculating the formant frequencies for a given area function which include the effect of yielding walls and radiation impedance. Further, the quantal theory of vowel production is studied using Steven’s and Mermelstein’s articulatory models. Nomograms have been generated for these models. The acoustic and perceptual effects of perturbing articulatory parameters centered around the target values for selected vowels are studied. Preliminary experiments show that (a) the so-called areas of stability (broad F2 maximum, F3 minimum) are, in fact, most sensitive for articulatory changes and (b) for a given change in articulation, while F2 is insensitive, F1 and/or F3 are highly sensitive. In view of these findings, the quantal theory may have to be restated.

Acoustic-to-articulatory mapping of fricatives

This research considers the acoustic-to-articulatory mapping of fricative speech. By incorporating additional knowledge about fricative production, perception, and dynamics into the mapping scheme, improved performance may be achieved over standard acoustic-to-articulatory mapping techniques. A hybrid time-frequency domain articulatory synthesizer and numerical optimization techniques are applied to steady-state, unvoiced fricatives. The work differs from the fricative inverse mapping experiments of Sorokin [V. Sorokin, Speech Commun. 14, 249–262 (1994)] and Shirai [K. Shirai and S. Masaki, Speech Commun. 2, 111–114 (1983)] in that a fricative-specific linked codebook is used to initialize optimization and amplitude sensitive distance measures are used. The fricative codebook produces starting configurations for numerical optimization based on their spectral characteristics and their ability to match frication source characteristics for a given lung pressure and glottal configuration. The use of amplitude sensitive spectral distance measures is necessary to account for the interaction between the tract and the flow-dependent frication source. Examples of acoustic-to-articulatory mapping are given that compare fricative articulatory models and fricative spectral distance measures for the inverse mapping scheme. Acoustic-to-articulatory performance is evaluated in terms of spectral distance and clustering ability. [Work supported by the AFOSR.]

Articulatory modeling of so-called advanced tongue root vowels

Many West African languages have two sets of vowels that are said to differ by one having an advanced tongue root [+ATR], and the other a retracted tongue root [−ATR]. The [+ATR] set may also have an enlarged pharynx and a lowered larynx. Words (roots) in these languages must contain vowels that are all of one set or the other. To test whether there is a constant articulatory difference between the two sets of vowels in Degema, a Delta Edoid language spoken in Nigeria, DAT recordings were made of 8 speakers. For 2 of the speakers, frontal and lateral videos were made of the lip aperture. The frequencies, amplitudes and bandwidths of the first 3 formants of the 5 pairs of vowels were determined from LPC analysis. These data were modeled using an articulatory synthesizer. After an appropriate set of vocal tract shapes had been determined for the [−ATR] set of 5 vowels, articulatory perturbations such as tongue root advancement, pharynx enlargement, and larynx lowering, were applied to the modeled [−ATR] vocal tract shapes to generate the [+ATR] set. A variety of articulatory gestures seem to be needed to convert vowels from one set into the other.

Limitation on labial anticipation in [i] in a language with an [i]–[y] contrast

In French, the acoustic effects of anticipation of a final high front rounded vowel, [y], can be observed in a preceding [i]. This labial anticipation induces a lowering of the formants being measured (F2,F3,F4) both in the middle and, to a greater extent, at the end of the [i] in [iCny] sequences. Although significant, this contextual influence does not jeopardize the acoustic contrast between [i] and [y]. To understand the articulatory basis of these results, acoustic consequences of the gestures involved in the so-called rounding contrast between [i] and [y] are investigated through articulatory modeling. An [i]-like area function is simulated with different degrees and combinations of anticipatory lipprotrusion, reductionoftheareaatthelips, and larynxlowering. [i] allows little variation in lip gestures; beyond a limited range of coarticulation [i] shifts acoustically to the region occupied by [y]. While [i] appears to be quantal when we consider variation in tongue constriction, it shows a sensitive response to lip perturbation. In the [iCy] sequences observed, it seems that constraints on distinctiveness prevent a greater labial anticipation.

The correlation of P-center adjustments with articulatory and acoustic events.

To evaluate articulatory models of perceptual center (P-center) location, listeners performed perceptual adjustments on stimuli which were extracted from a corpus of articulatory data. To avoid streaming effects, the stimuli were not edited to obtain temporal variation; instead, they varied in stress and segmental content. Adjustments were evaluated as to their simultaneity with acoustic and articulatory events. The first experiment yielded various articulatory and acoustic correlates of P-center location; the second yielded different articulatory predictors and no acoustic effective predictors. Multiple correlation analyses showed a variation from P-center locations predicted by the articulatory events that were associated with other predictors. Thus, P-center locations do not correspond to any particular kinematic articulatory event, but rather to a complex of events taken from throughout the stimuli. These results are discussed in terms of their relevance to a model of P-centers as indices of underlying gestural timing.

Speech production models in automatic speech recognition—Forming a lasting marriage between speech science and speech technology

At present, the performance of automatic speech recognition (ASR) systems is still limited by variabilities within and between speakers, by acoustic differences between training and application environments, and by the sensitivity of ASR systems against changing communication channels. This talk considers the conjecture that the use of speech-production models in ASR systems can contribute to making ASR systems more robust with respect to these sources of variability. Although it is well known that production-oriented representations of speech may be used to exploit the continuity of articulatory movements, several obstacles stand in the way of incorporating speech production models in recognizers. These include the difficult problem of acoustic-to-articulatory mapping, the hugh complexity of searching an articulatory space, and the lack of sensitive diagnostic performance metrics for evaluating strengths and weaknesses of a particular production model. Several research laboratories are actively involved in efforts towards incorporating articulatory models in various forms in working ASR systems. In addition to summarizing this work, mechanisms will be suggested for stimulating closer interaction between researchers in production, perception, and processing.

Recovering articulatory movement from formant frequency trajectories using task dynamics and a genetic algorithm: Preliminary model tests

Articulatory trajectories of an articulatory model were recovered by means of a genetic algorithm from acoustic information using a task-dynamic model of speech articulation. Tests on simulated utterances / əbæ/ and / ədæ/ show that the method can recover most of parts of an original trajectory, but it has trouble in obtaining precise timing. For the recovery of articulation, formant frequency trajectories should be supplemented by additional acoustic information, such as RMS amplitude.

From EMG to Formant Patterns of Vowels: The Implication of Vowel Spaces

With a few exceptions, EMG data are interpreted with reference to the intended output, such as the phonetic description of utterances spoken by speakers. For a more rigorous interpretation, the data should also be analysed in terms of the displacement of the articulators and the acoustic patterns. In this paper, we describe our attempts to calculate the formant patterns from EMG activity patterns via an articulatory model. The value of the model parameters, such as the tongue body position or tongue body shape, is derived from the EMG activities of the specific pairs of antagonistic tongue muscles. The model-calculated F1-F2 patterns for 11 American English vowels correspond rather well with those measured from the acoustic signals. What strikes us is the simplicity of the mappings from the muscle activities to vocal-tract configurations and to the formant patterns. We speculate that the brain optimally exploits the morphology of the vocal tract and the kinematic functions of the tongue muscles so that the mappings from the muscle activities (production) to the acoustic patterns (perception) are simple and robust.

Pitch-synchronous frame-by-frame and segment-based articulatory analysis by synthesis

This paper presents a pitch-synchronous analysis-by-synthesis procedure for estimating model parameters for voiced speech. These model parameters describe the vocal-tract shape and the time derivative of the glottal area function. The excitation waveform is derived from the glottal area function by incorporating source-tract interaction using the current vocal-tract input impedance. The corresponding analysis procedure for estimating the model parameters once every pitch period is outlined. A significant improvement in quality was obtained for the new pitch-synchronous analysis/synthesis procedure relative to the fixed-frame-length-based scheme used previously. It was also found that the new pitch-synchronous articulatory analysis/synthesis scheme achieves lower rms spectral distortion values than the 2.4 kb/s. Federal standard LPC-10E algorithm. A segment-based procedure for estimating the vocal-tract model parameters at a rate much lower than the current pitch is described. In this segment-based analysis-by-synthesis approach, the model parameters are estimated every 50–100 ms. The parameters for the intermediate pitch periods are derived by interpolation. The segments are selected using a maximum likelihood segmentation algorithm that segments an utterance into diphonelike units. A segment-based parameter optimization scheme could lead to a highly economical representation of the speech signal for potential applications in very low bit rate speech coding and speech storage. The above schemes were optimized for a pilot test sentence and then evaluated using eight test sentences for a log area and the Coker articulatory model representation of the vocal tract. Nine listeners were asked to judge the quality of the synthesis in a paired-comparison test and the results were analyzed using a nonparametric one-tailed sign test. For the log-area representation of the vocal tract, we found a significant degradation in speech quality for the segment-based optimization procedure relative to the frame-based procedure. However, for the Coker model representation, the degradation was found to be insignificant. This shows that unlike cross-sectional areas, the movement of various articulators in the vocal tract during speech production can be described with sufficient accuracy by specifying the position of these articulators and by using an interpolation function at time intervals much longer than a pitch period.

Vowel normalization by articulatory normalization: Preliminary results

This paper presents an approach to vowel normalization based on the exloitation of the concept of formant to cavity affiliation as described in the vocalic theory of speech production [G. Fant, AcousticTheoryofSpeechProduction (Mouton, La Hague, 1960)]. For closed vowels, this concept allows an estimation of the ratio of the respective back and front cavities lengths of two speakers, by the ratio of the associated formants. In order to propose a normalization of the vocalic space of different speakers, a ‘‘reference speaker’’ was used whose articulatory and acoustic properties are well defined: the articulatory model as proposed by Maeda [J. Acoust. Soc. Am. Suppl. 1 81, S146 (1988)]. Vowel normalization consists then in the projection of the vocalic space of one given speaker in the vocalic space of this model. Geometrical relations between the studied speaker and the ‘‘reference speaker’’ are extracted from [i] and [u] for front and velopalatal vowels, and from [a] and [u] for back and velopharyngeal vowels. Here, [y] and [u] give some indications about the lip shapes. This method is here tested by her ability to predict the vowels of the studied speaker starting from the vowels of this model.

Inversion and control of an articulatory model of the vocal tract: Recovering articulatory gestures from sounds

To what extent are the basic concepts of robotics and control theory useful for the understanding of speech communication processes? In this paper, this question is addressed and a quantitative modeling framework is proposed, which has some bearings on classical concepts of feedforward and feedback control systems. The main feature of the model is the capability to solve the ill-posed inversion problem of determining the articulatory commands from speech goals specifications, whatever they are. A theoretical analysis shows that former models for motor control, namely the task-dynamics approach [Saltzman and Munhall (1989), Ecolog. Psychol. 1(4)] and the forward-inverse modeling [Jordan and Rumelhart (1992), Cognitive Sci.], are subcases of the present model. Simulations of well-known phenomena in speech, like compensation for bite-block perturbation and coarticulation of simple VCV sequences, can be successfully done in this framework. This modeling effort can feed the theoretical debate about invariance/variability in speech because it is a natural framework to address the following issues: (1) the nature of the control space used in programming speech gestures; (2) the suitable principles of control, defined as the choice of an architecture for the control and the constraints imposed on the control law to solve the inversion problem; and (3) the learning of an inverse model capable of generating suitable articulatory commands from a specification of the goals to be reached.

Articulatory representation and speech technology.

In this paper we demonstrate the feasibility and usefulness of articulation-based approaches in two major areas of speech technology: speech recognition and speech synthesis. Our articulatory recognition model estimates probabilities of categories of manner and place of articulation, which establish the articulatory feature vector. The transformation from the articulatory level to the symbolic level is performed by hidden Markov models or multi-layer perceptions. Evaluations show that the articulatory approach is a good basis for speaker-independent and speaker-adaptive speech recognition. We are now working on a more realistic articulatory model for speech recognition. An algorithm based on an analysis by synthesis model maps the acoustic signal to 10 articulatory parameters which describe the position of the articulators. EMA (electro-magnetic articulograph) measurements recorded at the University of Munich provide good initial estimates of tongue coordinates. In order to improve articulatory speech synthesis we investigated an accurate physical model for the generation of the glottal source with the aid of a numerical simulation. This model takes into account nonlinear vortical flow and its interaction with sound-waves. The simulation results can be used to improve the articulatory synthesis model developed by Ishizaka and Flanagan (1972).

Articulatory Actions within a Phonological System and the Resulting Complexity of Speech Signals

Abstract The importance of overlapping coordinated gestures in achieving the auditory goals of speech is considered. Articulatory synthesis can help our understanding of the coordination problems and the variety of solutions found by real speakers. The time required to perform specific essential gestures, such as abduction-adduction of the vocal folds for fricatives, imposes strong timing constraints. In different speech sequences different articulators act as the duration-determining factor (DDF). Timing variability needs to be modelled also. Further experimentation is needed to determine whether increased speaking rate more nearly retains constant phase or constant absolute durational separation of articulatory events and whether or not some gestures can disappear under the relaxed auditory criteria of a casual style of speech. Two examples are presented, from the Leeds articulatory modelling, of the ways in which the phonological system of the language interacts with the processes and constraints of the speech production system to yield context-dependent variation for individual phonemes.

On automatic estimation of articulatory parameters in a text-to-speech system

It is often conjectured that articulatory modelling of speech is desirable, if training data can be obtained. Articulatory parameters are likely to interpolate over long intervals, and reproduce many acoustic details from simple constraints governed by physical laws. For text-to-speech application, it may be easier to formulate rules in the articulatory domain. An analysis synthesis scheme for estimating the phoneme-level articulatory parameters to obtain best fits to natural speech, in the context of a text-to-speech (TTS) system, is presented. The working units of optimization are the parameters of an articulatory model (one vector per phoneme) and vectors of time and speed of transition for each parameter. The TTS system is used to initialize these parameters. Problems encountered in analysis synthesis are due to the following: (1) Automatic optimization methods work well only when the model is capable of matching the data closely. Otherwise, best solutions according to an objective error criteria are not necessarily sensible compromises according to perception. (2) Very different vocal tract shapes can produce speech spectra that are very similar (non-uniqueness). (3) Local minima can result from artifacts of the comparison strategy or due to the non-uniqueness in the spectrum-to-area transformation. Our solutions mainly consist of the following: (1) In each phoneme, adapt only those variables that have a large effect on the acoustic result (critical variables). (2) Use multiple error criteria sequentially to match large-scale spectral features in the first stage, and then, match the finer details using a high resolution criterion. It is demonstrated that good quality speech synthesis is possible through a phoneme-level control of an articulatory model. The synthesis is devoid of any artifacts, even in transitions, due to parameterization at the phoneme level. The quality of synthetic speech is comparable (at least in non-nasal voiced segments) to that of a good quality, frame-by-frame linear predictive coder. The models need to be embellished to reproduce other phonemes closely.

An efficient algorithm for articulatory synthesis.

One of the methods for efficient computation of articulatory synthesis is to discretize the area function into a small number of sections such as 10 or 12 resulting in a poor spatial resolution. The articulatory data have to be represented by 30 or more sections for a good spatial resolution requiring a large number of computations. However, good quality speech synthesis has been obtained using formant or LPC-based methods with filters of 10 or 12 coefficients for voiced sounds and 4 or 6 coefficients for unvoiced sounds. It is proposed that a computationally efficient method be used to obtain high quality synthesis using an articulatory model with a large number of sections (30 or more). It can be shown that when the adjacent sections are of equal area, the computations can be pruned without any loss of accuracy. For a given computational load, using an effective quantization rather than discretization of area function, one can achieve much lower spectral error in the transfer function. Trade-off relationships between spectral error and computational load as a function of quantization and discretization are reported.

Teaching Consonants to Profoundly Hearing-Impaired Speakers Using Palatometry

Five profoundly hearing-impaired children were taught the consonants /t,d,k,g,s,z integral of/ using palatometry. Changes in linguapalatal contact patterns and listener perceptions showed significant improvement in the place and manner of consonants produced by all subjects. Velar stops were as easily and accurately learned as alveolar stops. Distinctive sibilants were also found by the end of training. Sounds not previously present in a subject's phonetic repetoire were learned more accurately than those present but inaccurate prior to therapy. Voicing errors persisted. Two of the subjects showed evidence of newly established, unsolicited coarticulated movements. The results indicated that visual articulatory modeling and feedback of linguapalatal contact patterns is an effective means of teaching consonants and improving speech intelligibility.

On articulatory and acoustic variabilities

In our previous studies, an articulatory model has been established by applying a factor analysis to cineradiographic and labiofilm data. Seven parameters are sufficient to specify the entire vocal tract shape with reasonable accuracy. With this model, the temporal variations are described in terms of the successive frame-by-frame samples of these seven articulatory parameters. In this paper, temporal patterns of two of the seven parameters, corresponding to the jaw and tongue-dorsum positions during the production of the two unrounded vowels /i/ and /a/ were examined in detail. Articulatory “target” values of these two parameters for each of the two vowels embedded in different phonetic contexts vary significantly more than the corresponding acoustic targets expressed in terms of first (F 1 ) and second (F 2 ) formant frequency values. An index is proposed to express quantitatively the degree of variability for both articulatory and acoustic targets. Using this index, articulatory target variability is found to be about twice as large as acoustic target variability. For each vowel, the distribution of the targets in the jaw/tongue-dorsum space exhibits a fairly linear relationship between the two dimensions, which can be regarded as an indication of the interarticulatory coordination between the jaw and the tongue-dorsum. When the articulatory variability index is recalculated by subtracting the predicted effects of coordination, its value becomes comparable to that of the acoustic (F 1 /F 2 ) variability. Further calculations with the model suggest that the coordination is used by the speaker to achieve acoustic compensation. These findings confirm that vowel production is indeed compensatory and suggest that compensation can be accurately modeled without resorting to acoustic or other sensory feedback mechanism.

Coarticulation rules in an articulatory model

Coarticulation rules are of primary importance for good speech quality in an articulation-based speech synthesis system. This is demonstrated by a simple perception experiment. Auditory feedback strategies combined with general phonetic knowledge about the relation between articulation and acoustics provide a reliable basis for the generation of such coarticulation rules.

Three‐dimensional reconstruction of the palate and tongue during the production of sustained vowels

The purpose of the present investigation was to reconstruct the three‐dimensional (3‐D) shape of the oral cavity during the production of sustained vowels. Using a specially developed palatal tracing wand and ultrasonic transducer localization device, real time convex phase array images of the tongue were digitized and stored on an IBM PC/AT along with the 3‐D shape of the palate. The reconstructed 3‐D shapes of the tongue surface relative to the 3‐D shape of the palate will be presented. Data analysis strategies will be discussed along with implications for speech articulatory modeling.

Neural networks in articulatory speech analysis/synthesis

A major difficulty in articulatory analysis/synthesis is the estimation of vocal-tract parameters from input speech. The use of neural networks to extract these parameters is more attractive than codebook look-up due to the lower computational complexity. For example, a multilayer perceptron (MLP) with two hidden layers, trained and evaluated on a small data set was shown to perform a reasonable mapping of acousticto-geometric parameters. Increasing the training data, however, revealed ambiguity in the mapping that could not be resolved by a single network. This paper addresses the problem using an assembly of MLP's, each designated to a specific region in the articulatory space. Training data were generated by randomly sampling the parameters of an articulatory model of the vocal system. The resultant vocal-tract shapes were clustered into 128 regions, and an MLP with one hidden layer was assigned to each of these regions for mapping 18 cepstral coefficients into ten tract areas, and a nasalization parameter. Networks were selected by dynamic programming, and were used to control a time-domain articulatory synthesizer. After training, significant perceptual and objective improvements were achieved relative to using a single MLP. Comparable performance to codebook look-up with dynamic programming was obtained. This model, however, requires only 4% of the storage needed for the codebook, and performs the mapping faster by a factor of 20.