Articulatory Model Research Articles

Visual articulatory model and its application to therapy of speech disorders: a pilot study

A visual articulatory model based on static MRI-data of isolated sounds and its application in therapy of speech disorders is described. The model is capable of generating video sequences of articulatory movements or still images of articulatory target positions within the midsagittal plane. On the basis of this model (1) a visual stimulation technique for the therapy of patients suffering from speech disorders and (2) a rating test for visual recognition of speech movements was developed. Results indicate that patients produce recognition rates above level of chance already without any training and that patients are capable of increasing their recognition rate over the time course of therapy significantly.

Towards a 3D articulatory model of velum based on MRI and CT images

This paper describes the processing of MRI and CT images needed for developing a 3D linear articulatory model of velum. The 3D surface that defines each organ constitutive of the vocal and nasal tracts is extracted from MRI and CT images recorded on a subject uttering a corpus of artificially sustained French vowels and consonants. First, the 2D contours of the organs have been manually extracted from the corresponding images, expanded into 3D contours, and aligned in a common 3D coordinate system. Then, for each organ, a generic mesh has been chosen and fitted by elastic deformation to each of the 46 3D shapes of the corpus. This has finally resulted in a set of organ surfaces sampled with the same number of 3D vertices for each articulation, which is appropriate for Principal Component Analysis or linear decomposition. The analysis of these data has uncovered two main uncorrelated articulatory degrees of freedom for the velum's movement. The associated parameters are used to control the model. We have in particular investigated the question of a possible correlation between jaw / tongue and velum's movement and have not find more correlation than the one found in the corpus.

A production account of sound changes affecting diphthongs and triphthongs in Romance

This paper is an investigation processes of sound change (i.e., assimilations, dissimilations, elisions) affecting diphthongs and triphthongs derived from Latin mid low vowels in Romance. This analysis is carried out with reference to the Degree of Articulatory Constraint model of coarticulation according to which adaptation effects between consecutive segments in the speech chain, as well as their regressive or progressive direction, are determined by the requirements imposed by speakers upon the articulatory structures. Several findings are consistent with this theoretical framework, namely, assimilatory vowel raising in diphthongs and triphthongs appear to be facilitated by a homorganic onglide in accordance with the prominence of the carryover effects associated with the articulatory gestures for /j/ and /w/, and dissimilatory vowel lowering is not prone to be implemented in rising diphthongs with a (mid) high front vowel perhaps since speakers attempt to avoid vocalic sequences imposing high articulatory demands.

Construction and control of a physiological articulatory model.

A physiological articulatory model has been constructed using a fast computation method, which replicates midsagittal regions of the speech organs to simulate articulatory movements during speech. This study aims to improve the accuracy of modeling by using the displacement-based finite-element method and to develop a new approach for controlling the model. A "semicontinuum" tongue tissue model was realized by a discrete truss structure with continuum viscoelastic cylinders. Contractile effects of the muscles were systemically examined based on model simulations. The results indicated that each muscle drives the tongue toward an equilibrium position (EP) corresponding to the magnitude of the activation forces. The EPs shifted monotonically as the activation force increased. The monotonic shift revealed a unique and invariant mapping, referred to as an EP map, between a spatial position of the articulators and the muscle forces. This study proposes a control method for the articulatory model based on the EP maps, in which co-contractions of agonist and antagonist muscles are taken into account. By utilizing the co-contraction, the tongue tip and tongue dorsum can be controlled to reach their targets independently. Model simulation showed that the co-contraction of agonist and antagonist muscles could increase the stability of a system in dynamic control.

A model of acoustic interspeaker variability based on the concept of formant-cavity affiliation.

A method is proposed to model the interspeaker variability of formant patterns for oral vowels. It is assumed that this variability originates in the differences existing among speakers in the respective lengths of their front and back vocal-tract cavities. In order to characterize, from the spectral description of the acoustic speech signal, these vocal-tract differences between speakers, each formant is interpreted, according to the concept of formant-cavity affiliation, as a resonance of a specific vocal-tract cavity. Its frequency can thus be directly related to the corresponding cavity length, and a transformation model can be proposed from a speaker A to a speaker B on the basis of the frequency ratios of the formants corresponding to the same resonances. In order to minimize the number of sounds to be recorded for each speaker in order to carry out this speaker transformation, the frequency ratios are exactly computed only for the three extreme cardinal vowels [i, a, u] and they are approximated for the remaining vowels through an interpolation function. The method is evaluated through its capacity to transform the (F1,F2) formant patterns of eight oral vowels pronounced by five male speakers into the (F1,F2) patterns of the corresponding vowels generated by an articulatory model of the vocal tract. The resulting formant patterns are compared to those provided by normalization techniques published in the literature. The proposed method is found to be efficient, but a number of limitations are also observed and discussed. These limitations can be associated with the formant-cavity affiliation model itself or with a possible influence of speaker-specific vocal-tract geometry in the cross-sectional direction, which the model might not have taken into account.

Model‐based speaker normalization methods for speech recognition

AbstractA speaker normalization method using a speech generation model is proposed in order to achieve high‐performance speaker adaptation with a small amount of adaptation data. The speaker‐ and phoneme‐dependent vocal tract area function is approximated by the corresponding area function produced by the articulatory model of a standard speaker, combined with phoneme‐independent feature quantities of the vocal‐tract shape of the normalized target speaker as estimated from the formant frequencies of two vowels. The frequency warping functions are determined from the formant frequencies of speech calculated from the vocal‐tract area functions thus obtained, and normalization of the uttered speech is performed by stretching the speech spectrum in the frequency‐axis direction. Continuous phoneme recognition experiments using phoneme connection rules show that the recognition error using a gender‐dependent model is reduced by about 30% in the proposed method and that recognition performance superior to that of vocal‐tract length normalization is obtained. The recognition performance of the proposed method is also equivalent to that of speaker adaptation by moving vector field smoothing (VFS) using 10 phonetically balanced sentences, showing that high‐performance speaker adaptation using a small amount of adaptation data can be achieved by the proposed method. © 2003 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 86(2): 45–56, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjb.10119

Acoustic effects of speaking rate changes in articulatory models

Speaking rate effects on articulatory phasing, velocity, and magnitude have been reported in the speech-kinematic literature. Yet little is known about the acoustic consequences of manipulating these dimensions. Using an articulatory modeling method developed by Milenkovic, simulations of the acoustic effects of changes in articulatory phasing, velocity, and magnitude have been explored. Articulatory models have been optimized for speakers from the University of Wisconsin X-Ray Microbeam Speech Production Database. Preliminary work has revealed that models of rate change which use only a single dimension (e.g., phasing) may reveal less plausible acoustic solutions than models involving multiple dimensions. Using sample productions of ‘‘a boy’’ to determine articulatory parameter time histories, the acoustic consequences of rate change simulations have been described by the F2 value at onset (F2on), the maximum F2 value (F2tar), and the shape of the F2 trajectory throughout the diphthong. Speaking rate models that combine velocity and phase changes to articulatory parameters result in a reduction in the magnitude of the F2on effect, near elimination of any effect on F2tar, and a relative increase in the homogeneity of the F2 trajectory shapes across speaking rates when compared with phase-only models. [Work supported by NIH (NIDCD) P60 DC1409-10.]

Geometrical analysis of the tongue muscles based on MRI and functional modeling of the tongue

The geometry of the tongue muscles during production of Japanese five vowels was analyzed using MRI toward a realistic design of our physiological articulatory model [Dang et al., Acoust. Sci. Technol. 22, 415–425 (2001)]. Articulatory changes in shape of the muscle bundles and of the lingual arteries indicated two findings. (1) The styloglossus (SG) was previously thought to cause high-back tongue position by pulling its insertion point in the tongue. However, the MRI-based geometry of the extra-lingual part of SG was found to be almost constant across vowels. This implies that SG may contribute to tongue deformation mainly by the contraction of its intra-lingual part. (2) The right and left arterial landmarks within the tongue tissue indicated tissue deformation across vowels in the lateral direction. The distance between the landmarks was larger in front vowels, and smaller in back vowels. That is, front vowels are associated with lateral expansion, and back vowels are accompanied by medial compression. With the above results, the physiological model of the tongue was revised to have more layers so that the natural geometry of the muscles could be implemented.

Improvements of a physiological articulatory model in construction and control strategy

A physiological articulatory model that represents midsagittal regions of the speech organs has been used to simulate articulatory movements during speech. A fast simulation method was realized by modeling both soft tissues and rigid organs by mass-points and springs. It is, however, difficult to exactly represent characteristics of a continuous body using the network consisting of mass-points and volume-less springs. To model soft tissues accurately, the springs are replaced by viscoelastic pillars with both length and thickness, and the masses are distributed over the pillars. Each eight-vertex mesh of the soft tissue is filled with 28 pillars that connect the vertices one to another on the edges and surfaces, and inside diagonally. The initial thickness of the pillars for each mesh is calculated by making the total volume of the pillars equal to the mesh volume. During tongue deformation, pillars change in the length and thickness according to the physical laws in continuous bodies. The articulator model is driven by muscle contraction parameters. Muscle activation signals are generated based on a target-reaching task, and fed to drive the model to approach the targets. This study further accounts for the agonist–antagonist mechanism of the muscles in the control strategy according to EMG analysis. [Work supported in part by TAO, Japan.]

Three-dimensional linear articulatory modeling of tongue, lips and face, based on MRI and video images

In this study, previous articulatory midsagittal models of tongue and lips are extended to full three-dimensional models. The geometry of these vocal organs is measured on one subject uttering a corpus of sustained articulations in French. The 3D data are obtained from magnetic resonance imaging of the tongue, and from front and profile video images of the subject's face marked with small beads. The degrees of freedom of the articulators, i.e., the uncorrelated linear components needed to represent the 3D coordinates of these articulators, are extracted by linear component analysis from these data. In addition to a common jaw height parameter, the tongue is controlled by four parameters while the lips and face are also driven by four parameters. These parameters are for the most part extracted from the midsagittal contours, and are clearly interpretable in phonetic/biomechanical terms. This implies that most 3D features such as tongue groove or lateral channels can be controlled by articulatory parameters defined for the midsagittal model. Similarly, the 3D geometry of the lips is determined by parameters such as lip protrusion or aperture, that can be measured from a profile view of the face.

The potential Neandertal vowel space was as large as that of modern humans

Since Lieberman and Crelin (1971) postulated the theory that Neandertals “could not produce the range of sounds that characterize human speech”, the potential speech capability of Neandertals has been the subject of hot debate. Lieberman and Crelin claimed that the development of a low laryngeal position was a necessary condition for the realization of a sufficient number of vocalic contrasts, since the potential vowel space was enlarged due to an enlarged pharyngeal cavity. Like newborn infants, Neandertals did not possess this “anatomical basis of speech”, and therefore could not speak. Lieberman and Crelin further claimed that this fact may have caused the, otherwise mysterious, extinction of the Neandertal. In this study, we refute the articulatory and acoustic arguments developed by Lieberman and Crelin in their theory. Using a new anthropomorphic articulatory model, we infer that the vowel space of the Neandertal male was no smaller than that of a modern human, and we present vowel simulations to corroborate this hypothesis. Our study is strictly limited to the morphological and acoustic aspects of the vocal tract, and we cannot therefore offer any definitive answer to the question of whether Neandertals spoke or not. However, we do feel safe in claiming that Neandertals were not morphologically handicapped for speech. A low larynx (and large pharynx) cannot be considered to be the “anatomical prerequisites for producing the full range of human speech”. There is, therefore, no reason to believe that the lowering of the larynx and a concomitant increase in pharynx size are necessary evolutionary preadaptations for speech.

Estimation of vocal tract shapes from speech sounds with a physiological articulatory model

A 3D physiological articulatory model based on volumetric MRI data from a male speaker was used to estimate vocal tract shapes from speech sounds. The advantages of using the model for the inverse estimation are that the model is equipped with the morphological and dynamic constraints that are commonly used for such estimation and possesses the physiological constraints that are involved in human articulation. In this study, a dynamic muscle workspace was introduced to account for temporal variations of the muscle orientation with articulatory movements, and a multipoint control strategy was proposed for flexible control of the tongue tip and tongue dorsum. The control points were used as articulatory parameters, and formants were chosen as acoustic parameters. An articulatory constraint between the F1−F2 difference and tongue dorsum position was introduced in mapping formant patterns to control point positions, where the constraint was obtained based on X-ray microbeam data recorded from the target speaker and five other male speakers. The proposed estimation method was evaluated using vowel-to-vowel sequences. For the target speaker of the model, the average estimation error was 0.16 cm for the vocal tract shapes, and 1.8% for the four lower formants. This implies that our physiological articulatory model can be a valuable tool for the inverse estimation.

Prosody modeling with soft templates

This paper describes a novel prosody generation model. We intend it to broadly support many linguistic theories and multiple languages, for the model imposes no restriction on accent categories and shapes. This capability is crucial to the next generation of text-to-speech systems that will need to synthesize intonation variations for different speech acts, emotions, and styles of speech. The system supports mark-up tags that are mathematically defined and generate f 0 deterministically. Underlying the tags is an articulatory model of accent interaction which balances physiological and communication constraints. We specify the model by way of an algorithm for calculating the pitch, and by way of examples. The model allows localized, linguistically reasonable tags, and is suitable for a data-driven fitting process.

Auditory normalization of French vowels synthesized by an articulatory model simulating growth from birth to adulthood.

The present article aims at exploring the invariant parameters involved in the perceptual normalization of French vowels. A set of 490 stimuli, including the ten French vowels /i y u e ø o E oe (inverted c) a/ produced by an articulatory model, simulating seven growth stages and seven fundamental frequency values, has been submitted as a perceptual identification test to 43 subjects. The results confirm the important effect of the tonality distance between F1 and f0 in perceived height. It does not seem, however, that height perception involves a binary organization determined by the 3-3.5-Bark critical distance. Regarding place of articulation, the tonotopic distance between F1 and F2 appears to be the best predictor of the perceived front-back dimension. Nevertheless, the role of the difference between F2 and F3 remains important. Roundedness is also examined and correlated to the effective second formant, involving spectral integration of higher formants within the 3.5-Bark critical distance. The results shed light on the issue of perceptual invariance, and can be interpreted as perceptual constraints imposed on speech production.

Relating articulation and acoustics through a sinusoidal description of vocal tract shape

A description of vocal tract shape for vowels is given in terms of a weighted sum of two sinusoids, displaced from each other along the length of the tract. The impetus for the description comes from uncovering a particularly simple relationship between F2 and a function on the parameters of a model of articulatory data. When the statistically derived articulatory model is simplified as displaced sinusoids, the relationship can be easily understood in terms of the physics of acoustic tubes. Thus, while the description is based on articulatory data, it relates simply to acoustics. The function (the sum of the weights of the two sinusoids) creates sets of articulatory shapes that can be said to trade in the amount of tongue front raising and the amount of tongue back raising to produce a constant F2. Alternatively, these equivalence classes of shapes can be said to trade in place of articulation and amount of constriction: this is so despite the fact that neither of these two variables are explicit parameters in generating the shapes. In embodying this compensatory property, it is also a description that is consistent with theories of motor control based on specifying functional equivalence classes, which are controlled with few degrees of freedom and allow context-conditioned variability.

Dynamic articulatory model based on multidimensional invariant-feature task representation.

A dynamic model of articulatory movements is introduced. The research presented herein focuses on the method of representing the phonemic tasks, i.e., phoneme-specific articulatory targets. Phonemic tasks in our model are formally defined using invariant features of articulatory posture. The invariant features used in the model are characterized by the linear transformation of articulatory variables and found using a statistical analysis of measured articulatory movements, in which the articulatory features with minimum variability are taken to be the invariant features. Articulatory movements making vocal-tract constrictions or relative movements among articulators reflecting task-sharing structures are typical examples of the features found to have low variability. In the trajectory formation of articulatory movements, the dimension number of the phonemic task is set at a smaller value than that of articulatory variables. Consequently, the kinematic states of the articulators are partly constrained at given time instants by a sequence of phonemic tasks, and there remain unconstrained degrees of freedom of articulatory variables. Articulatory movements are determined so that they simultaneously satisfy given phonemic tasks and dynamic smoothness constraints. The dynamic smoothness constraints coupled with the underspecified phonemic targets allow our model to explain contextual articulatory variability using context-independent phonemic tasks. Finally, the capability of the model for predicting actual articulatory movements is quantitatively investigated using empirical articulatory data.

Linear degrees of freedom in speech production: analysis of cineradio- and labio-film data and articulatory-acoustic modeling.

The following contribution addresses several issues concerning speech degrees of freedom in French oral vowels, stop, and fricative consonants based on an analysis of tongue and lip shapes extracted from cineradio- and labio-films. The midsagittal tongue shapes have been submitted to a linear decomposition where some of the loading factors were selected such as jaw and larynx position while four other components were derived from principal component analysis (PCA). For the lips, in addition to the more traditional protrusion and opening components, a supplementary component was extracted to explain the upward movement of both the upper and lower lips in [v] production. A linear articulatory model was developed; the six tongue degrees of freedom were used as the articulatory control parameters of the midsagittal tongue contours and explained 96% of the tongue data variance. These control parameters were also used to specify the frontal lip width dimension derived from the labio-film front views. Finally, this model was complemented by a conversion model going from the midsagittal to the area function, based on a fitting of the midsagittal distances and the formant frequencies for both vowels and consonants.

A physiological articulatory model for simulating speech production process.

A physiological articulatory model has been developed to simulate the dynamic actions of speech organs during speech production. This model represents the midsagittal region of the tongue, jaw, hyoid bone, and the vocal tract wall in three dimensions. The soft tissue of the tongue is outlined in the midsagittal and parasagittal planes of MR images obtained from a male Japanese speaker, and constructed as a 2-cm thick layer. The palatal and pharyngeal walls are constructed as a hard shell of a 3-cm left-to-right width. The jaw and hyoid bone are modelled to yield rotation and translation motions. The muscle structure in the model is identified based on volumetric MR images of the same speaker. A fast simulation method is developed by modeling both the soft tissue and rigid organs using mass-points with two types of links: viscoelastic springs with a proper stiffness for connective tissue, and extremely high stiffness for bony organs. Muscle activation signals are generated by a model control strategy based on the target-reaching task, and then fed to drive the model to approach the targets. The model demonstrated realistic behaviors similar to coarticulation in human speech production (Dang and Honda, 1998, 1999, 2000).

Deriving Vowel-to-Vowel Transitions with the Apex Articulatory Model

Formant transitions, i.e., how formant frequencies change over time, carry much of the information conveyed in both speech and singing. As formant frequencies are controlled by the positions of the articulators in the vocal tract, formant transitions correspond to articulatory movements. The purpose of the present study was to explore the potential of an articulatory model to interpret formant frequency data of Swedish V 1 V 2 sequences in terms of articulatory movements. The strategy was to generate articulatory pathways for 3 V 1 V 2 pairs, V = [I] [?] [u], using the APEX articulatory model (Lindblom & Sundberg, 1971; Stark et al., 1996, 1999). The acoustic characteristics associated with these pathways were then compared with those observed in utterances produced by a Swedish speaker. The results for [ui] and [?I] suggest that the articulatory pathways between these vowels are realized via a slight tongue deactivation.

A self-learning predictive model of articulator movements during speech production.

A model is presented which predicts the movements of flesh points on the tongue, lips, and jaw during speech production, from time-aligned phonetic strings. Starting from a database of x-ray articulator trajectories, means and variances of articulator positions and curvatures at the midpoints of phonemes are extracted from the data set. During prediction, the amount of articulatory effort required in a particular phonetic context is estimated from the relative local curvature of the articulator trajectory concerned. Correlations between position and curvature are used to directly predict variations from mean articulator positions due to coarticulatory effects. Use of the explicit coarticulation model yields a significant increase in articulatory modeling accuracy with respect to x-ray traces, as compared with the use of mean articulator positions alone.