Articulatory Movement Data Research Articles

Identifying Psychological Symptoms Based on Facial Movements

This study investigated the relationship between clearly produced and plain citation form speech styles and motion of visible articulators. Using state-of-the-art computer-vision and image processing techniques, we examined both front and side view videos of speakers’ faces while they recited six English words (keyed, kid, cod, cud, cooed, could) containing various vowels differing in visible articulatory features (e.g., lip spreading, lip rounding, jaw displacement), and extracted measurements corresponding to the lip and jaw movements. We compared these measurements in clear and plain speech produced by 18 native English speakers. Based on statistical analyses, we found significant effects of speech style as well as speaker gender and saliency of visual speech cues. Compared to plain speech, we found in clear speech longer duration, greater vertical lip stretch and jaw displacement across vowels, greater horizontal lip stretch for front unrounded vowels, and greater degree of lip rounding and protrusion for rounded vowels. Additionally, greater plain-to-clear speech modifications were found for male speakers than female speakers. These articulatory movement data demonstrate that speakers modify their speech productions in response to communicative needs in different speech contexts. These results also establish the feasibility of utilizing novel computerized facial detection techniques to measure articulatory movements.

An Application of Network Science to Phonological Sequence Learning in Children With Developmental Language Disorder.

Network science has been a valuable tool in language research for investigating relationships between complex linguistic elements but has not yet been applied to sound sequencing in production. In the present work, we used standard error-based accuracy and articulatory kinematic approaches as well as novel measures from network science to evaluate variability and sequencing errors in speech production in children with developmental language disorder (DLD; aka specific language impairment). Twelve preschoolers with DLD and 12 age-matched controls participated in a 3-day novel word learning study. Transcription and articulatory movement data were collected to measure accuracy and variability of productions, and networks of speech productions were generated to analyze syllable co-occurrence patterns. Results indicated that children with DLD were less accurate than children with typical language at the segmental level. Crucially, these findings did not align with performance at the articulatory level, where there were no differences in movement variability between children with DLD and those with typical language. Network analyses revealed characteristics that were not captured by standard measures of phonetic accuracy, including a larger inventory of syllable forms, more connections between the forms, and less consistent production patterns. Network science provides significant insights into phonological learning trajectories in children with DLD and their typically developing peers. Importantly, errors in word production by children with DLD do not surface as a result of weakness in articulatory control. Instead, results suggest that speech errors in DLD may relate to deficits in sound sequencing.

Recognizing Whispered Speech Produced by an Individual with Surgically Reconstructed Larynx Using Articulatory Movement Data.

Individuals with larynx (vocal folds) impaired have problems in controlling their glottal vibration, producing whispered speech with extreme hoarseness. Standard automatic speech recognition using only acoustic cues is typically ineffective for whispered speech because the corresponding spectral characteristics are distorted. Articulatory cues such as the tongue and lip motion may help in recognizing whispered speech since articulatory motion patterns are generally not affected. In this paper, we investigated whispered speech recognition for patients with reconstructed larynx using articulatory movement data. A data set with both acoustic and articulatory motion data was collected from a patient with surgically reconstructed larynx using an electromagnetic articulograph. Two speech recognition systems, Gaussian mixture model-hidden Markov model (GMM-HMM) and deep neural network-HMM (DNN-HMM), were used in the experiments. Experimental results showed adding either tongue or lip motion data to acoustic features such as mel-frequency cepstral coefficient (MFCC) significantly reduced the phone error rates on both speech recognition systems. Adding both tongue and lip data achieved the best performance.

Examining visible articulatory features in clear and plain speech

This study investigated the relationship between clearly produced and plain citation form speech styles and motion of visible articulators. Using state-of-the-art computer-vision and image processing techniques, we examined both front and side view videos of speakers’ faces while they recited six English words (keyed, kid, cod, cud, cooed, could) containing various vowels differing in visible articulatory features (e.g., lip spreading, lip rounding, jaw displacement), and extracted measurements corresponding to the lip and jaw movements. We compared these measurements in clear and plain speech produced by 18 native English speakers. Based on statistical analyses, we found significant effects of speech style as well as speaker gender and saliency of visual speech cues. Compared to plain speech, we found in clear speech longer duration, greater vertical lip stretch and jaw displacement across vowels, greater horizontal lip stretch for front unrounded vowels, and greater degree of lip rounding and protrusion for rounded vowels. Additionally, greater plain-to-clear speech modifications were found for male speakers than female speakers. These articulatory movement data demonstrate that speakers modify their speech productions in response to communicative needs in different speech contexts. These results also establish the feasibility of utilizing novel computerized facial detection techniques to measure articulatory movements.

The use of articulatory movement data in speech synthesis applications: An overview — Application of articulatory movements using machine learning algorithms —

The use of articulatory movement data in speech synthesis applications: An overview — Application of articulatory movements using machine learning algorithms —

Articulatory Attributes in Korean Nonassimilating Contexts

This study examined several kinematic properties of the primary articulator (the tongue dorsum) and the supplementary articulator (the jaw) in the articulation of the voiceless velar stop (/k/) within nonassimilating contexts. We examined in particular the spatiotemporal properties (constriction duration and constriction maxima) from the constriction onset to the constriction offset by analyzing a velar (/k/) followed by the coronal fricative (/s/), the coronal stop (/t/), and the labial (/p/) in across-word boundary conditions (/k#s/, /k#t/, and /k#p/). Along with these measurements, we investigated intergestural temporal coordination between C1 and C2 and the jaw articulator in relation to its coordination with the articulation of consonant sequences. The articulatory movement data was collected by means of electromagnetic midsagittal articulometry (EMMA). Four native speakers of Seoul Korean participated in the laboratory experiment. The results showed several characteristics. First, a velar (/k/) in C1 was not categorically reduced. Constriction duration and constriction degree of the velar (/k/) were similar within nonassimilating contexts (/k#s/

Processing speech signal using auditory-like filterbank provides least uncertainty about articulatory gestures

Understanding how the human speech production system is related to the human auditory system has been a perennial subject of inquiry. To investigate the production-perception link, in this paper, a computational analysis has been performed using the articulatory movement data obtained during speech production with concurrently recorded acoustic speech signals from multiple subjects in three different languages: English, Cantonese, and Georgian. The form of articulatory gestures during speech production varies across languages, and this variation is considered to be reflected in the articulatory position and kinematics. The auditory processing of the acoustic speech signal is modeled by a parametric representation of the cochlear filterbank which allows for realizing various candidate filterbank structures by changing the parameter value. Using mathematical communication theory, it is found that the uncertainty about the articulatory gestures in each language is maximally reduced when the acoustic speech signal is represented using the output of a filterbank similar to the empirically established cochlear filterbank in the human auditory system. Possible interpretations of this finding are discussed.

Overreliance on auditory feedback may lead to sound/syllable repetitions: Simulations of stuttering and fluency-inducing conditions with a neural model of speech production

This paper investigates the hypothesis that stuttering may result in part from impaired readout of feedforward control of speech, which forces persons who stutter (PWS) to produce speech with a motor strategy that is weighted too much toward auditory feedback control. Over-reliance on feedback control leads to production errors which if they grow large enough, can cause the motor system to “reset” and repeat the current syllable. This hypothesis is investigated using computer simulations of a “neurally impaired” version of the DIVA model, a neural network model of speech acquisition and production. The model's outputs are compared to published acoustic data from PWS’ fluent speech, and to combined acoustic and articulatory movement data collected from the dysfluent speech of one PWS. The simulations mimic the errors observed in the PWS subject's speech, as well as the repairs of these errors. Additional simulations were able to account for enhancements of fluency gained by slowed/prolonged speech and masking noise. Together these results support the hypothesis that many dysfluencies in stuttering are due to a bias away from feedforward control and toward feedback control. Educational objectives : The reader will be able to (a) describe the contribution of auditory feedback control and feedforward control to normal and stuttered speech production, (b) summarize the neural modeling approach to speech production and its application to stuttering, and (c) explain how the DIVA model accounts for enhancements of fluency gained by slowed/prolonged speech and masking noise.

Control of Anthropomorphic Talking Robot based on Human Articulatory Movement Data(<Feature Article>Methodology for Speech Physiology Research)

Control of Anthropomorphic Talking Robot based on Human Articulatory Movement Data(<Feature Article>Methodology for Speech Physiology Research)

A modeling investigation of articulatory variability and acoustic stability during American English /r/ production

This paper investigates the functional relationship between articulatory variability and stability of acoustic cues during American English /r/ production. The analysis of articulatory movement data on seven subjects shows that the extent of intrasubject articulatory variability along any given articulatory direction is strongly and inversely related to a measure of acoustic stability (the extent of acoustic variation that displacing the articulators in this direction would produce). The presence and direction of this relationship is consistent with a speech motor control mechanism that uses a third formant frequency (F3) target; i.e., the final articulatory variability is lower for those articulatory directions most relevant to determining the F3 value. In contrast, no consistent relationship across speakers and phonetic contexts was found between hypothesized vocal-tract target variables and articulatory variability. Furthermore, simulations of two speakers' productions using the DIVA model of speech production, in conjunction with a novel speaker-specific vocal-tract model derived from magnetic resonance imaging data, mimic the observed range of articulatory gestures for each subject, while exhibiting the same articulatory/acoustic relations as those observed experimentally. Overall these results provide evidence for a common control scheme that utilizes an acoustic, rather than articulatory, target specification for American English /r/.

Patterns of jaw coarticulatory direction and linguomandibular coordination in VCV sequences

Data on lingual movement, dorsopalatal contact and F2 frequency presented in previous papers of ours (Recasens, 2002; Recasens and Pallarès, 2001; Recasens, Pallarès and Fontdevila, 1997) suggest that the degree of articulatory constraint (DAC) model accounts to a large extent for the extent and direction of tongue dorsum coarticulation in VCV and CC sequences. A goal of this investigation is to verify the predictions of this model with respect to jaw V-to-V effects in VCV sequences using articulatory movement data collected with electromagnetic articulometry (EMA).

Dynamic units in speech production: Evidence from speech production errors

Measurements of vocal tract activity during speech production show many distinct parts, all exhibiting different patterns of near-continuous motion. This description stands in sharp contrast to the description of speech as composed of a small number of discrete phonological units. In recent years the tools of dynamical systems have been brought to bear on the problem of relating these descriptions. The continuous vocal tract activity can be theoretically decomposed into discrete units of action, or gestures, each of which is a dynamical control regime for vocal tract articulators. These gestures can also serve as primitive phonological elements. Evidence for gestures as units of the speech production process has recently been obtained in experiments that measure the kinematics of speech errors produced when a simple phrase (e.g., ‘‘cop top’’) is repeated. In errors, one of the gestures comprising the phrase may be duplicated in an anomalous position in the utterance. Since the anomalous gesture is often reduced in magnitude, such errors could not be detected without articulatory movement data. The nature of the errors and their asymmetries are consistent with the hypothesis that they represent phase transitions of a system of coupled dynamic units. [Work supported by NIH.]

A gesture-based dynamic model describing articulatory movement data

A quantitative dynamic model for the description of speech movements using a critically damped linear second-order system is proposed. This six-parameter model is able to fit natural movement data with high accuracy. Since in this approach the actual location of gestural onset and gestural offset, i.e., the location and duration of gestural activation, results from the fitting procedure, no advance sectioning of movement traces is necessary. The model parameters are target position, eigenperiod, and four time parameters describing the temporal location of gestural onset and offset. The fitting algorithm is tested on simulated and natural data in order to evaluate the accuracy of the fits and the repeatability of the dynamic parameters extracted.

EMMA and x‐ray microbeam comparison

In the past, much articulatory movement data have been obtained using the x‐ray microbeam. Currently, however, magnetometer systems such as EMMA are becoming predominant. A unique data set, addressing concerns of consistency between these instruments is reported. A single speaker was recorded with both instruments reading the same utterances: ‘‘It’s a [pV’CVp] again.’’ Data from two x‐ray microbeam (XRMB) runs were collected on the same day (pellets re‐placed for the second run); 27 months later the parallel EMMA data were collected. Vertical movement of the lower lip, tongue tip, and tongue body during the VCV’s was analyzed. Extrema positions (amplitudes) and distance and time between these positions (gestural displacements and durations) were obtained. The results demonstrate that the correlation between EMMA and XRMB runs is very high, almost as high as that between the two XRMB runs. Using further analyses of variance it was concluded confidently that the results obtained using EMMA for articulatory data collection do not differ substantially from those obtained for similar data collected at different time with the x‐ray microbeam. [Work supported by NIH].

An x-ray microbeam study of Hindi vowel articulations

Using the x-ray microbeam system of the University of Wisconin at Madison, articulatory movement data were recorded from pellets attached to the tongue, upper and lower lips, lower jaw, and velum while a native speaker of Hindi produced ten oral vowels contained in non-sense words of the form bVb. The data do not support the traditional descriptions of Hindi vowels given in terms of either tongue height or vocal tract openness. For example, overall position of the tongue dorsum was higher for /e/ than /ɪ/, and similar for /ᴜ/ and /o/, and also for /ʌ/ and /ɑ/. Position of the jaw was lower for /ɔ/ than for /ɑ/. The lips were more open for /o/, /ᴜ/, and /u/ than for /ɔ/, and more open for /i/ than /ɪ/, /e/, and /ɛ/. Position of the lips for /e/ and /ɛ/ was identical. The velum showed somewhat greater elevation for /e/ than /ɪ/, and identical elevation for /ᴜ/ and /o/. The remaining relations among various structures involved in vowel production were as expected. However, overall relationships among these structures were quite complex.

Articulatory gestures as phonological units

We have argued that dynamically defined articulatory gestures are the appropriate units to serve as the atoms of phonological representation. Gestures are a natural unit, not only because they involve task-oriented movements of the articulators, but because they arguably emerge as prelinguistic discrete units of action in infants. The use of gestures, rather than constellations of gestures as in Root nodes, as basic units of description makes it possible to characterise a variety of language patterns in which gestural organisation varies. Such patterns range from the misorderings of disordered speech through phonological rules involving gestural overlap and deletion to historical changes in which the overlap of gestures provides a crucial explanatory element.Gestures can participate in language patterns involving overlap because they are spatiotemporal in nature and therefore have internal duration. In addition, gestures differ from current theories of feature geometry by including the constriction degree as an inherent part of the gesture. Since the gestural constrictions occur in the vocal tract, which can be charactensed in terms of tube geometry, all the levels of the vocal tract will be constricted, leading to a constriction degree hierarchy. The values of the constriction degree at each higher level node in the hierarchy can be predicted on the basis of the percolation principles and tube geometry. In this way, the use of gestures as atoms can be reconciled with the use of Constriction degree at various levels in the vocal tract (or feature geometry) hierarchy.The phonological notation developed for the gestural approach might usefully be incorporated, in whole or in part, into other phonologies. Five components of the notation were discussed, all derived from the basic premise that gestures are the primitive phonological unit, organised into gestural scores. These components include (1) constriction degree as a subordinate of the articulator node and (2) stiffness (duration) as a subordinate of the articulator node. That is, both CD and duration are inherent to the gesture. The gestures are arranged in gestural scores using (3) articulatory tiers, with (4) the relevant geometry (articulatory, tube or feature) indicated to the left of the score and (5) structural information above the score, if desired. Association lines can also be used to indicate how the gestures are combined into phonological units. Thus, gestures can serve both as characterisations of articulatory movement data and as the atoms of phonological representation.

Wisconsin x‐ray microbeam system: Articulatory pellet tracking specifications

The x‐ray microbeam system for studying tongue movements and other articulatory gestures has been in routine operation for the past year. The precision and accuracy with which the system can track pellets is dependent upon a number of factors. These include the x‐ray system operating parameters and characteristics, pellet size, background conditions, and image processing algorithms used for real‐time tracking. The results of studies that address these issues, and their importance with respect to interpretation and use of articulatory movement data, will be discussed. Other methodological issues of importance to potential users include head movement correction via the use of reference pellets and empirical specification of anatomical reference planes for expressing the acquired speech movement data. The x‐ray dosage incurred by subjects studied to date has also been carefully examined. Even when using peak spatial dose, a conservative estimate, subjects can participate in repeated speech production studies without exceeding the yearly Federal whole body dose limit. The status of the system and planned updates will also be presented. [Support for this facility is provided by NINCDS (NS‐16373).]

A dynamic analysis of reiterant speech production in three languages

Previously, supralaryngeal articulatory motion in reiterant speech produced by speakers of English and Japanese in terms of an underlying dynamical model [Kelso et al., 1. Acoust. Soc. Am. 77, 266–280 (1985); Vatikiotis‐Bateson and Kelso, J. Acoust. Soc. Am. Suppl. 1 78, S38 (1985)] has been analyzed. Here, the analysis is extended to French, a language whose temporal structure is believed to be different from that of English or Japanese, and a preliminary comparison of movements of the lower‐lip/jaw complex for speakers of the three languages is presented. The cross‐language comparison is based on articulatory movement data from 13 speakers (five English, five Japanese, and three French). Analysis of reiterant /ba/ and /ma/ productions reveals that French speakers make language‐specific linguistic (e.g., stress) and performance (e.g., speaking rate) distinctions that are qualitatively similar in terms of kinematic relations to those observed for Japanese and English. Quantitative differences due to the intrinsic temporal properties of each language are also evident (e.g., in the slope of the peak velocity‐displacement relation). It is suggested that the same abstract dynamical model, appropriately parameterized for each language, can characterize the observed articulatory behavior. [Work supported by NINCDS and ONR.]

Initial observations concerning developmental characteristics of labio-mandibular kinematics.

Acoustic measurements have shown that children's segment durations are often longer than those of adults. It is unclear from previous research, however, why this occurs. One possibility is that children's less mature peripheral speech mechanisms limit the rate at which they can perform articulatory movements and that this results in longer acoustic segment durations. Another possible cause of children's longer segment durations is that "higher-order," organizational factors might affect their ability to plan and sequence phonetic strings. This too could limit the rate at which they produce speech gestures, thus resulting in longer segment durations. The present study attempted to clarify such issues by obtaining articulatory movement data from four children and three adults using strain gage instrumentation to monitor superior-inferior lip and jaw displacement and velocity. Results indicate that the children tended to move their articulators more slowly than the adults, despite the fact that both groups exhibited comparable articulatory displacements. Differences observed between the adults and children concerning vocal tract size and movement variability indicate that "lower-level," physical factors may be partially involved in children's slower articulatory movements. However, differences between the two groups regarding anticipation of certain articulatory sequences suggest that "higher-order," organizational effects may also be associated with children's slower movements and, therefore, their longer segment durations.