Electromagnetic Midsagittal Articulography Research Articles

Simulation of vowel-vowel utterances using a 3D biomechanical-acoustic model.

A link is established between biomechanical and acoustic 3D models for the numerical simulation of vowel-vowel utterances. The former rely on the activation and contraction of relevant muscles for voice production, which displace and distort speech organs. However, biomechanical models do not provide a closed computational domain of the 3D vocal tract airway where to simulate sound wave propagation. An algorithm is thus proposed to extract the vocal tract boundary from the surrounding anatomical structures at each time step of the transition between vowels. The resulting 3D geometries are fed into a 3D finite element acoustic model that solves the mixed wave equation for the acoustic pressure and particle velocity. An arbitrary Lagrangian-Eulerian framework is considered to account for the evolving vocal tract. Examples include six static vowels and three dynamic vowel-vowel utterances. Plausible muscle activation patterns are first determined for the static vowel sounds following an inverse method. Dynamic utterances are then generated by linearly interpolating the muscle activation of the static vowels. Results exhibit nonlinear trajectory of the vocal tract geometry, similar to that observed in electromagnetic midsagittal articulography. Clear differences are appreciated when comparing the generated sound with that obtained from direct linear interpolation of the vocal tract geometry. That is, interpolation between the starting and ending vocal tract geometries of an utterance, without resorting to any biomechanical model.

Articulatory-to-acoustic conversion using BLSTM-RNNs with augmented input representation

This paper presents a method to convert articulatory movements into speech waveforms using a data-driven approach. In this method, recorded electromagnetic midsagittal articulography (EMA) measurements are converted into both spectral features (i.e., Mel-cepstra) and excitation features (i.e., power, voiced/unvoiced flag, and F0) from which speech waveforms are then reconstructed. By considering the nonlinear and dynamic dependency relationships between articulatory movements and acoustic signals, this study adopts bidirectional long short-term memory (BLSTM) based recurrent neural networks (RNN) for the articulatory-to-acoustic conversion. Due to the limitations of current data acquisition technology, recorded articulatory movements are inevitably insufficient to completely describe the articulatory configuration during pronunciation. Therefore, this paper proposes to further augment the model input by concatenating EMA vectors with two other representations. First, the posterior probabilities derived from a phoneme classifier are concatenated with EMA features to provide a linguistic description of each frame for acoustic feature prediction. The classifier is trained to determine the phoneme label of each frame based on the observed EMA features. Second, a cascaded prediction strategy is designed to utilize the predicted spectral features as auxiliary input to boost the prediction accuracy of the excitation features. The results of experiment show that BLSTM-RNNs can achieve a better objective and subjective performance than deep neural networks (DNN) and Gaussian mixture models (GMM) in articulatory-to-acoustic conversion. In addition, our results show that the proposed methods for integrating linguistic representation and utilizing a cascaded prediction strategy can further improve the accuracy of acoustic feature prediction.

Seeing is treating: 3D electromagnetic midsagittal articulography (EMA) visual biofeedback for the remediation of residual speech errors

Production distortions or errors on the sounds /s/ and /r/ are among the most resistant to remediation with traditional speech therapies, even after years of weekly treatment sessions (e.g., Gibbon et al., 1996; McAuliffe & Cornwell, 2008; McLeod, Roberts, & Sita, 2006). In this study, we report on the results of treating residual speech errors in older children and adults with a new visual biofeedback treatment called Opti-Speech. Opti-Speech uses streaming positional data from the Wave EMA device to animate real-time motion of a tongue avatar on a screen. Both the clinician and the client can visualize movements as they occur relative to target shapes, set by the clinician, intended to guide the client to produce distortion-free and accurate speech sounds. Analyses of positional data and associated kinematics were completed during baseline, treatment, and follow-up phases for four participants, two who produced pre-treatment residual errors on /s/, and two with residual errors on /r/. Measures included absolute position of the 5 tongue sensors, variability of position, perceptual quality ratings, and acoustic measures of the consonants. Results indicate that Opti-speech effectively remediated the residual speech errors with corresponding evidence of generalization to untreated contexts and maintenance of improvements at follow-up.

Coupling Dynamics Interlip Coordination in Lower Lip Load Compensation

PURPOSE To study the effects of lower lip loading on lower and upper lip movements and their coordination to test predictions on coupling dynamics derived from studies in limb control. METHOD Movement data were acquired using electromagnetic midsagittal articulography under 4 conditions: (a) without restrictions, serving as a baseline; (b) with a small carrier device attached to the lower lip; (c) with a 50-g weight added to the device; and, at the end of the session (d) with the weight and device removed. For all conditions, 8 participants repeated nonwords at 2 speaking rates. Movement data were used to derive discrete kinematic measures, a cyclic index of spatiotemporal variability, phase deviations, and standard deviations of relative phase for interlip coupling. RESULTS Kinematic variables were not systematically affected by lower lip load. Phase deviations also showed no change, but in contrast, phase variability showed a significant increase for the lower lip load condition at fast rates. CONCLUSION Lower lip load effects are comparable to the reported impact of homologous limb loading, showing evidence for a tight coupling between both lips in line with predictions from coordination dynamics accounts in the literature.

Rate-Related Kinematic Changes in Younger and Older Adults

Aims: This study aimed to investigate the effects of speech rate changes on kinematic characteristics and stability of speech movements in younger and older speakers using electromagnetic midsagittal articulography. Patients and Methods: Eight young adults and 8 older adults engaged in a series of syllable repetition tasks of /pa/, /sa/ and /ta/ obtained at self-paced slow, habitual and fast speech rates, as well as in a series of metronome-guided speech rates, ranging from 2 to 4 syllables per second. The kinematic parameters duration, amplitude and peak velocity were obtained for opening and closing movements. Results: Older speakers were able to increase speech rate to the same degree or higher compared to younger speakers in both pacing conditions. Kinematic data show that older adults increased duration and decreased peak velocity in closing movements of alveolar constrictions at slower rates more prominently than younger adults. The results on movement stability revealed no differences between age groups. Conclusions: The results suggest that an age-related difference in speed-accuracy trade-off can be ruled out. Differences in kinematic characteristics point towards the possibility that older adults aimed to facilitate a closed-loop control system to maintain movement stability at slower speech rates.

Stability and composition of functional synergies for speech movements in children with developmental speech disorders

The aim of this study was to investigate the consistency and composition of functional synergies for speech movements in children with developmental speech disorders. Kinematic data were collected on the reiterated productions of syllables spa (/spaː/) and paas (/paːs/) by 10 6- to 9-year-olds with developmental speech disorders (five with speech sound disorder [SSD] and five with subtype childhood apraxia of speech [CAS]) and six normally speaking children using electro-magnetic midsagittal articulography (EMMA). Results showed a higher variability of tongue tip movement trajectories and a larger contribution of the lower lip relative to the jaw in oral closures for the five children with CAS compared to normally developing controls, indicating that functional synergies for speech movements in children with CAS may be both delayed and less stable. Furthermore, the SSD group showed a composition of tongue tip movements that is different from both CAS and controls. These results suggest that the differences in speech motor characteristics between SSD and subtype CAS are qualitative rather than quantitative. At the same time, the results suggest that both SSD and subtype CAS increase movement amplitude as an adaptive strategy to increase articulatory stability. Although in direct comparison no exclusive characteristics were found to differentiate subtype CAS from the group of children with SSD and from normally developing children, these preliminary results are promising for quantifying the role of speech motor processes in childhood speech sound disorders. Learning outcomes: The reader will be able to: (1) describe the development of speech motor control and explain the role of functional synergies/coordinative structures; (2) explain the measurement of the stability and composition of speech movements; (3) identify the difficulties in studying disordered speech motor development; (4) describe the differences in speech motor characteristics between SSD and subtype CAS; (5) describe the potential role of motor control strategies in developmental speech disorders.

Effects of Online Augmented Kinematic and Perceptual Feedback on Treatment of Speech Movements in Apraxia of Speech

Apraxia of speech (AOS) is a motor speech disorder characterized by disturbed spatial and temporal parameters of movement. Research on motor learning suggests that augmented feedback may provide a beneficial effect for training movement. This study examined the effects of the presence and frequency of online augmented visual kinematic feedback (AVKF) and clinician-provided perceptual feedback on speech accuracy in 2 adults with acquired AOS. Within a single-subject multiple-baseline design, AVKF was provided using electromagnetic midsagittal articulography (EMA) in 2 feedback conditions (50 or 100%). Articulator placement was specified for speech motor targets (SMTs). Treated and baselined SMTs were in the initial or final position of single-syllable words, in varying consonant-vowel or vowel-consonant contexts. SMTs were selected based on each participant’s pre-assessed erred productions. Productions were digitally recorded and online perceptual judgments of accuracy (including segment and intersegment distortions) were made. Inter- and intra-judge reliability for perceptual accuracy was high. Results measured by visual inspection and effect size revealed positive acquisition and generalization effects for both participants. Generalization occurred across vowel contexts and to untreated probes. Results of the frequency manipulation were confounded by presentation order. Maintenance of learned and generalized effects were demonstrated for 1 participant. These data provide support for the role of augmented feedback in treating speech movements that result in perceptually accurate speech production. Future investigations will explore the independent contributions of each feedback type (i.e. kinematic and perceptual) in producing efficient and effective training of SMTs in persons with AOS.

Acoustic and articulatory manifestations of vowel reduction in German

Recent phonological approaches incorporate phonetic principles in the motivation of phonological regularities, e.g. vowel reduction and neutralization in unstressed position by target undershoot. So far, evidence for this hypothesis is based on impressionistic and acoustic data but not on articulatory data. The major goal of this study is to compare formant spaces and lingual positions during the production of German vowels for combined effects of stress, accent and corrective contrast. In order to identify strategies for vowel reduction independent of speaker-specific vocal-tract anatomies and individual biomechanical properties, an approach similar to the Generalized Procrustes Analysis was applied to formant spaces and lingual vowel target positions. The data basis consists of the German stressed and unstressed full vowels /iù ɪ yù ʏ eù ɛ ɛùφùœaù a où ɔ uù ʊ/ from seven speakers recorded by means of electromagnetic midsagittal articulography (EMMA). Speaker normalized articulatory and formant spaces gave evidence for a greater degree of coarticulation with the consonant context for unstressed vowels as compared to stressed vowels. However, only for tense vowels could spatial reduction patterns be attributed to vowel shortening, whereas lax vowels were reduced without shortening. The results are discussed in the light of current theories of vowel reduction, i.e. target undershoot, Adaptive Dispersion Theory and Prominence Alignment.

Articulatory Variability in Cluttering

In order to investigate the articulatory processes of the hasty and mumbled speech in cluttering, the kinematic variability was analysed by means of electromagnetic midsagittal articulography. In contrast to persons with stuttering, those with cluttering improve their intelligibility by concentrating on their speech task. Variability has always been an important criterion in comparable studies of stuttering and is discussed in terms of the stability of the speech motor system. The aim of the current study was to analyse the spatial and temporal variability in the speech of three persons with cluttering (PWC) and three control speakers. All participants were native speakers of German. The speech material consisted of repetitive CV syllables and loan words such as ‘emotionalisieren’, because PWC have the severest problems with long words with a complex syllable structure. The results showed a significantly higher coefficient of variation for PWC in loan word production, both in the temporal and in the spatial domain, whereas the means of displacements and durations did not differ between groups. These findings were discussed in terms of the effects of the linguistic complexity, since for the syllable repetition task, no significant differences between PWC and controls were found.

The Dynamics of Lingual-Mandibular Coordination During Liquid Swallowing

Previous literature on tongue-jaw relationships during swallowing has focused on behaviors observed with chewable solid foods. The present investigation was undertaken to evaluate both the nature and stability of coordinative relationships between the jaw and three points located along the midsagittal groove of the tongue--anterior (blade), middle (body), and posterior (dorsum)--during swallowing of thin and honey-thick liquids. A reiterative swallowing paradigm was used, with two task conditions (discrete and sequential), to explore the stability of tongue-jaw coordination across different frequencies of swallowing. Eight healthy participants in two age groups (young, older) performed sets of repeated swallows. Tongue and jaw movements were measured using electromagnetic midsagittal articulography. The data were analyzed in terms of variability in the spatiotemporal movement pattern for each fleshpoint of interest, and the temporal coupling (frequency entrainment) and relative phasing of movement for each tongue segment compared to the mandible. The results illustrate a stereotypical but not invariant sequence of movement phasing in the tongue-jaw complex during liquid swallowing and task-related reductions in variability at higher frequencies of swallowing in tongue dorsum movements. This evidence supports the idea that different segments of the tongue couple with the jaw as a synergy for swallowing, but can modify their coupling relationship to accommodate task demands.

Interarticulator cohesion within coronal consonant production

If more than one articulator is involved in the execution of a phonetic task, then the individual articulators have to be temporally coordinated with each other in a lawful manner. The present study aims at analyzing tongue-jaw cohesion in the temporal domain for the German coronal consonants [s, f, t, d, n, l], i.e., consonants produced with the same set of articulators--the tongue blade and the jaw--but differing in manner of articulation. The stability of obtained interaction patterns is evaluated by varying the degree of vocal effort: comfortable and loud. Tongue and jaw movements of five speakers of German were recorded by means of electromagnetic midsagittal articulography (EMMA) during [aCa] sequences. The results indicate that (1) tongue-jaw coordination varies with manner of articulation, i.e., a later onset and offset of the jaw target for the stops compared to the fricatives, the nasal and the lateral; (2) the obtained patterns are stable across vocal effort conditions; (3) the sibilants are produced with smaller standard deviations for latencies and target positions; and (4) adjustments to the lower jaw positions during the surrounding vowels in loud speech occur during the closing and opening movement intervals and not the consonantal target phases.

Does Barium Influence Tongue Behaviors During Swallowing?

The validity of videofluoroscopic swallowing assessments rests on the understanding that thin, nectar-, honey-, and spoon-thick radiopaque liquids resemble nonopaque liquids, both in their consistency and in the variations in swallowing that they elicit. Tongue movements during sequential swallows of opaque and nonopaque liquids were studied in 8 healthy participants in 2 age groups (<30 years, >50 years) using electromagnetic midsagittal articulography. Differences included smaller sip size, longer oropharyngeal transit times, and greater variability in tongue movement patterns with opaque liquids compared to nonopaque liquids, but effect sizes for these differences were small. Transit times were significantly longer for older participants than younger participants. We recommend matching radiopaque bolus size in videofluoroscopy to the patient's habitually preferred sip mass for comparator nonopaque liquids.

Articulatory variability of clutterers

In order to investigate the articulatory processes of the hasty and mumbled speech of clutterers, the kinematic variability was analysed by means of electromagnetic midsagittal articulography (EMMA). In contrast to stutterers, clutterers improve their intelligibility by concentrating on their speech task. Variability is an important criterion in comparable studies of stuttering and is discussed in terms of the stability of the speech motor system. The aim of the current study was to analyse the spatial and temporal variability in the speech of three clutterers and three control speakers. All speakers were native speakers of German. The speech material consisted of repetitive CV-syllables and foreign words, because clutterers have the most severe problems with long words which have a complex syllable structure. The results showed a higher quotient of variation for clutterers in the foreign word production. For the syllable repetition task, no significant differences between clutterers and controls were found. The extremely large and variable displacements were interpreted as a strategy that helps clutterers to improve the intelligibility of their speech.&#x0D;

Influence of bolus consistency on lingual behaviors in sequential swallowing.

Thickened liquids are a commonly recommended intervention for dysphagia. Previous research has documented differences in temporal aspects of bolus transit for paste versus liquid consistencies; however, the influence of liquid viscosity on tongue movements during swallowing remains unstudied. We report an analysis of the influence of bolus consistency on lingual kinematics during swallowing. Electromagnetic midsagittal articulography was used to trace tongue body and dorsum movement during sequential swallows of three bolus consistencies: thin, nectar-thick, and honey-thick liquids. Rheological profiling was conducted to characterize viscosity and density differences among six liquids (two of each consistency). Eight healthy volunteers participated; four were in a younger age cohort (under age 30) and four were over the age of 50. The primary difference observed across the liquids of interest was a previously unreported phenomenon of sip-mass modulation; both flavor and density appeared to influence sip-sizing behaviors. Additionally, significantly greater variability in lingual movement patterns was observed in the older subject group. Systematic variations in lingual kinematics related to bolus consistency were restricted to the variability of downward tongue dorsum movement. Otherwise, the present analysis failed to find empirical evidence of significant modulations in tongue behaviors across the thin to honey-thick consistency range.

Use of electromagnetic midsagittal articulography in the study of swallowing.

The tongue functions as the primary articulator during the oropharyngeal stages of swallowing. However, detailed descriptions of the kinematics and spatiotemporal variability of tongue behaviors during swallowing are limited to a handful of analyses of data from the X-ray microbeam database. In this article, a new technique, electromagnetic midsagittal articulography (EMMA), is introduced for the high-resolution description of oral articulatory movements during swallowing. Data from 8 healthy, nondysphagic participants are used to illustrate the methods used for data collection and analysis. Movement data were collected for 3 fleshpoint positions on the tongue (blade, body, dorsum) during sequences of repeated discrete water swallows, and were characterized for variables of spatiotemporal variability and 4 discrete kinematic parameters (movement amplitude, peak velocity, duration, and kinematic stiffness). These data show that the movement trajectories measured using EMMA are consistent with descriptions from previous X-ray microbeam studies, indicating that EMMA is a feasible method for the detailed study of tongue movements during swallowing.

Effects of Morpheme Boundaries on Intergestural Timing: Evidence from Korean

This paper examines the effects of morpheme boundaries on intergestural timing, and demonstrates that low-level phonetic realization is influenced by morphological structure, i.e. compounding and affixation. It reports two experiments, one using electromagnetic midsagittal articulography (EMA) and one electropalatography (EPG), examining Korean data. The results of the EMA study show that intergestural timing is less variable for adjacent gestures across the word boundary inside a lexicalized compound than inside a nonlexicalized compound, and inside a monomorphemic word than across a morpheme boundary. The EPG study (which examined the timing in palatalization of a coronal) shows that both [ti] and [ni] have more variability in gestural timing when heteromorphemic than when tautomorphemic. Furthermore, the phonetic details of gestural overlap shed light on the asymmetry on palatalization between tautomorphemic and heteromorphemic gestural sequences (e.g. ni vs. n-i), presumably driven by paradigmatic contrast and preference of overlap. In short, what emerges from two experiments is that gestures are coordinated more stably within a single lexical item (a morpheme or a lexicalized compound) than across a boundary between lexical items. In accounting for the stability of intergestural timing within a lexical entry, several hypotheses were discussed including the Phase Window, Bonding Strength, Phonological Timing and Extended Phase Window model newly proposed here. The implication is that the morphological structure may be encoded in the phonetic realization, as is the case with other linguistic structure (e.g. prosodic structure).

Traditional temporal measures are unstable and thus are inappropriate parameters to compare speech kinematics of control and stuttering populations

The aims of the study are to: (1) explore further the short- and long-term flexibility in normal speech kinematics, and (2) identify relatively stable temporal characteristics that could function as appropriate experimental and control group comparative parameters. Movements of the tongue, lips, and jaw were transduced by electromagnetic midsagittal articulography. A single session included 20 perceptually fluent repetitions of /pap/, /tat/, and /sas/ imbedded in a carrier phrase at normal, slow, and fast speech rates. Seven stutterers and matched controls completed three sessions at 2-week intervals. Reported here are across session comparisons of temporal ordering and relative timing for bilabial closure at normal rates. The prevalence and stability of the sequence pattern is dependent upon interarticulator relative time. Four control subjects and two stutterers showed idiosyncratic but stable temporal organizational characteristics within and across sessions. However, three of the controls and five of the stutterers were inconsistent in at least one of the two temporal parameters across sessions. Thus neither temporal ordering nor interarticulator relative time is an appropriate group comparative parameter because of natural long-term flexibility. Natural flexibility is compatible with the notion of coordinative structures within a task-dynamic point of view.

Across session variability in lip–jaw synergies for bilabial closure.

Movements of the tongue, lips, and jaw were transduced by electromagnetic midsagittal articulography. A single session included 20 repetitions of

Across session temporal stability of the lip–jaw complex in bilabial closure

Movements of the tongue, lips, and jaw were transduced by electromagnetic midsagittal articulography. A single session included 20 repetitions of /pap/, /tat/, and /sas/ imbedded in a carrier phrase at normal, slow, and fast speech rates. Seven talkers completed three sessions at 1 week intervals. Reported here are across session comparisons of temporal ordering and relative timing of the movement of the lips and jaw for labial closure at normal rates. The predominant sequence patterns are those in which either upper or lower lip movement occurs first and jaw movement occurs last. Across-subject comparisons show that either lip lead sequence is equally likely to occur. Within-subject across-session comparisons show no clear preference for either lip lead sequence for some subjects, and reversals in dominate lip lead sequences for other subjects. Across-session stability of temporal order is related to interarticulator relative time: Subjects who demonstrate tight coupling of the upper and lower lips have a higher probability of producing both lip-lead sequences, whereas subjects who demonstrate longer interarticulator relative timing produce consistent sequence patterns across sessions. Temporal measures representing the central tendencies of the complete set of session closure gestures are most stable across sessions. [Work supported by NIH DC-00121 to Haskins Laboratories and University of Illinois research grant.]

Across session spatial stability of the tongue-jaw complex in alveolar closure

The movements of the tongue, lips, and jaw were transduced by electromagnetic midsagittal articulography. A single session included 20 repetitions of /pap/, /tat/, and /sas/ imbedded in a carrier phrase at normal, slow, and fast speech rates. Seven talkers completed three sessions at 1-week intervals. Reported here are measures of across session stability of tongue and jaw spatial synergies for alveolar closure at normal rates. Preliminary analyses indicate that idiosyncratic organization of the tongue and jaw for alveolar closure is relatively stable within and across sessions. However, across session stability, in terms of the relative contribution of the tongue and jaw towards closure and motor equivalence covariability of the summed tongue-jaw, the jaw, and the derived independent tongue displacements, is sensitive to variations in normal speech rate. For example, tongue-jaw synergies varied when the difference between session average duration of the tongue and/or jaw gestures approached 30%. [Work supported by NIH DC-00121 to Haskins Laboratories and University of Illinois research grant.]