Articulatory Actions Research Articles

Broad and fine acoustic categories in bod, bond, bald, and bard: A step toward acoustic phonology

Acoustics is central to the study of speech communication, but it is conspicuously under-represented in abstract representations of speech. Many flavors of phonological analysis tend toward articulatory descriptions, and transcriptions focus on strings of articulatory actions. All this is despite acoustics being easier to measure than articulation with current technology. The present study explores basic concepts for an acoustic phonology, with two types of postulated categories: broad and fine. Resonant, turbulent, transient, and occludent types of sounds comprise the broad categories, as general methods of filtering the speech source. Fine categories are conceptualized as specific types of acoustic actions within a broad category. These acoustic actions are goal-oriented, as for achieving a particular acoustic effect like the presence of antiformants or a lowered F2 or F3. However, these actions are not explicitly restricted to manipulating traditional phonetic features like formants. By default, fine categories are assumed to be produced in parallel when possible, yielding overlap effects like anticipatory nasalization, lateralization, and rhotacization. These concepts are explored in a microanalysis of bod, bond, bald, and bard from the speaker in the Massive Auditory Lexical Decision data set, with an eye to seeding the ground for a future acoustic phonology.

A study of the resonance peak characteristics of Mongolian long-key ode

The lyrics of the ode are mainly praising ancestors, princes, and nobles, singing about parents, hometowns, and horses, with a solemn style. In 2005, the Mongolian long tune was selected as a world-class intangible cultural heritage, and the study of Mongolian long tunes has become a worldwide issue. At present, domestic, and foreign research on long-key odes mainly focuses on music theory, artistic expression, historical development, cultural connotation, performance methods, etc. The research contents and fields are relatively rich. However, little research has been done on the generation principle and articulation mechanism of the Mongolian long-key ode due to the limitation of research techniques and means. By extracting some physiological acoustic parameters of articulation, establishing an orally transmitted cultural parameter model, and analyzing the relationship between parameters and articulatory actions, this modern phonological method based on computer and language science to study the fundamental mechanism of long-tune vocalization can deepen the theory of speech vocalization, explore the mechanism of speech vocalization. The study has found a new application for modern phonology accumulated over the years, which can help to preserve culture digitally and promote the presentation of cultural multimodality.

Predicting One's Turn With Both Body and Mind: Anticipatory Speech Postures During Dyadic Conversation.

In natural conversation, turns are handed off quickly, with the mean downtime commonly ranging from 7 to 423 ms. To achieve this, speakers plan their upcoming speech as their partner’s turn unfolds, holding the audible utterance in abeyance until socially appropriate. The role played by prediction is debated, with some researchers claiming that speakers predict upcoming speech opportunities, and others claiming that speakers wait for detection of turn-final cues. The dynamics of articulatory triggering may speak to this debate. It is often assumed that the prepared utterance is held in a response buffer and then initiated all at once. This assumption is consistent with standard phonetic models in which articulatory actions must follow tightly prescribed patterns of coordination. This assumption has recently been challenged by single-word production experiments in which participants partly positioned their articulators to anticipate upcoming utterances, long before starting the acoustic response. The present study considered whether similar anticipatory postures arise when speakers in conversation await their next opportunity to speak. We analyzed a pre-existing audiovisual database of dyads engaging in unstructured conversation. Video motion tracking was used to determine speakers’ lip areas over time. When utterance-initial syllables began with labial consonants or included rounded vowels, speakers produced distinctly smaller lip areas (compared to other utterances), prior to audible speech. This effect was moderated by the number of words in the upcoming utterance; postures arose up to 3,000 ms before acoustic onset for short utterances of 1–3 words. We discuss the implications for models of conversation and phonetic control.

EEG-based classification of bilingual unspoken speech using ANN.

The ability to interpret unspoken or imagined speech through electroencephalography (EEG) is of therapeutic interest for people suffering from speech disorders and `lockedin' syndrome. It is also useful for brain-computer interface (BCI) techniques not involving articulatory actions. Previous work has involved using particular words in one chosen language and training classifiers to distinguish between them. Such studies have reported accuracies of 40-60% and are not ideal for practical implementation. Furthermore, in today's multilingual society, classifiers trained in one language alone might not always have the desired effect. To address this, we present a novel approach to improve accuracy of the current model by combining bilingual interpretation and decision making. We collect data from 5 subjects with Hindi and English as primary and secondary languages respectively and ask them 20 `Yes'/`No' questions (`Haan'/`Na' in Hindi) in each language. We choose sensors present in regions important to both language processing and decision making. Data is preprocessed, and Principal Component Analysis (PCA) is carried out to reduce dimensionality. This is input to Support Vector Machine (SVM), Random Forest (RF), AdaBoost (AB), and Artificial Neural Networks (ANN) classifiers for prediction. Experimental results reveal best accuracy of 85.20% and 92.18% for decision and language classification respectively using ANN. Overall accuracy of bilingual speech classification is 75.38%.

Nasality patterns in word productions of children with cochlear implants: Evidence from Greek

Typical speech production development requires adequate auditory input. Children who are born deaf or become deaf in early childhood display atypical speech patterns, especially for articulatory actions that cannot be seen. Such actions include tongue backing, laryngeal actions, and velar function. Several previous authors have discussed how inadequate auditory input affects anterior-posterior tongue positioning and laryngeal parameters, compromising phonetic contrasts (e.g., vowel and voicing distinctions), but few studies have directly assessed velar control in hearing-impaired children. This work presents nasometer data for Greek-learning children with cochlear implants [CI] and age- and sex-matched normal hearing [NH] controls, ages 5-16 years. Participants produced single words, elicited in response to visual and auditory prompts. Bisyllabic word types of varying stress pattern were contrasted in nasality and position of the target consonant. Based on the two nasometer microphone signals (oral and nasal), word and segment boundaries were defined, and nasometer values over these intervals were extracted. We will present whole-word data, comparing nasalance values between NH and CI children, and examples showing atypical oral-nasal timing in the CI population. This preliminary analysis shows some of the ways in which hearing-impaired children with cochlear implants may show unusual patterns of velar control.

A Near-Infrared Spectroscopy Study on Cortical Hemodynamic Responses to Normal and Whispered Speech in 3- to 7-Year-Old Children.

The purpose of this study was to assess cortical hemodynamic response patterns in 3- to 7-year-old children listening to two speech modes: normally vocalized and whispered speech. Understanding whispered speech requires processing of the relatively weak, noisy signal, as well as the cognitive ability to understand the speaker's reason for whispering. Near-infrared spectroscopy (NIRS) was used to assess changes in cortical oxygenated hemoglobin from 16 typically developing children. A profound difference in oxygenated hemoglobin levels between the speech modes was found over left ventral sensorimotor cortex. In particular, over areas that represent speech articulatory body parts and motion, such as the larynx, lips, and jaw, oxygenated hemoglobin was higher for whisper than for normal speech. The weaker stimulus, in terms of sound energy, thus induced the more profound hemodynamic response. This, moreover, occurred over areas involved in speech articulation, even though the children did not overtly articulate speech during measurements. Because whisper is a special form of communication not often used in daily life, we suggest that the hemodynamic response difference over left ventral sensorimotor cortex resulted from inner (covert) practice or imagination of the different articulatory actions necessary to produce whisper as opposed to normal speech.

Physics-Based Analysis of Articulatory Player Actions in Single-Reed Woodwind Instruments

Musicians use various articulation techniques during expressive performance. In the case of single-reed woodwind instruments these may involve tongue strokes to the reed or modulation of the blowing pressure. To analyse the emerging transient phenomena, a time-domain physical model of the player-instrument interaction is employed. In order to avoid adding new terms and complexity to the model, the effect of tonguing is simulated by allowing temporal variation of existing, physically meaningful parameters. Experimental measurements show differences between tones separated using the tongue or the blowing pressure and comparisons with numerical simulations are carried out. In particular the synthesised mouthpiece pressure and reed displacement are compared with data obtained under real playing conditions. In an attempt to understand how articulatory actions affect the excitation mechanism of the instrument, the physical model parameters that are required to resynthesise the recorded sounds are estimated. This allows tracking the evolution of parameters that evade direct measurement.

ERP adaptation provides direct evidence for early mirror neuron activation in the inferior parietal lobule

Mirror neuron systems are frequently investigated by assessing overlapping brain activity during observation and execution of actions; however, distinct neuronal subpopulations may be activated that fall below the spatial resolution of magnetic resonance techniques. This shortfall can be resolved using repetition suppression paradigms that identify physiological adaptation processes caused by repeated activation of identical neuronal circuits. Here, event-related potentials were used to investigate the time course of mirror neuron circuit activation using repetition suppression within and across action observation and action execution modalities. In a lip-reading and speech production paradigm, the N170 component indexed stimulus repetition by adapting to both cross-modal and intra-modal repetitions in the left hemisphere. Neuronal source localization revealed activation of the left inferior parietal lobule during cross-modal relative to intra-modal trials. These results provide support for the position that the same neuronal circuits are activated in perceiving and performing articulatory actions. Moreover, our data strongly suggest that inferior parietal lobule mirror neurons are activated relatively early in time, which indicates partly automatic processes of linguistic perception and mirroring. Repetition suppression paradigms therefore help to elucidate neuronal correlates of different cognitive processes and may serve as a starting point for advanced electrophysiological research on mirror neurons.

Moving from hand to mouth: echo phonology and the origins of language.

Although the sign languages in use today are full human languages, certain of the features they share with gestures have been suggested to provide information about possible origins of human language. These features include sharing common articulators with gestures, and exhibiting substantial iconicity in comparison to spoken languages. If human proto-language was gestural, the question remains of how a highly iconic manual communication system might have been transformed into a primarily vocal communication system in which the links between symbol and referent are for the most part arbitrary. The hypothesis presented here focuses on a class of signs which exhibit: “echo phonology,” a repertoire of mouth actions which are characterized by “echoing” on the mouth certain of the articulatory actions of the hands. The basic features of echo phonology are introduced, and discussed in relation to various types of data. Echo phonology provides naturalistic examples of a possible mechanism accounting for part of the evolution of language, with evidence both of the transfer of manual actions to oral ones and the conversion of units of an iconic manual communication system into a largely arbitrary vocal communication system.

The cognitive basis of spontaneous imitation: Evidence from the visual world

It is well-established that, when people are asked to identify and quickly repeat spoken words, they show a strong tendency to spontaneously imitate the vocal and/or phonetic characteristics of the stimulus tokens. There is mixed evidence, however, regarding the underlying basis of such imitation: Does it only reflect gestural attunement (as in Direct Realism), or does it also reflect cognitive principles of word perception and memory? The gestural attunement view has face validity, as imitation seems to require tacit knowledge of other peoples' articulatory actions. The role of memory is less obvious, although people can certainly imitate others from memory. In this talk, I will present evidence from three new experiments, pairing procedures from the world paradigm with a speech production task. Across studies, there is clear evidence that degrees of speech imitation are tightly connected to attention and memory processes that were engaged during initial exposure to spoken words. The results show clear imitation (in naming depicted objects), both with and without spoken words prompting responses, and show strong effects of competition among visual objects: Imitation increases when other potential objects have similar names, or even similar appearances. Spontaneous imitation is both a gestural and a cognitive behavior.

Auditory-Motor Mapping Training as an Intervention to Facilitate Speech Output in Non-Verbal Children with Autism: A Proof of Concept Study

Although up to 25% of children with autism are non-verbal, there are very few interventions that can reliably produce significant improvements in speech output. Recently, a novel intervention called Auditory-Motor Mapping Training (AMMT) has been developed, which aims to promote speech production directly by training the association between sounds and articulatory actions using intonation and bimanual motor activities. AMMT capitalizes on the inherent musical strengths of children with autism, and offers activities that they intrinsically enjoy. It also engages and potentially stimulates a network of brain regions that may be dysfunctional in autism. Here, we report an initial efficacy study to provide ‘proof of concept’ for AMMT. Six non-verbal children with autism participated. Prior to treatment, the children had no intelligible words. They each received 40 individual sessions of AMMT 5 times per week, over an 8-week period. Probe assessments were conducted periodically during baseline, therapy, and follow-up sessions. After therapy, all children showed significant improvements in their ability to articulate words and phrases, with generalization to items that were not practiced during therapy sessions. Because these children had no or minimal vocal output prior to treatment, the acquisition of speech sounds and word approximations through AMMT represents a critical step in expressive language development in children with autism.

Non-Invasive Brain Stimulation Enhances the Effects of Melodic Intonation Therapy

Research has suggested that a fronto-temporal network in the right hemisphere may be responsible for mediating melodic intonation therapy’s (MIT) positive effects on speech recovery. We investigated the potential for a non-invasive brain stimulation technique, transcranial direct current stimulation (tDCS), to augment the benefits of MIT in patients with non-fluent aphasia by modulating neural activity in the brain during treatment with MIT. The polarity of the current applied to the scalp determines the effects of tDCS on the underlying tissue: anodal-tDCS increases excitability, whereas cathodal tDCS decreases excitability. We applied anodal-tDCS to the posterior inferior frontal gyrus of the right hemisphere, an area that has been shown both to contribute to singing through the mapping of sounds to articulatory actions and to serve as a key region in the process of recovery from aphasia, particularly in patients with large left hemisphere lesions. The stimulation was applied while patients were treated with MIT by a trained therapist. Six patients with moderate to severe non-fluent aphasia underwent three consecutive days of anodal-tDCS + MIT, and an equivalent series of sham-tDCS + MIT. The two treatment series were separated by 1 week, and the order in which the treatments were administered was randomized. Compared to the effects of sham-tDCS + MIT, anodal-tDCS + MIT led to significant improvements in fluency of speech. These results support the hypothesis that, as the brain seeks to reorganize and compensate for damage to left hemisphere language centers, combining anodal-tDCS with MIT may further recovery from post-stroke aphasia by enhancing activity in a right hemisphere sensorimotor network for articulation.

Measures of intraoral pressure pulse shape during stop consonants

To maintain voicing during a stop consonant, adult speakers typically perform articulatory actions that increase supraglottal volumes and slow the rate at which of intraoral pressure (Pio) rises. In an influential study, Müller and Brown [in Speech and Language: Advances in Basic Research and Practice 4, edited by N. Lass (Academic, New York, 1980), pp. 318–389] proposed a quantitative measure of Pio pulse shape, based on the difference between (a) the slope from pressure at closure to average pressure and (b) from average pressure to pressure at release. We recently applied these techniques to data from women and child speakers of English and observed some shortcomings in Müller and Brown’s measurement technique, such that the slope difference angle did not clearly reflect the actual pressure pulse shape. The purpose of this paper is to explore new methods of analyzing intraoral pressure contours, based on fitting a mathematical model to the pressure pulse shapes. It will report our progress and compare our results with the previous technique.

Articulatory Actions within a Phonological System and the Resulting Complexity of Speech Signals

Abstract The importance of overlapping coordinated gestures in achieving the auditory goals of speech is considered. Articulatory synthesis can help our understanding of the coordination problems and the variety of solutions found by real speakers. The time required to perform specific essential gestures, such as abduction-adduction of the vocal folds for fricatives, imposes strong timing constraints. In different speech sequences different articulators act as the duration-determining factor (DDF). Timing variability needs to be modelled also. Further experimentation is needed to determine whether increased speaking rate more nearly retains constant phase or constant absolute durational separation of articulatory events and whether or not some gestures can disappear under the relaxed auditory criteria of a casual style of speech. Two examples are presented, from the Leeds articulatory modelling, of the ways in which the phonological system of the language interacts with the processes and constraints of the speech production system to yield context-dependent variation for individual phonemes.

Use of articulatory signals in automatic speech recognition

Automatic speech recognition systems generally attempt to determine the spoken message from analysis of the acoustic speech waveform. In this research we evaluated the performance of the IBM Speech Recognition System [F. Jelinek, Proc. IEEE 73, 1616–1624 (1986)] when the input included measurements of selected articulatory actions occurring during speech production. The system achieved significant recognition rates (for isolated words in sentences) when the acoustical signal was disabled and the input was restricted to articulatory signal similar to those sensed by users of the Tadoma method of tactile‐speech recognition [e.g., Reed et al., J. Acoust. Soc. Am. 77, 247–257 (1985)]. In other tests the availability of articulatory inputs improved recognition performance when the acoustical signal was sufficiently degraded by additive white noise. Improvements were observed independent of whether the recognition system made use of the likelihood that words would appear in the vicinity of the other words in a given sentence.

Speech production simulated with a functional model of the larynx and the vocal tract

A composite model of speech production has been used to try to match the articulatory, aerodynamic and acoustic patterns for some real speakers of British English. The aim is to characterize particular speakers or speaker types acoustically through a match to their individual articulatory actions. In the model, the larynx contributes two independent articulatory actions; in addition it generates the acoustic source for voicing. The problems of direct matching of acoustic patterns are very great. The particular advantage of the analysis-by-synthesis methods described here is that the aerodynamic stage, which links actions to sounds, is included. As a result, the articulatory match can be made with some confidence before acoustic source and filter properties are varied; acoustic sources and acoustic pattern features of the synthetic speech co-vary in ways similar to those observed in natural speech. The model has been used to simulate disordered as well as normal larynx behaviour.