Models Of Speech Production Research Articles

Modulations of the auditory M100 in an imitation task

Models of speech production explain event-related suppression of the auditory cortical response as reflecting a comparison between auditory predictions and feedback. The present MEG study was designed to test two predictions from this framework: (1) whether the reduced auditory response varies as a function of the mismatch between prediction and feedback; (2) whether individual variation in this response is predictive of speech-motor adaptation.Participants alternated between online imitation and listening tasks. In the imitation task, participants began each trial producing the same vowel (/e/) and subsequently listened to and imitated auditorily-presented vowels varying in acoustic distance from /e/.Results replicated suppression, with a smaller M100 during speaking than listening. Although we did not find unequivocal support for the first prediction, participants with less M100 suppression were better at the imitation task. These results are consistent with the enhancement of M100 serving as an error signal to drive subsequent speech-motor adaptation.

Control strategy of physiological articulatory model for speech production

In speech production the articulatory apparatus includes the organs that execute efferent motor commands from the central nervous system. In order to simulate the behavior of the articulatory apparatus, computational modeling is a commonly-used method. We have constructed a full 3D physiological articulatory model that includes the tongue, jaw, hyoid bone and vocal-tract wall based on the continuum finite element method. This model comprises articulatory muscles with realistic properties and geometrical arrangements. The muscle activation patterns are used to control the movements of the model. In order to use the model to investigate the speech motor control mechanism, generate speech sounds, predict the effect of surgical operation, etc., we have to realize an automatic control strategy. The task of the control strategy is given a particular target, how to generate muscle activation patterns that can control the model to achieve the target. There are two main control strategies for the physiological articulatory model: feedforward control and feedback control. Feedforward control is a kind of mapping used to directly find muscle activation patterns according to the desired target, and feedback control is used to adjust muscle activation patterns to reduce the distance between the desired target and the realized position. In speech production, feedforward mapping is used to rapidly generate muscle activation patterns to control the articulators to produce fluent speech. Feedback control plays the role of learning and maintaining the feedforward mapping. When the degree of accuracy using feedforward mapping cannot satisfy the requirement, feedback control can be used to realize fine motor control. In this paper, we describe how to use feedback control as a learning loop to construct feedforward mapping. The constructed feedforward mapping was assessed through an open-set test, and reasonable articulatory positions were obtained by comparison with the desired targets. Furthermore, the ability of feedback control to improve control accuracy was proved by a large quantity of simulations. 提要: 在言語產生的過程中,發音器官是執行來自中樞神經系統的運動指令的 終端組織。電腦建模的方法常常被應用於模擬發音器官的行為。為此, 我們使用連續體有限元的方法建立了一個三維的發音器官的生理模型, 該模型包含舌、下頜、舌骨以及聲道壁等發音器官。該模型還包括了根 據其生理解剖屬性建立的用於控制發音器官運動的肌肉模型。為了將該 生理模型應用於探索發音器官的運動控制機制,產生自然流暢的語音, 對發音器官的手術後功能進行預測等,我們需要對模型建立一個自動的 控制機制。控制機制的任務在於給定一個發音目標,如何自動的產生肌 肉激活模式去控制模型到達目標。對於發音器官的生理模型而言,有兩 種主要的控制方式:前饋控制和回饋控制。前饋控制是一種從發音控制 目標到肌肉激活模式的映射,用於根據發音目標產生肌肉激活模式;而 回饋控制主要用於調整肌肉激活模式來減少模型實現的位置到目標之間 的距離,最終控制模型到達目標。在言語產生的過程中,前饋控制用於 快速的產生肌肉激活模式控制發音器官產生流暢的語音。回饋控制主要 用於學習前饋控制的映射,並且維持前饋控制可行性。當前饋控制的精 度不能滿足需求時,回饋控制可以用於提高控制精度以實現精確控制。 在本文中,我們將重點介紹如何使用回饋控制作為學習回路,建立前饋 控制的映射。對已建立的前饋映射的開集測試表明該映射可以用於控制 模型在誤差允許範圍內達到目標。而且,大量的模型模擬表明回饋控制 可用於改善前饋控制的控制精度。

Nonlinear interactive source-filter models for speech

The linear source-filter model of speech production assumes that the source of the speech sounds is independent of the filter. However, acoustic simulations based on the physical speech production models show that when the fundamental frequency of the source harmonics approaches the first formant of the vocal tract filter, the filter has significant effects on the source due to the nonlinear coupling between them. In this study, two interactive system models are proposed under the quasi steady Bernoulli flow and linear vocal tract assumptions. An algorithm is developed to estimate the model parameters. Glottal flow and the linear vocal tract parameters are found by conventional methods. Rosenberg model is used to synthesize the glottal waveform. A recursive optimization method is proposed to find the parameters of the interactive model. Finally, glottal flow produced by the nonlinear interactive system is computed. The experimental results show that the interactive system model produces fine details of glottal flow source accurately.

Timing in talking: what is it used for, and how is it controlled?

In the first part of the paper, we summarize the linguistic factors that shape speech timing patterns, including the prosodic structures which govern them, and suggest that speech timing patterns are used to aid utterance recognition. In the spirit of optimal control theory, we propose that recognition requirements are balanced against requirements such as rate of speech and style, as well as movement costs, to yield (near-)optimal planned surface timing patterns; additional factors may influence the implementation of that plan. In the second part of the paper, we discuss theories of timing control in models of speech production and motor control. We present three types of evidence that support models of speech production that involve extrinsic timing. These include (i) increasing variability with increases in interval duration, (ii) evidence that speakers refer to and plan surface durations, and (iii) independent timing of movement onsets and offsets.

Reduction in gesture during the production of repeated references

In dialogue, repeated references contain fewer words (which are also acoustically reduced) and fewer gestures than initial ones. In this paper, we describe three experiments studying to what extent gesture reduction is comparable to other forms of linguistic reduction. Since previous studies showed conflicting findings for gesture rate, we systematically compare two measures of gesture rate: gesture rate per word and per semantic attribute (Experiment I). In addition, we ask whether repetition impacts the form of gestures, by manual annotation of a number of features (Experiment I), by studying gradient differences using a judgment test (Experiment II), and by investigating how effective initial and repeated gestures are at communicating information (Experiment III). The results revealed no reduction in terms of gesture rate per word, but a U-shaped reduction pattern for gesture rate per attribute. Gesture annotation showed no reliable effects of repetition on gesture form, yet participants judged gestures from repeated references as less precise than those from initial ones. Despite this gradient reduction, gestures from initial and repeated references were equally successful in communicating information. Besides effects of repetition, we found systematic effects of visibility on gesture production, with more, longer, larger and more communicative gestures when participants could see each other. We discuss the implications of our findings for gesture research and for models of speech and gesture production.

Feature overlap slows lexical selection: evidence from the picture-word interference paradigm.

How does the presence of a categorically related word influence picture naming latencies? In order to test competitive and noncompetitive accounts of lexical selection in spoken word production, we employed the picture-word interference (PWI) paradigm to investigate how conceptual feature overlap influences naming latencies when distractors are category coordinates of the target picture. Mahon et al. (2007. Lexical selection is not by competition: A reinterpretation of semantic interference and facilitation effects in the picture-word interference paradigm. Journal of Experimental Psychology. Learning, Memory, and Cognition, 33(3), 503-535. doi:10.1037/0278-7393.33.3.503 ) reported that semantically close distractors (e.g., zebra) facilitated target picture naming latencies (e.g., HORSE) compared to far distractors (e.g., whale). We failed to replicate a facilitation effect for within-category close versus far target-distractor pairings using near-identical materials based on feature production norms, instead obtaining reliably larger interference effects (Experiments 1 and 2). The interference effect did not show a monotonic increase across multiple levels of within-category semantic distance, although there was evidence of a linear trend when unrelated distractors were included in analyses (Experiment 2). Our results show that semantic interference in PWI is greater for semantically close than for far category coordinate relations, reflecting the extent of conceptual feature overlap between target and distractor. These findings are consistent with the assumptions of prominent competitive lexical selection models of speech production.

Monitoring amyotrophic lateral sclerosis by biomechanical modeling of speech production

Neuromotor Degenerative Diseases (NDD) affecting mainly sub-thalamic and extra-pyramidal neuromotor structures leave significant marks in speech and phonation correlates. These may be used in the characterization, detection, grading and monitoring diseases and their progress in a non-invasive way. Considering that speech and phonation recording can be carried out using handy and low-cost instrumentation, speech and phonation correlates may be quite adequate candidates to define specific NDD biomarkers for disease progress monitoring protocols. The purpose of the paper is to present the fundamentals of speech articulation biomechanical modeling from the level of signal processing to neuromotor activity inference. This backward pathway involves several inverse problems, which are addressed separately. Results from study cases relevant in Amyotrophic Lateral Sclerosis are presented and discussed. The conclusions of the research show that several correlates may be reliably established, and that monitoring disease state and progress may rely on some biomechanical correlates informing on jaw and tongue neuromotor residual activity. Possible applications of the methodology to other neurodegenerative diseases are also discussed.

A real-time MRI investigation of anticipatory posturing in prepared responses

Speakers can anticipatorily configure their vocal tracts in order to facilitate the production of an upcoming vocal response. We find that this anticipatory articulation results in decoherence of articulatory movements that are otherwise coordinated; moreover, speakers differ in the strategies they employ for response anticipation. Real-time MRI images were acquired from eight native English speakers performing a consonant-vowel response task; the task was embedded in a 2 × 2 design, which manipulated preparation (whether speakers were informed of the target response prior to a go-signal) and postural constraint (whether the response was preceded by a prolonged vowel). Analyses of pre-response articulatory postures show that all speakers exhibited anticipatory posturing of the tongue root in unconstrained responses. Some exhibited interactions between preparation and constraint, such that anticipatory posturing was more extensive in prepared- vs. unprepared-unconstrained responses. Cross-speaker variation was also observed in anticipatory posturing of the velum: some speakers raised the velum in anticipation of non-nasal responses, while others failed to do so. The results show that models of speech production must be flexible enough to allow for gestures to be executed individually, and that speakers differ in the strategies they employ for response initiation.

Acoustical bases for the perception of simulated laryngeal vocal tremor

Vocal tremor involves atypical modulation of the fundamental frequency (F0) and intensity of the voice. Previous research on vocal tremor has focused on measuring the modulation rate and extent of the F0 and intensity without characterizing other modulations present in the acoustic signal (i.e., modulation of the harmonics). Characteristics of the voice source and vocal tract filter are known to affect the amplitude of the harmonics and could potentially be manipulated to reduce the perception of vocal tremor. The purpose of this study was to determine the adjustments that could be made to the voice source or vocal tract filter to alter the acoustic output and reduce the perception of modulation. This research was carried out using a computational model of speech production that allows for precise control and modulation of the glottal and vocal tract configurations. Results revealed that listeners perceived a higher magnitude of voice modulation when simulated samples had a higher mean F0, greater degree of vocal fold adduction, and vocal tract shape for /i/ vs. /ɑ/. Based on regression analyses, listeners’ judgments were predicted by modulation information present in both low and high frequency bands. [Work supported by NIH F31-DC012697.]

Voice Onset Time in Consonant Cluster Errors: Can Phonetic Accommodation Differentiate Cognitive From Motor Errors?

Phonetic accommodation in speech errors has traditionally been used to identify the processing level at which an error has occurred. Recent studies have challenged the view that noncanonical productions may solely be due to phonetic, not phonological, processing irregularities, as previously assumed. The authors of the present study investigated the relationship between phonological and phonetic planning processes on the basis of voice onset time (VOT) behavior in consonant cluster errors. Acoustic data from 22 German speakers were recorded while eliciting errors on sibilant-stop clusters. Analyses consider VOT duration as well as intensity and spectral properties of the sibilant. Of all incorrect responses, 28% failed to show accommodation. Sibilant intensity and spectral properties differed from correct responses irrespective of whether VOT was accommodated. The data overall do not allow using (a lack of) accommodation as a diagnostic as to the processing level at which an error has occurred. The data support speech production models that allow for an integrated view of phonological and phonetic processing.

DIFFICULTIES EMERGING IN THE PROCESS OF TEACHING RUSSIAN STUDENTS TO MAKE A SPEECH IN JAPANESE AT THE STAGE OF THE SPEECH TEXT PRELIMINARY DEVELOPMENT

The article gives a detailed description of difficulties that emerge in the process of teaching Russian students to make a speech in Japanese at the stage of the speech text preliminary development. The conducted research shows that in the process of acquiring speech making skills students have considerable difficulties and make a number of various mistakes at different stages (phases) of oral speech production. We suppose that for developing an efficient model of teaching students to make a speech in Japanese it is necessary to introduce corrections into the speech production model which are directly connected with the peculiarities of making an oral presentation in Japanese at the initial stage of education.

The effect of speech situation on the occurrence of reduced word pronunciation variants

This article presents two studies investigating how the situation in which speech is uttered affects the frequency with which words are reduced. Study 1 is based on the Spoken Dutch Corpus, which consists of 15 components, nearly all representing a different speech situation. This study shows that the components differ in how often ten semantically weak words are highly reduced. The differences are especially large between the components with scripted and unscripted speech. Within the component group of unscripted speech, the formality of the situation shows an effect. Study 2 investigated segment reduction in a shadowing experiment in which participants repeated Dutch carefully and casually articulated sentences. Prefixal schwa and suffixal /t/ were absent in participants' responses to both sentences types as often as in formal interviews. If a segment was absent, this appeared to be mostly due to extreme co-articulation, unlike in speech produced in less formal situations. Speakers thus adapted more to the formal situation of the experiment than to the stimuli to be shadowed. We conclude that speech situation affects the occurrence of reduced word pronunciation variants, which should be accounted for by psycholinguistic models of speech production and comprehension.

Disfluencies as intra-utterance dialogue moves

Although disfluent speech is pervasive in spoken conversation, disfluencies have received little attention within formal theories of grammar. The majority of work on disfluent language has come from psycholinguistic models of speech production and comprehension and from structural approaches designed to improve performance in speech applications. In this paper, we argue for the inclusion of this phenomenon in the scope of formal grammar, and present a detailed formal account which: (a) unifies disfluencies (self-repair) with Clarification Requests, without conflating them, (b) offers a precise explication of the roles of all key components of a disfluency, including editing phrases and filled pauses, and (c) accounts for the possibility of self addressed questions in a disfluency. http://dx.doi.org/10.3765/sp.7.9 BibTeX info

Influence of Second Language (L2) Proficiency on the Measure of Spatiotemporal Index of Bilabial Utterances in Typical Kannada (L1) English (L2) Bilingual Speakers

The Spatiotemporal Index (STI) of Lower lip (LL) across L1 (Kannada) and L2 (English) in High Proficient (HP)/Early and ˜Low Proficient (LP)/Late typical Kannada (L1)-English (L2) bilinguals was analyzed. A bilabial utterance pair matched for syllable structure and length, phonetic complexity, and overall duration across L1 and L2 were uttered 10 times in each language randomly using language switching paradigm by HP/Early (n = 7) and LP/Late (n = 7) bilinguals. The mean STI was significantly different across languages and L2 language proficiency, supporting the use of differential speech motor control strategies by participants to maintain phonetic distinctiveness across languages. The STI was more variable in HP/Early bilinguals across L1 and L2 than LP/Late bilinguals and HP/Early bilinguals showed higher variability in Kannada (L1), reflecting higher organizational flexibility and movement control along with successful suppression of the non target language (L2). The mean STI in LP/Late bilinguals did not differ significantly across languages suggesting reduced movement flexibility and inability to suppress the interfering effect of L1 on L2. The results are discussed in the perspective of the measure of STI reflecting an intra articulatory global movement variability that is sensitive in understanding the movement control differences in the speech production of bilinguals. The findings are also discussed in the backdrop of bilingual speech production models and possible influence of âcross Linguistic Interference on articulatory movement stability across L1 and L2.

Segmental and suprasegmental errors in Spanish learning cochlear implant users: Neurolinguistic interpretation

This study aims at investigating the impact of cochlear implant (CI) use for phonological development. The main participants were a group of 14 deaf children who had received their CIs in the second year of life, and who had been wearing them for 24 months. A group of normally hearing (NH) children aged 24 months old was also evaluated. Data was obtained from a non-word repetition (NWR) task. Various segmental and suprasegmental measures were obtained from the NWR data. The CI children scored significantly below the controls for one feature (i.e. place of articulation) and for segment substitutions. Suprasegmental analyses revealed that the CI children made fewer errors with unfooted syllables and more stress errors than the NH children. Stress errors were correlated with segmental/feature errors in the CI children exclusively. We conclude that CI users struggle to acquire consonants, which may cascade into further prosodic deficits. The results are interpreted in terms of a motor control model of speech production. We suggest that CIs provide sufficient information to learn rudimentary auditory representations for syllables; however, such auditory representations might not be detailed enough to implicitly derive the somatosensory consequences of the individual consonants. • We investigate how cochlear implant (CI) use modulates the acquisition of phonology. • Place of articulation, consonant substitution and stress errors were frequent. • The substitution of sibilants and syllable omissions were relatively infrequent. • CIs provide sufficient information to learn the auditory targets of syllables. • But CI users struggle to implicitly derive the somato-sensory targets of consonants.

Independent and interacting effects of sentential context and phonological neighborhood structure in spoken word production

Models of speech production typically invoke at least two cognitive systems: a structurally static lexicon and a dynamic, compositional system. A speaker must have access to a network of lexical representations that is vested with certain intrinsically associated information (e.g., words' phonological structure), but typical speech involves sequential, online production of words to construct meaningful sentences. Thus, a complete model of speech communication must overlay the lexicon's static architecture with a dynamic, context-driven system. The present study examines effects of these two systems on spoken words' phonetic realizations. Participants produced monosyllabic words beginning with voiceless stop consonants (e.g., coat) that varied in their phonological neighborhood density. Targets were produced aloud after subjects silently read sentence contexts that were either highly predictive (e.g., She sported her stylish new fur...) or relatively neutral (e.g., The artist had trouble drawing the...). R...

Auditory feedback of one's own voice is used for high-level semantic monitoring: the "self-comprehension" hypothesis.

What would it be like if we said one thing, and heard ourselves saying something else? Would we notice something was wrong? Or would we believe we said the thing we heard? Is feedback of our own speech only used to detect errors, or does it also help to specify the meaning of what we say? Comparator models of self-monitoring favor the first alternative, and hold that our sense of agency is given by the comparison between intentions and outcomes, while inferential models argue that agency is a more fluent construct, dependent on contextual inferences about the most likely cause of an action. In this paper, we present a theory about the use of feedback during speech. Specifically, we discuss inferential models of speech production that question the standard comparator assumption that the meaning of our utterances is fully specified before articulation. We then argue that auditory feedback provides speakers with a channel for high-level, semantic “self-comprehension”. In support of this we discuss results using a method we recently developed called Real-time Speech Exchange (RSE). In our first study using RSE (Lind et al., in press) participants were fitted with headsets and performed a computerized Stroop task. We surreptitiously recorded words they said, and later in the test we played them back at the exact same time that the participants uttered something else, while blocking the actual feedback of their voice. Thus, participants said one thing, but heard themselves saying something else. The results showed that when timing conditions were ideal, more than two thirds of the manipulations went undetected. Crucially, in a large proportion of the non-detected manipulated trials, the inserted words were experienced as self-produced by the participants. This indicates that our sense of agency for speech has a strong inferential component, and that auditory feedback of our own voice acts as a pathway for semantic monitoring. We believe RSE holds great promise as a tool for investigating the role of auditory feedback during speech, and we suggest a number of future studies to serve this purpose.

Implicit target substitution and sequencing for lexical tone production in Chinese: an FMRI study.

In this study, we examine the neural substrates underlying Tone 3 sandhi and tone sequencing in Mandarin Chinese using fMRI. Tone 3 sandhi is traditionally described as the substitution of Tone 3 with Tone 2 when followed by another Tone 3 (i.e., 33→23). According to current speech production models, target substitution is expected to engage the posterior inferior frontal gyrus. Since Tone 3 sandhi is, to some extent, independent of segments, which makes it more similar to singing, right-lateralized activation in this region was predicted. As for tone sequencing, based on studies in sequencing, we expected the involvement of the supplementary motor area. In the experiments, participants were asked to produce twelve four-syllable sequences with the same tone assignment (the repeated sequences) or a different tone assignment (the mixed sequences). We found right-lateralized posterior inferior frontal gyrus activation for the sequence 3333 (Tone 3 sandhi) and left-lateralized activation in the supplementary motor area for the mixed sequences (tone sequencing). We proposed that tones and segments could be processed in parallel in the left and right hemispheres, but their integration, or the product of their integration, is hosted in the left hemisphere.

Two Distinct Auditory-Motor Circuits for Monitoring Speech Production as Revealed by Content-Specific Suppression of Auditory Cortex

Speech production, both overt and covert, down-regulates the activation of auditory cortex. This is thought to be due to forward prediction of the sensory consequences of speech, contributing to a feedback control mechanism for speech production. Critically, however, these regulatory effects should be specific to speech content to enable accurate speech monitoring. To determine the extent to which such forward prediction is content-specific, we recorded the brain's neuromagnetic responses to heard multisyllabic pseudowords during covert rehearsal in working memory, contrasted with a control task. The cortical auditory processing of target syllables was significantly suppressed during rehearsal compared with control, but only when they matched the rehearsed items. This critical specificity to speech content enables accurate speech monitoring by forward prediction, as proposed by current models of speech production. The one-to-one phonological motor-to-auditory mappings also appear to serve the maintenance of information in phonological working memory. Further findings of right-hemispheric suppression in the case of whole-item matches and left-hemispheric enhancement for last-syllable mismatches suggest that speech production is monitored by 2 auditory-motor circuits operating on different timescales: Finer grain in the left versus coarser grain in the right hemisphere. Taken together, our findings provide hemisphere-specific evidence of the interface between inner and heard speech.

Synchronization of speech and gesture: Evidence for interaction in action.

Language and action systems are highly interlinked. A critical piece of evidence is that speech and its accompanying gestures are tightly synchronized. Five experiments were conducted to test 2 hypotheses about the synchronization of speech and gesture. According to the interactive view, there is continuous information exchange between the gesture and speech systems, during both their planning and execution phases. According to the ballistic view, information exchange occurs only during the planning phases of gesture and speech, but the 2 systems become independent once their execution has been initiated. In all experiments, participants were required to point to and/or name a light that had just lit up. Virtual reality and motion tracking technologies were used to disrupt their gesture or speech execution. Participants delayed their speech onset when their gesture was disrupted. They did so even when their gesture was disrupted at its late phase and even when they received only the kinesthetic feedback of their gesture. Also, participants prolonged their gestures when their speech was disrupted. These findings support the interactive view and add new constraints on models of speech and gesture production.