Speech Production Process Research Articles

Processing of acoustic and phonological information of lexical tones in Mandarin Chinese revealed by mismatch negativity

The accurate perception of lexical tones in tonal languages involves the processing of both acoustic information and phonological information carried by the tonal signal. In this study we evaluated the relative role of the two types of information in native Chinese speaker’s processing of tones at a preattentive stage with event-related potentials (ERPs), particularly the mismatch negativity (MNN). Specifically, we distinguished the acoustic from the phonological information by manipulating phonological category and acoustic interval of the stimulus materials. We found a significant main effect of phonological category for the peak latency of MMN, but a main effect of both phonological category and acoustic interval for the mean amplitude of MMN. The results indicated that the two types of information, acoustic and phonological, play different roles in the processing of Chinese lexical tones: acoustic information only impacts the extent of tonal processing, while phonological information affects both the extent and the time course of tonal processing. Implications of these findings are discussed in light of neurocognitive processes of phonological processing.

Multi-level processing of phonetic variants in speech production and visual word processing: evidence from Mandarin lexical tones

Two picture–word interference experiments provide new evidence on the nature of phonological processing in speech production and visual word processing. In both experiments, responses were significantly faster either when distractor and target matched in tone category, but had different overt realisations (toneme condition) or when target and distractor matched in overt realisation, but mismatched in tone category (contour condition). Tone 3 sandhi is an allophone of Beijing Mandarin Tone 3 (T3). Its contour is similar to another tone, Tone 2. In Experiment 1, sandhi picture naming was faster with contour (Tone 2) and toneme (low Tone 3) distractors, compared to control distractors. This indicates both category and context-specific representations are activated in sandhi word production. In Experiment 2, both contour (Tone 2) and toneme (low Tone 3) picture naming was facilitated by visually presented sandhi distractors, compared to controls, evidence that category and context-specific instantiated representations are automatically activated during processing of visually presented words. Combined, the results point to multi-level processing of phonology, whether words are overtly produced or processed visually. Interestingly, there were differences in the time course of effects.

Identifying underlying articulatory targets of Thai vowels from acoustic data based on an analysis-by-synthesis approach

This paper investigates the estimation of underlying articulatory targets of Thai vowels as invariant representation of vocal tract shapes by means of analysis-by-synthesis based on acoustic data. The basic idea is to simulate the process of learning speech production as a distal learning task, with acoustic signals of natural utterances in the form of Mel-frequency cepstral coefficients (MFCCs) as input, VocalTractLab - a 3D articulatory synthesizer controlled by target approximation models as the learner, and stochastic gradient descent as the target training method. To test the effectiveness of this approach, a speech corpus was designed to contain contextual variations of Thai vowels by juxtaposing nine Thai long vowels in two-syllable sequences. A speech corpus consisting of 81 disyllabic utterances was recorded from a native Thai speaker. Nine vocal tract shapes, each corresponding to a vowel, were estimated by optimizing the vocal tract shape parameters of each vowel to minimize the sum of square error of MFCCs between original and synthesized speech. The stochastic gradient descent algorithm was used to iteratively optimize the shape parameters. The optimized vocal tract shapes were then used to synthesize Thai vowels both in monosyllables and in disyllabic sequences. The results, both numerically and perceptually, indicate that this model-based analysis strategy allows us to effectively and economically estimate the vocal tract shapes to synthesize accurate Thai vowels as well as smooth formant transitions between adjacent vowels.

Dissecting the functional anatomy of auditory word repetition.

This fMRI study used a single, multi-factorial, within-subjects design to dissociate multiple linguistic and non-linguistic processing areas that are all involved in repeating back heard words. The study compared: (1) auditory to visual inputs; (2) phonological to non-phonological inputs; (3) semantic to non-semantic inputs; and (4) speech production to finger-press responses. The stimuli included words (semantic and phonological inputs), pseudowords (phonological input), pictures and sounds of animals or objects (semantic input), and colored patterns and hums (non-semantic and non-phonological). The speech production tasks involved auditory repetition, reading, and naming while the finger press tasks involved one-back matching. The results from the main effects and interactions were compared to predictions from a previously reported functional anatomical model of language based on a meta-analysis of many different neuroimaging experiments. Although many findings from the current experiment replicated many of those predicted, our within-subject design also revealed novel results by providing sufficient anatomical precision to dissect several different regions within the anterior insula, pars orbitalis, anterior cingulate, SMA, and cerebellum. For example, we found one part of the pars orbitalis was involved in phonological processing and another in semantic processing. We also dissociated four different types of phonological effects in the left superior temporal sulcus (STS), left putamen, left ventral premotor cortex, and left pars orbitalis. Our findings challenge some of the commonly-held opinions on the functional anatomy of language, and resolve some previously conflicting findings about specific brain regions—and our experimental design reveals details of the word repetition process that are not well captured by current models.

Bilingualism protects anterior temporal lobe integrity in aging

Cerebral gray-matter volume (GMV) decreases in normal aging but the extent of the decrease may be experience-dependent. Bilingualism may be one protective factor and in this article we examine its potential protective effect on GMV in a region that shows strong age-related decreases—the left anterior temporal pole. This region is held to function as a conceptual hub and might be expected to be a target of plastic changes in bilingual speakers because of the requirement for these speakers to store and differentiate lexical concepts in 2 languages to guide speech production and comprehension processes. In a whole brain comparison of bilingual speakers (n = 23) and monolingual speakers (n = 23), regressing out confounding factors, we find more extensive age-related decreases in GMV in the monolingual brain and significantly increased GMV in left temporal pole for bilingual speakers. Consistent with a specific neuroprotective effect of bilingualism, region of interest analyses showed a significant positive correlation between naming performance in the second language and GMV in this region. The effect appears to be bilateral though because there was a nonsignificantly different effect of naming performance on GMV in the right temporal pole. Our data emphasize the vulnerability of the temporal pole to normal aging and the value of bilingualism as both a general and specific protective factor to GMV decreases in healthy aging.

The roles of family history of dyslexia, language, speech production and phonological processing in predicting literacy progress

It is well established that speech, language and phonological skills are closely associated with literacy, and that children with a family risk of dyslexia (FRD) tend to show deficits in each of these areas in the preschool years. This paper examines what the relationships are between FRD and these skills, and whether deficits in speech, language and phonological processing fully account for the increased risk of dyslexia in children with FRD. One hundred and fifty-three 4-6-year-old children, 44 of whom had FRD, completed a battery of speech, language, phonology and literacy tasks. Word reading and spelling were retested 6 months later, and text reading accuracy and reading comprehension were tested 3 years later. The children with FRD were at increased risk of developing difficulties in reading accuracy, but not reading comprehension. Four groups were compared: good and poor readers with and without FRD. In most cases good readers outperformed poor readers regardless of family history, but there was an effect of family history on naming and nonword repetition regardless of literacy outcome, suggesting a role for speech production skills as an endophenotype of dyslexia. Phonological processing predicted spelling, while language predicted text reading accuracy and comprehension. FRD was a significant additional predictor of reading and spelling after controlling for speech production, language and phonological processing, suggesting that children with FRD show additional difficulties in literacy that cannot be fully explained in terms of their language and phonological skills.

Phonetic processing of non-native speech in semantic vs non-semantic tasks

Research with speakers with acquired production difficulties has suggested phonetic processing is more difficult in tasks that require semantic processing. The current research examined whether similar effects are found in bilingual phonetic processing. English-French bilinguals' productions in picture naming (which requires semantic processing) were compared to those elicited by repetition (which does not require semantic processing). Picture naming elicited slower, more accented speech than repetition. These results provide additional support for theories integrating cognitive and phonetic processes in speech production and suggest that bilingual speech research must take cognitive factors into account when assessing the structure of non-native sound systems.

Neural network connectivity differences in children who stutter

Affecting 1% of the general population, stuttering impairs the normally effortless process of speech production, which requires precise coordination of sequential movement occurring among the articulatory, respiratory, and resonance systems, all within millisecond time scales. Those afflicted experience frequent disfluencies during ongoing speech, often leading to negative psychosocial consequences. The aetiology of stuttering remains unclear; compared to other neurodevelopmental disorders, few studies to date have examined the neural bases of childhood stuttering. Here we report, for the first time, results from functional (resting state functional magnetic resonance imaging) and structural connectivity analyses (probabilistic tractography) of multimodal neuroimaging data examining neural networks in children who stutter. We examined how synchronized brain activity occurring among brain areas associated with speech production, and white matter tracts that interconnect them, differ in young children who stutter (aged 3-9 years) compared with age-matched peers. Results showed that children who stutter have attenuated connectivity in neural networks that support timing of self-paced movement control. The results suggest that auditory-motor and basal ganglia-thalamocortical networks develop differently in stuttering children, which may in turn affect speech planning and execution processes needed to achieve fluent speech motor control. These results provide important initial evidence of neurological differences in the early phases of symptom onset in children who stutter.

The Role of Interhemisphere Interactions in Children Aged 5–6 Years and Adults in Supporting Verbal-Mnestic Activity Associated with the Formation and Analysis of Verbal Utterances

This report describes studies of the establishment of the neurophysiological mechanisms underlying mental speech activity associated with the synthesis of speech units from sets of more elementary units in children aged 5‐6 years. Comparison of the results with data on analogous activity in adults, as well as in children and adults during verbal activity directed to recognizing phonemes in words and grammatical and semantic errors in sentences, showed that support of the speech production processes underlying the mental formation of words and sentences, as well as verbal activity requiring recognition and a critical approach to speech perception, requires coordinated simultaneous activity of the left and right hemispheres. In children and especially in adults, this is apparent as increases in interhemisphere interactions, mainly in the temporal lobes of the two hemispheres, the inferior frontal areas, and the TPO zone (the zone at which the temporal, parietal, and occipital areas of the cortex overlap). These data indicate that the neurophysiological mechanisms originating verbal constructs in preschool children are more mature than the processes accompanying the analytical approach to the grammatical, semantic, and phonetic characteristics of perceived speech.

Classification of speech under stress based on modeling of the vocal folds and vocal tract

In this study, we focus on the classification of neutral and stressed speech based on a physical model. In order to represent the characteristics of the vocal folds and vocal tract during the process of speech production and to explore the physical parameters involved, we propose a method using the two-mass model. As feature parameters, we focus on stiffness parameters of the vocal folds, vocal tract length, and cross-sectional areas of the vocal tract. The stiffness parameters and the area of the entrance to the vocal tract are extracted from the two-mass model after we fit the model to real data using our proposed algorithm. These parameters are related to the velocity of glottal airflow and acoustic interaction between the vocal folds and the vocal tract and can precisely represent features of speech under stress because they are affected by the speaker’s psychological state during speech production. In our experiments, the physical features generated using the proposed approach are compared with traditionally used features, and the results demonstrate a clear improvement of up to 10% to 15% in average stress classification performance, which shows that our proposed method is more effective than conventional methods.

Is lexical selection in spoken word production competitive? Introduction to the special issue on lexical competition in language production

A common assumption in research on spoken word production is that lexical selection is a competition-based process among co-activated lexical representations. This assumption of competitive lexical selection has been challenged by an alternative account, which places competition at a postlexical response selection stage. A complex pattern of empirical findings from picture-word interference and other tasks has emerged which constrains current thinking about word production. In this article we provide an overview over the main positions, empirical findings, and put the various contributions to this Special Issue into a wider context. The theoretical debate is far from closed, but it has drawn attention to some critical points that we emphasise in this Editorial: The speech production process needs an element of competition, but this competition need not necessarily take place during lexical selection. Behavioural interference effects are caused by a combination of facilitation and interference, but there is discord about the processing levels at which these mechanisms are located. Finally, we stress the necessity to use findings from different experimental paradigms for theory-building and advocate a shift from the narrow focus on the picture-word paradigm that has dominated the theoretical discussion in this Special Issue and elsewhere.

The Development of a Speech Recognition Software Application Using HMM, ANN and LPC Models to Minimize Error

The purpose of this application is to develop speech recognition software applications and test it. The developed speech recognition system for human language solves for processing the voice input and answering the queries of the user. This system begins with an introduction to Speech Recognition Technology then it explains how it works, and the level of accuracy that can be expected. The system is considered for the possible applications of speech technology, including every single task related with the action of the human voice. In this sense, the application fields can vary from speech production, storage, transmission and recognition processes. This research pertains to discuss the theories and models on speech communication and more importantly the use of technology for speech recognition. Several authors’ views are discussed but a great emphasis is given to the Hidden Markov Models (HMM), Artificial Neural Network (ANN) and Liner Predictive Coding (LPC). The aim of the research is to test these models and outline voice recognition algorithms which are important in improving the voice recognition performance. The application is tested with speakers used to represent different tones, emotions and lengths in metres as a sample of relationships methodology, the reports the system permits are used in the results analysis.

Effects of prosodic strengthening and lexical boundary on /s/-stop sequences in English

This study examined how the effects of prosodic strengthening (from prosodic boundary and accent) and lexical boundary (e.g., “ice # can” vs. “eye # scan”) are acoustic-phonetically realized on English /s/-stop sequences in a sentence. First, the domain-initial strengthening effect was not strictly confined to the first segment, but could extend into the second consonant and, at least partially, into the following vowel in the #/sCV/ sequence (e.g., in “scan”). Second, the accent-induced strengthening effect was robust in all acoustic measures for the #/sCV/ sequence. Third, prosodic strengthening arising with boundary and accent gave rise to the “shortened” VOT for the voiceless stop in the #/sCV/ sequence, suggesting that prosodic strengthening can operate on the phonetic manifestation of a phonological rule to reinforce the language-specific phonetic feature, which is, in this case, {-spread glottis}. Fourth, domain-initial strengthening and accent-induced strengthening differ substantially in some aspects, suggesting that they may be encoded separately in speech production process. Finally, “ice # can” and “eye # scan” were indeed very differently realized, suggesting that the underlying lexical boundary is signaled by fine-phonetic details even when the sequences occurred phrase-internally where they appeared to be homophonous, at least impressionistically, and syllabified the same.

Acoustic Features of Filled Pauses in Polish Task-Oriented Dialogues

Abstract Filled pauses (FPs) have proved to be more than valuable cues to speech production processes and important units in discourse analysis. Some aspects of their form and occurrence patterns have been shown to be speaker- and language-specific. In the present study, basic acoustic properties of FPs in Polish task-oriented dialogues are explored. A set of FPs was extracted from a corpus of twenty task- oriented dialogues on the basis of available annotations. After initial scrutiny and selection, a subset of the signals underwent a series of pitch, formant frequency and voice quality analyses. A significant amount of variation found in the realisations of FPs justifies their potential application in speaker recognition systems. Regular monosegmental FPs were confirmed to show relatively stable basic acoustic parameters, which allows for their easy identification and measurements but it may result in less significant differences among the speakers.

Effects of sentence-structure complexity on speech initiation time and disfluency

There is general agreement that stuttering is caused by a variety of factors, and language formulation and speech motor control are two important factors that have been implicated in previous research, yet the exact nature of their effects is still not well understood. Our goal was to test the hypothesis that sentences of high structural complexity would incur greater processing costs than sentences of low structural complexity and these costs would be higher for adults who stutter than for adults who do not stutter. Fluent adults and adults who stutter participated in an experiment that required memorization of a sentence classified as low or high structural complexity followed by production of that sentence upon a visual cue. Both groups of speakers initiated most sentences significantly faster in the low structural complexity condition than in the high structural complexity condition. Adults who stutter were over-all slower in speech initiation than were fluent speakers, but there were no significant interactions between complexity and group. However, adults who stutter produced significantly more disfluencies in sentences of high structural complexity than in those of low complexity.Educational objectives: After reading this article, the learner will be able to: (a) identify integral parts of all well-known models of adult sentence production; (b) summarize the way that sentence structure might negatively influence the speech production processes; (c) discuss whether sentence structure influences speech initiation time and disfluencies.

Converging toward a common speech code: imitative and perceptuo-motor recalibration processes in speech production.

Auditory and somatosensory systems play a key role in speech motor control. In the act of speaking, segmental speech movements are programmed to reach phonemic sensory goals, which in turn are used to estimate actual sensory feedback in order to further control production. The adult's tendency to automatically imitate a number of acoustic-phonetic characteristics in another speaker's speech however suggests that speech production not only relies on the intended phonemic sensory goals and actual sensory feedback but also on the processing of external speech inputs. These online adaptive changes in speech production, or phonetic convergence effects, are thought to facilitate conversational exchange by contributing to setting a common perceptuo-motor ground between the speaker and the listener. In line with previous studies on phonetic convergence, we here demonstrate, in a non-interactive situation of communication, online unintentional and voluntary imitative changes in relevant acoustic features of acoustic vowel targets (fundamental and first formant frequencies) during speech production and imitation. In addition, perceptuo-motor recalibration processes, or after-effects, occurred not only after vowel production and imitation but also after auditory categorization of the acoustic vowel targets. Altogether, these findings demonstrate adaptive plasticity of phonemic sensory-motor goals and suggest that, apart from sensory-motor knowledge, speech production continuously draws on perceptual learning from the external speech environment.

Are Number Words Fundamentally Different? A Qualitative Analysis of Aphasic Errors in Word and Number Word Production

A number of previous studies reported dissociations between the processing of number words and other words and reasoned that there are fundamentally different mental representations and/or processing strategies related to these types of words. Messina et al. reported the performance of Italian aphasic patients with words and number words in different tasks [1]. In line with previous studies, they found that lexical errors formed the dominant error type in number words, whereas phonological errors were the most frequent error type in other words. Messina et al. [1] concluded that speech production processes differ categorically between number words and other words, leading to qualitatively different error patterns in language breakdown. The present study examined error types in number words and other words in a sample of 15 German patients with aphasia using the same tasks as Messina et al. [1]. Performance in reading and repeating number words and other words was analysed. In general, we replicated the dissociation of error types between number words and other words reported by Messina et al. [1] and others. However, in contrast to previous assumptions this dissociation was not categorical but rather gradual. We suggest that psycholinguistic stimulus properties (such as number of repetitions per morpheme) and type of task influenced error types in a gradual fashion. Keywords: Aphasia, number word processing, transcoding, phonological errors, lexical errors, dissociation. Read more →

A multi-modal approach to intervention for one adolescent's frontal lisp

An adolescent with a persistent frontal lisp participated in a two-part 11-session intervention case study. The first phase used ultrasound imagery and acoustic, phonetic and voice education to provide information about articulatory setting (AS) and general awareness of the speech production process. The second phase used traditional articulation therapy, online visual–acoustic biofeedback and fluency strategies to target the frontal lisp directly (specifically /s/, /z/, /ʃ/ and /ʧ/). Trained listener evaluations of pre-intervention, post-phase 1 and post-phase 2 assessments showed no improvement after phase 1, but notable improvement in all treatment targets immediately after phase 2. These improvements were substantially maintained at assessment 4 months post-intervention. The outcomes suggest that direct training was more effective than the AS approach; however, the client's ability to self-monitor in phase 2, rapid acquisition of the targets and maintenance at 4 months post-intervention possibly reflected the knowledge gained in phase 1 about AS.

The effect of alcohol on short term memory and speech production processes

Kutatásunkban az alkohol beszédre és rövid távú emlékezetre gyakorolt hatását vizsgáltuk jól kontrollált feltételek mellett. Ennek vizsgálatára a Jacobs-féle számterjedelemtesztet és nyelvi szintek szerint csoportosított nyelvtörőket használtunk, és összehasonlítottuk alkoholos és kontrollfeltételben a számterjedelemteszten nyújtott teljesítményt, illetve a nyelvtörők kimondása során elkövetett hibák számát és típusát. Különválasztottuk és elemeztük az emlékezeti teljesítmény romlásából fakadó és a motoros-artikulációs folyamatokból eredő hibákat. Az eredmények azt mutatják, hogy az alkohol nincs hatással a számterjedelemre, azonban hatás-sal van a nyelvtörők közvetlen felidézésére. Ugyanakkor a nyelvtörők visszamondásában elkövetett hibák 58–61% a a pontatlan artikulációból fakad. Elemeztük az artikulációs tempó, a hangmagasság és a szünetek számának változását is. Az eredmények nagyrészt egybevágnak korábbi angol és német kutatási eredményekkel, de részben ki is egészítik azokat.

Sparse Linear Prediction and Its Applications to Speech Processing

The aim of this paper is to provide an overview of Sparse Linear Prediction, a set of speech processing tools created by introducing sparsity constraints into the linear prediction framework. These tools have shown to be effective in several issues related to modeling and coding of speech signals. For speech analysis, we provide predictors that are accurate in modeling the speech production process and overcome problems related to traditional linear prediction. In particular, the predictors obtained offer a more effective decoupling of the vocal tract transfer function and its underlying excitation, making it a very efficient method for the analysis of voiced speech. For speech coding, we provide predictors that shape the residual according to the characteristics of the sparse encoding techniques resulting in more straightforward coding strategies. Furthermore, encouraged by the promising application of compressed sensing in signal compression, we investigate its formulation and application to sparse linear predictive coding. The proposed estimators are all solutions to convex optimization problems, which can be solved efficiently and reliably using, e.g., interior-point methods. Extensive experimental results are provided to support the effectiveness of the proposed methods, showing the improvements over traditional linear prediction in both speech analysis and coding.