Speech Production System Research Articles

Eerie voices: Odd combinations, extremes, and irregularities

The human voice can project an eerie quality when certain characteristics are present in a particular context. Some types of eerie voices may be derived from physiological scaling of the speech production system that is either humanly impossible or nearly so. By combining previous work on adult speech, and current research on speech development, the purpose of this study was to simulate vocalizations and speech based on unusual configurations of the vocal tract and vocal folds, and by imposing irregularities on movement and vibration. The resulting sound contains qualities that are human-like, but not typical, and hence may give the perceptual impression of eeriness. [Supported in part by NIH R01-DC011275.]

Sleep Underpins the Plasticity of Language Production

The constraints that govern acceptable phoneme combinations in speech perception and production have considerable plasticity. We addressed whether sleep influences the acquisition of new constraints and their integration into the speech-production system. Participants repeated sequences of syllables in which two phonemes were artificially restricted to syllable onset or syllable coda, depending on the vowel in that sequence. After 48 sequences, participants either had a 90-min nap or remained awake. Participants then repeated 96 sequences so implicit constraint learning could be examined, and then were tested for constraint generalization in a forced-choice task. The sleep group, but not the wake group, produced speech errors at test that were consistent with restrictions on the placement of phonemes in training. Furthermore, only the sleep group generalized their learning to new materials. Polysomnography data showed that implicit constraint learning was associated with slow-wave sleep. These results show that sleep facilitates the integration of new linguistic knowledge with existing production constraints. These data have relevance for systems-consolidation models of sleep.

The role of the input on the development of the LC bias: A crosslinguistic comparison

Previous studies have described the existence of a phonotactic bias called the Labial–Coronal (LC) bias, corresponding to a tendency to produce more words beginning with a labial consonant followed by a coronal consonant (i.e. “bat”) than the opposite CL pattern (i.e. “tap”). This bias has initially been interpreted in terms of articulatory constraints of the human speech production system. However, more recently, it has been suggested that this presumably language-general LC bias in production might be accompanied by LC and CL biases in perception, acquired in infancy on the basis of the properties of the linguistic input. The present study investigates the origins of these perceptual biases, testing infants learning Japanese, a language that has been claimed to possess more CL than LC sequences, and comparing them with infants learning French, a language showing a clear LC bias in its lexicon. First, a corpus analysis of Japanese IDS and ADS revealed the existence of an overall LC bias, except for plosive sequences in ADS, which show a CL bias across counts. Second, speech preference experiments showed a perceptual preference for CL over LC plosive sequences (all recorded by a Japanese speaker) in 13- but not in 7- and 10-month-old Japanese-learning infants (Experiment 1), while revealing the emergence of an LC preference between 7 and 10 months in French-learning infants, using the exact same stimuli. These crosslinguistic behavioral differences, obtained with the same stimuli, thus reflect differences in processing in two populations of infants, which can be linked to differences in the properties of the lexicons of their respective native languages. These findings establish that the emergence of a CL/LC bias is related to exposure to a linguistic input.

A comparative analysis of Serbian phonemes: Linear and non-linear models

This paper presents the results of a comparative analysis of Serbian phonemes. The characteristics of vowels are quasi-periodicity and clearly visible formants. Non-vowels are short-term quasi-periodical signals having a low power excitation signal. For the purpose of this work, speech production systems were modelled with linear AR models and the corresponding non-linear models, based feed-forward neural networks with one hidden-layer. Sum squared error minimization as well as the back-propagation algorithm were used to train models. The selection of the optimal model was based on two stopping criteria: the normalized mean squares test error and the final prediction error. The Levenberg-Marquart method was used for the Hessian matrix calculation. The Optimal Brain Surgeon method was used for pruning. The generalization properties, based on the time-domain and signal spectra of outputs at hidden-layer neurons, are presented.

Joint Spatio-Temporal Filtering Methods for DOA and Fundamental Frequency Estimation

In this paper, spatio-temporal filtering methods are proposed for estimating the direction-of-arrival (DOA) and fundamental frequency of periodic signals, like those produced by the speech production system and many musical instruments using microphone arrays. This topic has quite recently received some attention in the community and is quite promising for several applications. The proposed methods are based on optimal, adaptive filters that leave the desired signal, having a certain DOA and fundamental frequency, undistorted and suppress everything else. The filtering methods simultaneously operate in space and time, whereby it is possible resolve cases that are otherwise problematic for pitch estimators or DOA estimators based on beamforming. Several special cases and improvements are considered, including a method for estimating the covariance matrix based on the recently proposed iterative adaptive approach (IAA). Experiments demonstrate the improved performance of the proposed methods under adverse conditions compared to the state of the art using both synthetic signals and real signals, as well as illustrate the properties of the methods and the filters.

Left Lateralized Enhancement of Orofacial Somatosensory Processing Due to Speech Sounds

Somatosensory information associated with speech articulatory movements affects the perception of speech sounds and vice versa, suggesting an intimate linkage between speech production and perception systems. However, it is unclear which cortical processes are involved in the interaction between speech sounds and orofacial somatosensory inputs. The authors examined whether speech sounds modify orofacial somatosensory cortical potentials that were elicited using facial skin perturbations. Somatosensory event-related potentials in EEG were recorded in 3 background sound conditions (pink noise, speech sounds, and nonspeech sounds) and also in a silent condition. Facial skin deformations that are similar in timing and duration to those experienced in speech production were used for somatosensory stimulation. The authors found that speech sounds reliably enhanced the first negative peak of the somatosensory event-related potential when compared with the other 3 sound conditions. The enhancement was evident at electrode locations above the left motor and premotor area of the orofacial system. The result indicates that speech sounds interact with somatosensory cortical processes that are produced by speech-production-like patterns of facial skin stretch. Neural circuits in the left hemisphere, presumably in left motor and premotor cortex, may play a prominent role in the interaction between auditory inputs and speech-relevant somatosensory processing.

Anxiety and speaking in people who stutter: An investigation using the emotional Stroop task

People with anxiety disorders show an attentional bias towards threat or negative emotion words. This exploratory study examined whether people who stutter (PWS), who can be anxious when speaking, show similar bias and whether reactions to threat words also influence speech motor planning and execution. Comparisons were made between 31 PWS and 31 fluent controls in a modified emotional Stroop task where, depending on a visual cue, participants named the colour of threat and neutral words at either a normal or fast articulation rate. In a manual version of the same task participants pressed the corresponding colour button with either a long or short duration. PWS but not controls were slower to respond to threat words than neutral words, however, this emotionality effect was only evident for verbal responding. Emotionality did not interact with speech rate, but the size of the emotionality effect among PWS did correlate with frequency of stuttering. Results suggest PWS show an attentional bias to threat words similar to that found in people with anxiety disorder. In addition, this bias appears to be contingent on engaging the speech production system as a response modality. No evidence was found to indicate that emotional reactivity during the Stroop task constrains or destabilises, perhaps via arousal mechanisms, speech motor adjustment or execution for PWS.Educational objectives: The reader will be able to: (1) explain the importance of cognitive aspects of anxiety, such as attentional biases, in the possible cause and/or maintenance of anxiety in people who stutter, (2) explain how the emotional Stroop task can be used as a measure of attentional bias to threat information, and (3) evaluate the findings with respect to the relationship between attentional bias to threat information and speech production in people who stutter.

Production of child-like vowels with nonlinear interaction of glottal flow and vocal tract resonances

Acoustically, the mechanisms of vocal sound production may be considered to exist along a continuum. At one end, the glottal flow wave is weakly coupled to the resonances of the vocal tract such that the output is a linear combination of their respective acoustic characteristics, whereas at the other end there is strong nonlinear coupling of the flow source to the vocal tract resonances. To express phonetic properties in the output, such as formants, the linear case requires that the source produce sound that is rich in harmonic or broadband energy. In contrast, the nonlinear case allows for the possibility of an harmonically-rich source signal to be generated even when the glottal area variation is so simple that it may contain only one harmonic (i.e., a sinusoid) [Titze, J. Acoust. Soc. Am. 123 (2008)]. The latter case is most likely to occur when the fundamental frequency is relatively high, such as in children’s speech. The purpose of this study was to investigate the nonlinear end of the continuum with respect to the harmonic content of the glottal flow and pressure waveforms for vowels generated with a model of a child-like speech production system. [Research supported by NIH R01-DC011275, NSF BCS-1145011.]

Morphological Variation in the Adult Hard Palate and Posterior Pharyngeal Wall

Adult human vocal tracts display considerable morphological variation across individuals, but the nature and extent of this variation has not been extensively studied for many vocal tract structures. There exists a need to analyze morphological variation and, even more basically, to develop a methodology for morphological analysis of the vocal tract. Such analysis will facilitate fundamental characterization of the speech production system, with broad implications from modeling to explaining interspeaker variability. A data-driven methodology to automatically analyze the extent and variety of morphological variation is proposed and applied to a diverse subject pool of 36 adults. Analysis is focused on two key aspects of vocal tract structure: the midsagittal shape of the hard palate and the posterior pharyngeal wall. Result Palatal morphology varies widely in its degree of concavity but also in anteriority and sharpness. Pharyngeal wall morphology, by contrast, varies mostly in terms of concavity alone. The distribution of morphological characteristics is complex, and analysis suggests that certain variations may be categorical in nature. Major modes of morphological variation are identified, including their relative magnitude, distribution, and categorical nature. Implications of these findings for speech articulation strategies and speech acoustics are discussed.

Aberrant connectivity of areas for decoding degraded speech in patients with auditory verbal hallucinations

Auditory verbal hallucinations (AVH) are a hallmark of psychotic experience. Various mechanisms including misattribution of inner speech and imbalance between bottom-up and top-down factors in auditory perception potentially due to aberrant connectivity between frontal and temporo-parietal areas have been suggested to underlie AVH. Experimental evidence for disturbed connectivity of networks sustaining auditory-verbal processing is, however, sparse. We compared functional resting-state connectivity in 49 psychotic patients with frequent AVH and 49 matched controls. The analysis was seeded from the left middle temporal gyrus (MTG), thalamus, angular gyrus (AG) and inferior frontal gyrus (IFG) as these regions are implicated in extracting meaning from impoverished speech-like sounds. Aberrant connectivity was found for all seeds. Decreased connectivity was observed between the left MTG and its right homotope, between the left AG and the surrounding inferior parietal cortex (IPC) and the left inferior temporal gyrus, between the left thalamus and the right cerebellum, as well as between the left IFG and left IPC, and dorsolateral and ventrolateral prefrontal cortex (DLPFC/VLPFC). Increased connectivity was observed between the left IFG and the supplementary motor area (SMA) and the left insula and between the left thalamus and the left fusiform gyrus/hippocampus. The predisposition to experience AVH might result from decoupling between the speech production system (IFG, insula and SMA) and the self-monitoring system (DLPFC, VLPFC, IPC) leading to misattribution of inner speech. Furthermore, decreased connectivity between nodes involved in speech processing (AG, MTG) and other regions implicated in auditory processing might reflect aberrant top-down influences in AVH.

Shared and distinct neural correlates of vowel perception and production

Recent neurobiological models postulate that sensorimotor interactions play a key role in speech perception and speech motor control, especially under adverse listening conditions or in case of complex articulatory speech sequences. The present fMRI study aimed to investigate whether isolated vowel perception and production might also induce sensorimotor activity, independently of syllable sequencing and coarticulation mechanisms and using a sparse acquisition technique in order to limit influence of scanner noise. To this aim, participants first passively listened to French vowels previously recorded from their own voice. In a subsequent production task, done within the same imaging session and using the same acquisition parameters, participants were asked to overtly produce the same vowels. Our results demonstrate that a left postero-dorsal stream, linking auditory speech percepts with articulatory representations and including the posterior inferior frontal gyrus, the adjacent ventral premotor cortex and the temporoparietal junction, is an influential part of both vowel perception and production. Specific analyses on phonetic features further confirmed the involvement of the left postero-dorsal stream in vowel processing and motor control. Altogether, these results suggest that vowel representations are largely distributed over sensorimotor brain areas and provide further evidence for a functional coupling between speech perception and production systems. ► Sparse sampling fMRI study of vowel perception and production. ► Implication of a left postero-dorsal stream in both vowel perception and production. ► Vowel representations distributed over sensorimotor brain areas. ► Support for a close relationship between speech perception and production.

Nepali Text to Speech Synthesis System using ESNOLA Method of Concatenation

synthesis the input text is converted into such form that the low level synthesizer can produce the output speech. The three basic methods for low level synthesis are the formant, concatenative, and articulatory synthesis. The formant synthesis is based on the modeling of the resonances in the vocal tract and is perhaps the most commonly used during the last decades. However, the concatenative synthesis which is based on playing pre-recorded samples from natural speech is becoming more popular. The most accurate method is articulatory synthesis which models the human speech production system directly, but it is also the most difficult approach. Since the quality of synthetic speech is improving steadily, the application field is also expanding rapidly. Different methodologies for Concatenative Synthesis are there like TDPSOLA, PSOLA, MBROLA and ESNOLA. The Nepali text to speech synthesis system described here is based on Concatenative technique that uses Epoch Synchronous Non Overlap Add Method (ESNOLA) where the speech signal dictionary is created by cutting the pre-recorded natural speech such that the speech segments begin and ends all at the positive zero crossing to eliminate mismatch or distortion during concatenation. The system uses partneme as the smallest signal units for concatenation. ESNOLA technique provides the complete control on implementation of intonation and prosody [2, 10]. It allows judicious selection of signal segment so that smaller fundamental parts of the phonemes may be used as units reducing both the number and the size of the signal elements in the dictionary. Further the methodology of concatenation provides adequate processing for proper matching between different segments during concatenation. The use of special type of basic signal segment makes the size of signal dictionary very small so that there is a possibility of its implementation in low-cost, general-purpose electronic devices.

Are speech attractor models useful in diagnosing vocal fold pathologies?

The speech is an easily accessible signal which clearly represents the characteristics of larynx and vocal folds. Therefore, application of some proper machine learning algorithms on a small part of a recorded speech signal may help in non-invasive diagnosing of vocal fold diseases. Since there are some experimental evidences that suggest the existence of chaotic behavior in speech production system, in this paper a new method is proposed to predict vocal fold pathologies using its chaotic characteristics. The proposed method is based on modeling of pathological voice as a speech trajectory in the reconstructed phase space.

The labial–coronal effect revisited: Japanese adults say pata, but hear tapa

The labial–coronal effect has originally been described as a bias to initiate a word with a labial consonant–vowel–coronal consonant (LC) sequence. This bias has been explained with constraints on the human speech production system, and its perceptual correlates have motivated the suggestion of a perception–production link. However, previous studies exclusively considered languages in which LC sequences are globally more frequent than their counterpart. The current study examined the LC bias in speakers of Japanese, a language that has been claimed to possess more CL than LC sequences. We first conducted an analysis of Japanese corpora that qualified this claim, and identified a subgroup of consonants (plosives) exhibiting a CL bias. Second, focusing on this subgroup of consonants, we found diverging results for production and perception such that Japanese speakers exhibited an articulatory LC bias, but a perceptual CL bias. The CL perceptual bias, however, was modulated by language of presentation, and was only present for stimuli recorded by a Japanese, but not a French, speaker. A further experiment with native speakers of French showed the opposite effect, with an LC bias for French stimuli only. Overall, we find support for a universal, articulatory motivated LC bias in production, supporting a motor explanation of the LC effect, while perceptual biases are influenced by distributional frequencies of the native language.

Statistical methods for estimation of direct and differential kinematics of the vocal tract

We present and evaluate two statistical methods for estimating kinematic relationships of the speech production system: artificial neural networks and locally-weighted regression. The work is motivated by the need to characterize this motor system, with particular focus on estimating differential aspects of kinematics. Kinematic analysis will facilitate progress in a variety of areas, including the nature of speech production goals, articulatory redundancy and, relatedly, acoustic-to-articulatory inversion. Statistical methods must be used to estimate these relationships from data since they are infeasible to express in closed form. Statistical models are optimized and evaluated – using a heldout data validation procedure – on two sets of synthetic speech data. The theoretical and practical advantages of both methods are also discussed. It is shown that both direct and differential kinematics can be estimated with high accuracy, even for complex, nonlinear relationships. Locally-weighted regression displays the best overall performance, which may be due to practical advantages in its training procedure. Moreover, accurate estimation can be achieved using only a modest amount of training data, as judged by convergence of performance. The algorithms are also applied to real-time MRI data, and the results are generally consistent with those obtained from synthetic data.

USING NONLINEAR MODELING OF RECONSTRUCTED PHASE SPACE AND FREQUENCY DOMAIN ANALYSIS TO IMPROVE AUTOMATIC SPEECH RECOGNITION PERFORMANCE

This paper introduces a combinational feature extraction approach to improve speech recognition systems. The main idea is to simultaneously benefit from some features obtained from nonlinear modeling applied to speech reconstructed phase space (RPS) and typical Mel frequency Cepstral coefficients (MFCCs) which have a proved role in speech recognition field. With an appropriate dimension, the reconstructed phase space of speech signal is assured to be topologically equivalent to the dynamics of the speech production system, and could therefore include information that may be absent in linear analysis approaches. In the first part of this paper the application of Lyapunov Exponents (LE) and Fractal Dimension as two usual chaotic features in speech recognition are tested and then a short discussion is made on the weakness of these features in speech recognition. In the following, a statistical modeling approach based on Gaussian mixture models (GMMs) is applied to speech RPS. A final pruned feature set is obtained by applying an efficient feature selection approach to the combination of the parameters of the GMM model and MFCC-based features. A hidden Markov model-based (HMM) speech recognition system and TIMIT speech database are used to evaluate the performance of the proposed feature set by conducting isolated and continuous speech recognition experiments. In final Continuous Speech Recognition (CSR) Experiments, using tri-phone models, 3.7% absolute phoneme recognition accuracy improvement against using MFCC features alone were obtained.

Statistical estimation of speech kinematics from real-time MRI data

The human speech production system can be fundamentally characterized by the kinematic relationships between low-level articulator variables and relatively high-level tasks. Such kinematics can be illuminating about many system aspects from degrees of freedom and redundancy to dynamics and even control. Since these relationships are generally complex and infeasible to express in closed form, recent work has focused on statistical methods for estimating the relevant relationships from data (Saltzman, 2006, in Dynamics of Speech Production and Perception; Lammert, 2010, Proc. INTERSPEECH). Such methods have been applied to synthetic speech data in order to evaluate their effectiveness, but they have not yet been demonstrated on real data. Here, we apply these methods to real speech data acquired from real-time magnetic resonance imaging. We extract articulator variables that are consistent with those chosen by various articulatory models, and we relate them to high-level task variables such as constriction locations and degrees and formant frequencies. This is done to facilitate an analysis of articulatory motor control during speech production. [Work supported by NIH.]

Shared Language

Whether the brain’s speech-production system is also involved in speech comprehension is a topic of much debate. Research has focused on whether motor areas are involved in listening, but overlap between speaking and listening might occur not only at primary sensory and motor levels, but also at linguistic levels (where semantic, lexical, and syntactic processes occur). Using functional MRI adaptation during speech comprehension and production, we found that the brain areas involved in semantic, lexical, and syntactic processing are mostly the same for speaking and for listening. Effects of primary processing load (indicative of sensory and motor processes) overlapped in auditory cortex and left inferior frontal cortex, but not in motor cortex, where processing load affected activity only in speaking. These results indicate that the linguistic parts of the language system are used for both speaking and listening, but that the motor system does not seem to provide a crucial contribution to listening.

Abnormal Structure-Function Relationship in Spasmodic Dysphonia

Spasmodic dysphonia (SD) is a primary focal dystonia characterized by involuntary spasms in the laryngeal muscles during speech production. Although recent studies have found abnormal brain function and white matter organization in SD, the extent of gray matter alterations, their structure-function relationships, and correlations with symptoms remain unknown. We compared gray matter volume (GMV) and cortical thickness (CT) in 40 SD patients and 40 controls using voxel-based morphometry and cortical distance estimates. These measures were examined for relationships with blood oxygen level-dependent signal change during symptomatic syllable production in 15 of the same patients. SD patients had increased GMV, CT, and brain activation in key structures of the speech control system, including the laryngeal sensorimotor cortex, inferior frontal gyrus (IFG), superior/middle temporal and supramarginal gyri, and in a structure commonly abnormal in other primary dystonias, the cerebellum. Among these regions, GMV, CT and activation of the IFG and cerebellum showed positive relationships with SD severity, while CT of the IFG correlated with SD duration. The left anterior insula was the only region with decreased CT, which also correlated with SD symptom severity. These findings provide evidence for coupling between structural and functional abnormalities at different levels within the speech production system in SD.

Processing speech signal using auditory-like filterbank provides least uncertainty about articulatory gestures

Understanding how the human speech production system is related to the human auditory system has been a perennial subject of inquiry. To investigate the production-perception link, in this paper, a computational analysis has been performed using the articulatory movement data obtained during speech production with concurrently recorded acoustic speech signals from multiple subjects in three different languages: English, Cantonese, and Georgian. The form of articulatory gestures during speech production varies across languages, and this variation is considered to be reflected in the articulatory position and kinematics. The auditory processing of the acoustic speech signal is modeled by a parametric representation of the cochlear filterbank which allows for realizing various candidate filterbank structures by changing the parameter value. Using mathematical communication theory, it is found that the uncertainty about the articulatory gestures in each language is maximally reduced when the acoustic speech signal is represented using the output of a filterbank similar to the empirically established cochlear filterbank in the human auditory system. Possible interpretations of this finding are discussed.