Speech Production Mechanism Research Articles

Speaker change detection in casual conversations using excitation source features

In this paper we propose a method for speaker change detection using features of excitation source of the speech production mechanism. The method uses neural network models to capture the speaker-specific information from a signal that represents predominantly the excitation source. The focus in this paper is on speaker change detection in casual telephone conversations, in which short (<5 s) speaker turns are common. Excitation source features are a better choice for modeling a speaker, when limited amount of speech data is available, when compared to the vocal tract system features. Linear prediction residual is used as an estimate of the excitation source signal. Autoassociative neural network models are proposed to capture the higher order relations among the samples of the residual signal. Speaker models are generated for every one second of voiced speech from the first few seconds of the conversation. These models are used to detect the speaker change points. Performance of the proposed method for speaker change detection is evaluated on a database containing several two-speaker conversations.

Changes In Speech Production In An Early Deafened Adult With A Cochlear Implant

The current study is a first investigation reporting the speech production characteristics of an early deafened adult cochlear implant user after a course of speech-language treatment. The participant is culturally deaf and received the cochlear implant when she was 43 years old. A 24-week ABCABC single-subject treatment programme was conducted addressing articulation, the oral production of printed words, and voice production, with two 4-week segments for each area. Treatment-specific progress, revealed by untrained stimuli, was made in areas of articulation and oral production of printed words, but not voice production. Formal measures also confirmed the patient's progress. These results were discussed in relation to how long-term reduction of general auditory input and under-use of the speech production mechanisms can be remediated by technological and behavioural treatment.

Cortical mechanism of communicative speech production

Communicative speech requires conformity not only to linguistic rules but also to behavior that is appropriate for social interaction. The existence of a special brain mechanism for such behavioral aspects of communicative speech has been suggested by studies of social impairment in autism, and it may be related to communicative vocalization in animals. We used functional magnetic resonance imaging (fMRI) to measure cortical activation while normal subjects casually talked to an actor (communication task) or verbally described a situation (description task) while observing video clips of an action performed by a familiar or an unfamiliar actor in a typical daily situation. We assumed that the communication task differed from the description task in the involvement of behavioral aspects of communicative speech production, which may involve the processing of interaction-relevant biographical information. Significantly higher activation was observed during the communication task than during the description task in the medial prefrontal cortex (polar and dorsal parts), the bilateral anterior superior temporal sulci, and the left temporoparietal junction. The results suggest that these regions play a role in the behavioral aspects of communicative speech production, presumably in understanding of the context of the social interaction. The activation of the polar part of the medial prefrontal cortex during the communication task was greater when the actor was familiar than when the actor was unfamiliar, suggesting that this region is involved in communicative speech production with reference to biological information. The precuneus was activated during the communication task only with the familiar actor, suggesting that this region is related to access to biographical information per se.

Is fundamental frequency a cue to aspiration in initial stops?

One production and one perception experiment were conducted to investigate the interaction of consonant voicing and fundamental frequency at the onset of voicing (onset f0) in Cantonese, a tonal language. Consonantal voicing in English can affect onset f0 up to 100 ms after voicing onset, but existing research provides inconclusive information regarding the effects of voicing on f0 in tonal languages where f0 variability is constrained by the demands of the lexical tone system. Previous research on consonantal effects on onset f0 provides two contrasting theories: These effects may be automatic, resulting from physiological constraints inherent to the speech production mechanism or they may be controlled, produced as part of a process of cue enhancement for the perception of laryngeal contrasts. Results of experiment 1 showed that consonant aspiration affects onset f0 in Cantonese only within the first 10 ms following voicing onset, comparable to results for other tonal languages. Experiment 2 showed that Cantonese listeners can use differences in onset f0 to cue perception of the voicing contrast, but the minimum extent of f0 perturbation necessary for this is greater than is found in Cantonese production, and comparable to that observed in acoustic studies of nontonal languages. These results suggest that consonantal effects on onset f0 are at least partially controlled by talkers, but that their role in the perception of voicing/aspiration may be a consequence of language independent properties of audition rather than listeners' experience with the phonological contrasts of a specific language.

Effects of transitory lingual nerve impairment on speech: an acoustic study of sibilant sound /s/

By measuring spectral characteristics of the sibilant /s/ this study investigated whether the reduced orosensory feedback caused by lingual nerve impairment affects the acoustics and articulation of sibilants. A further goal was to examine speakers’ capability to compensate for the deviant control of the delicate movements required for the proper production of /s/ by experimentally modifying the function of the tongue in a way that reduces the necessary somatosensory information in articulation. Five healthy men with no speech, language or hearing abnormalities were enrolled. They produced the sibilant /s/ in a variety of phonetic contexts in two sessions: first in normal conditions and then with local anaesthesia of the right lingual nerve. From the speech samples, the spectral characteristics of the sibilant sound (i.e. the centre of gravity, standard deviation, skewness and kurtosis) were analysed acoustically. The results showed that the reduced tactile sensation has effects on the tongue function resulting in individual and variable spectral alterations. The variation between different speakers indicates individual ability to compensate for the effects caused by the sensory dysfunction of the tongue. It seems, therefore, that the compensatory mechanisms for speech production are highly speaker-dependent.

Motor cortex maps articulatory features of speech sounds

The processing of spoken language has been attributed to areas in the superior temporal lobe, where speech stimuli elicit the greatest activation. However, neurobiological and psycholinguistic models have long postulated that knowledge about the articulatory features of individual phonemes has an important role in their perception and in speech comprehension. To probe the possible involvement of specific motor circuits in the speech-perception process, we used event-related functional MRI and presented experimental subjects with spoken syllables, including [p] and [t] sounds, which are produced by movements of the lips or tongue, respectively. Physically similar nonlinguistic signal-correlated noise patterns were used as control stimuli. In localizer experiments, subjects had to silently articulate the same syllables and, in a second task, move their lips or tongue. Speech perception most strongly activated superior temporal cortex. Crucially, however, distinct motor regions in the precentral gyrus sparked by articulatory movements of the lips and tongue were also differentially activated in a somatotopic manner when subjects listened to the lip- or tongue-related phonemes. This sound-related somatotopic activation in precentral gyrus shows that, during speech perception, specific motor circuits are recruited that reflect phonetic distinctive features of the speech sounds encountered, thus providing direct neuroimaging support for specific links between the phonological mechanisms for speech perception and production.

Speech and language processing mechanisms in verbal serial recall

We report two experiments examining the role of concreteness and word phonological neighborhood characteristics on immediate serial recall. In line with previous findings concreteness, word frequency, and larger neighborhood size are associated with better serial recall. Both concreteness and word neighborhood size were also positively associated with a measure of access to a speech production code from semantics (the speed and accuracy of providing a spoken word from its spoken definition). In contrast, concreteness had no effects on speech perception measures, and words from large neighborhoods were actually identified in speech with more difficulty than words from small neighborhoods. This pattern of results suggests that speech production mechanisms are more closely related to immediate serial recall performance than are speech perception mechanisms.

Blind speech separation using a joint model of speech production

We propose a new blind signal separation (BSS) technique, developed specifically for speech, that exploits a priori knowledge of speech production mechanisms. In our approach, the autoregressive (AR) structure and fundamental frequency (F0) production mechanisms of speech are jointly modeled. We compare the separation performance of our joint AR-F0 algorithm to existing BSS algorithms that model either speech's AR structure or F0 individually. Experimental results indicate that the joint algorithm demonstrates superior separation performance to both the individual AR algorithm (up to 77% improvement) and F0 (up to 50% improvement) algorithms. This suggests that speech separation performance is improved by employing a BSS model with a more realistic description of the speech production process.

Processing of reverberant speech for time-delay estimation

In this paper, we present a method of extracting the time-delay between speech signals collected at two microphone locations. Time-delay estimation from microphone outputs is the first step for many sound localization algorithms, and also for enhancement of speech. For time-delay estimation, speech signals are normally processed using short-time spectral information (either magnitude or phase or both). The spectral features are affected by degradations in speech caused by noise and reverberation. Features corresponding to the excitation source of the speech production mechanism are robust to such degradations. We show that these source features can be extracted reliably from the speech signal. The time-delay estimate can be obtained using the features extracted even from short segments (50-100 ms) of speech from a pair of microphones. The proposed method for time-delay estimation is found to perform better than the generalized cross-correlation (GCC) approach. A method for enhancement of speech is also proposed using the knowledge of the time-delay and the information of the excitation source.

Acoustic and spectral characteristics of young children’s fricative productions: A developmental perspective

Scientists have made great strides toward understanding the mechanisms of speech production and perception. However, the complex relationships between the acoustic structures of speech and the resulting psychological percepts have yet to be fully and adequately explained, especially in speech produced by younger children. Thus, this study examined the acoustic structure of voiceless fricatives (/f, theta, s, S/) produced by adults and typically developing children from 3 to 6 years of age in terms of multiple acoustic parameters (durations, normalized amplitude, spectral slope, and spectral moments). It was found that the acoustic parameters of spectral slope and variance (commonly excluded from previous studies of child speech) were important acoustic parameters in the differentiation and classification of the voiceless fricatives, with spectral variance being the only measure to separate all four places of articulation. It was further shown that the sibilant contrast between /s/ and /S/ was less distinguished in children than adults, characterized by a dramatic change in several spectral parameters at approximately five years of age. Discriminant analysis revealed evidence that classification models based on adult data were sensitive to these spectral differences in the five-year-old age group.

Spectral and prosodic transformations of hearing-impaired Mandarin speech

This paper studies the combined use of spectral and prosodic conversions to enhance the hearing-impaired Mandarin speech. The analysis-synthesis system is based on a sinusoidal representation of the speech production mechanism. By taking advantage of the tone structure in Mandarin speech, pitch contours are orthogonally transformed and applied within the sinusoidal framework to perform pitch modification. Also proposed is a time-scale modification algorithm that finds accurate alignments between hearing-impaired and normal utterances. Using the alignments, spectral conversion is performed on subsyllabic acoustic units by a continuous probabilistic transform based on a Gaussian mixture model. Results of perceptual evaluation indicate that the proposed system greatly improves the intelligibility and the naturalness of hearing-impaired Mandarin speech.

Integration of articulatory and spectrum features based on the hybrid HMM/BN modeling framework

Most of the current state-of-the-art speech recognition systems are based on speech signal parametrizations that crudely model the behavior of the human auditory system. However, little or no use is usually made of the knowledge on the human speech production system. A data-driven statistical approach to incorporate this knowledge into ASR would require a substantial amount of data, which are not widely available since their acquisition is difficult and expensive. Furthermore, during recognition, it is nearly impossible to obtain observations of articulators movement. Thus, research on speech production mechanisms in ASR has largely focused on modeling the hidden articulatory trajectories and using prior phonetic and phonological knowledge. Nevertheless, it has been shown that combining the acoustic and articulatory information can lead to improved speech recognition performance. The approach taken in this study is to integrate features extracted from actual articulatory data with acoustic MFCC features in a way that allows recognition using MFCC only. Rather than trying to map articulatory features to the corresponding acoustic features, we use the probabilistic dependency between them. Bayesian Networks (BN) are ideally suited for this purpose. They can model complex joint probability distributions with many discrete and continuous variables and have great flexibility in representing their dependencies. Our speech recognition system is based on the hybrid HMM/BN acoustic model where the BN is used to describe the HMM states’ probability distributions. HMM transitions, on the other hand, model the temporal speech characteristics. Articulatory and acoustic features are represented by different variables of the BN. Dependencies are learned from the observable articulatory and acoustic training data. During recognition, when only the acoustic observations are available, articulatory variables are assumed hidden. We have evaluated our ASR system by using a small database consisting of articulatory and acoustic data recorded from three speakers. The articulatory data are actual measurements of articulators position at several points. In all experiments involving both speaker-dependent and multi-speaker acoustic models, the HMM/BN system outperformed the baseline HMM system trained on acoustic data only. In experimenting with different BN topologies, we found that integrating the velocity and acceleration coefficients calculated as first and second derivatives of the articulatory position data can further improve recognition performance.

Is speech lazy or just efficient? A control-theoretic analysis

This paper introduces a control-theoretic model that allows us to address the energy dynamics of the vocal tract system. The model can be generated directly from articulatory data. The model allows computation of the energy in a state transfer, from an initial to a final articulator configuration. This method can help determine the degree of physical feasibility of various proposed articulatory trajectories. The basic assumption is that the set of articulators evolve through configurations that minimize the energy spent by the system to produce an utterance. Minimum control energy gives a measure of how hard it is to reach a target point for the different articulators. Simulation results are presented corresponding to the computation of the minimum energy from the MOCHA database. The linear model is shown to be adequate for short, well-labeled segments. The results show the intriguing fact that minimum control energy seems to have an oscillatory (swinging) nature for the production of speech. Physical features such as time constants and natural frequencies of the articulators are derived. A control-theoretic model of the dynamics of the mechanical articulators of speech production could be a fundamental tool to understand the mechanism of speech production.

Putting the interaction back into dialogue

I share the authors' stance on the dialogic or interactional character of language. The authors, however, have left actual interaction out of their conception of dialogue. I sketch a number of organizations of practices of talking and understanding that supply the basic arena for talk-in-interaction. It is by reference to these that mechanisms for speech production and understanding need to be understood.

Voices from the Past A Diachronic Perspective on Voicing

Diachronic phenomena of voicing and devoicing are explained here as resulting from cumulative effects of phonetic tendencies and preferences underlying speech production and perception. Regular diachronic change in voicing, either inferred from dialectal variation or manifested in written records, is the equalized result of sociolinguistic tuning processes in the transmission of the phonetic habits and perceptual feedback control from one generation of speakers to the other. Since phonation is not an autonomous component of the speech production mechanism, but dependent on both initiation and articulation, fluctuation in the execution of the phonation gesture is inherent to speech and thus governed not only by segmental context, but also by prosody. From the dynamic interaction of phonation with initiation and articulation, tendencies of change can in principle be predicted, but only in terms of probability: the implementation of sound change in a given historical scenario largely remains a matter of reselection within the spectrum of realisational variants, related to sociolinguistic processes of levelling and/or segregation that are hardly recoverable from what is usually known about speech communities of the past. On the one hand, synchronic variation with respect to voicing, especially in obstruents, can persist for ages without causing any change in the phonological system of the respective language or dialect. On the other hand, obstruent systems can be rearranged quite radically (in the course of so-called sound shifts) due to processes triggered by the antagonism of supraglottal constriction and transglottal airflow. Subsequent phase displacement of the phonation gesture can restore this antagonism, so that the system never reaches a state of rest. Diachronic tendencies of voicing and devoicing reflect the inherent dynamics of speech production, often to the detriment of perceptual redundancy.

Logarithmic temporal axis manipulation and its application for measuring auditory contributions in F0 control using a transformed auditory feedback procedure

A new parameter extraction procedure based on logarithmic transformation of the temporal axis was applied to investigate auditory effects on voice F0 control to overcome artifacts due to natural fluctuations and nonlinearities in speech production mechanisms. The proposed method may add complementary information to recent findings reported by using frequency shift feedback method [Burnett and Larson, J. Acoust. Soc. Am. 112 (2002)], in terms of dynamic aspects of F0 control. In a series of experiments, dependencies of system parameters in F0 control on subjects, F0 and style (musical expressions and speaking) were tested using six participants. They were three male and three female students specialized in musical education. They were asked to sustain a Japanese vowel /a/ for about 10 s repeatedly up to 2 min in total while hearing F0 modulated feedback speech, that was modulated using an M-sequence. The results replicated qualitatively the previous finding [Kawahara and Williams, Vocal Fold Physiology, (1995)] and provided more accurate estimates. Relations with designing an artificial singer also will be discussed. [Work partly supported by the grant in aids in scientific research (B) 14380165 and Wakayama University.]

A preprocessor for low-bit-rate speech coding

In this letter, we propose a preprocessor that modifies the input signal such that it can be coded more effectively in a low-bit-rate speech coder. Since most of the low-bit-rate speech coders are designed based on the human speech production mechanism, the perceived quality of the speech reconstructed in the decoder degrades seriously if the original input signal deviates from the pure speech. In order to alleviate this problem, we introduce a criterion that compromises the quantization error with the distortion incurred due to the modification. The coder-decoder characteristic is described in terms of a transfer matrix, and it is estimated according to the least squares criterion. The proposed approach is found to be effective in reducing audible distortions through a number of listening tests.

Perceptual analysis of speech following traumatic brain injury in childhood

Primary objective : To investigate perceptually the speech dimensions, oromotor function, and speech intelligibility of a group of individuals with traumatic brain injury (TBI) acquired in childhood. Research design : The speech of 24 children with TBI was analysed perceptually and compared with that of a group of non-neurologically impaired children matched for age and sex. Main outcome and results : The 16 dysarthric TBI subjects were significantly less intelligible than the control subjects, and demonstrated significant impairment in 12 of the 33 speech dimensions rated. In addition, the eight non-dysarthric TBI subjects were significantly impaired in many areas of oromotor function on the Frenchay Dysarthria Assessment, indicating some degree of pre-clinical speech impairment. Conclusion : The results of the perceptual analysis are discussed in terms of the possible underlying pathophysiological bases of the deviant speech features identified, and the need for a comprehensive instrumental assessment, to more accurately determine the level of breakdown in the speech production mechanism in children following TBI.

Effects of time pressure on mechanisms of speech production and self-monitoring.

The purpose of the present study was to examine effects of time pressure on mechanisms of speech production and self-monitoring. The most widely accepted monitoring theory (Levelt, 1989) suggests that monitoring proceeds through language perception, that is, speech error detection is primarily based on the parsing of one's own inner and overt speech. Twenty-four subjects described visual networks at two different rates (normal and fast). The time pressure manipulation affected a number of temporal characteristics: the error to cutoff and cutoff to repair times were shorter in the fast than in the normal condition. The results indicate that the monitor adjusts its speed of error detection and repair planning to the faster speech output rate. The time pressure manipulation did not affect the accuracy of error detection. The implications for the perception theory of monitoring are discussed.

The periodic structure of vowel sounds is reflected in human electromagnetic brain responses

Periodicity, which is caused by the vibration of the vocal folds, is an inherent feature of vowel sounds. Whether this periodic structure is reflected in cerebral processing of vowels was addressed via the use of non-invasive brain research methods combined with advanced stimulus production methodology. We removed the contribution of the source of the periodic structure, the glottal excitation produced by the vocal folds, from vowel stimuli and found that electromagnetic responses generated in the auditory cortex reflect this removal. The N1(m) amplitude decreased even though the rest of the acoustical features of the stimuli were identical. Thus, we conclude that speech production mechanisms have significant effects on human brain dynamics as reflected by magnetoencephalography and electroencephalograph.