Speech Perception Processing Research Articles

Preparation for speech perception and production functionally dissociates ventral, dorsal and articulatory pathways in the human brain

Introduction: Current thinking divides speech processing in speech perception and speech recognition. A dual-stream model of speech processing proposes a left-lateralized dorsal stream mapping acoustic speech signals to frontal articulatory networks aiding in speech perception. The bilaterally organized ventral stream is supposed to subserve speech recognition (Hickok and Poeppel, Nature Reviews Neuroscience 2007). Yet, overt articulation does not engage the dorsal stream, at least for overt reading (Kell et al., Brain 2009 and Kell et al., Cerebral Cortex 2010). We set out to disentangle the functional roles of the dorsal and ventral speech processing stream and their contribution to lateralization by comparing preparation for and execution of speech processing with speech production.

Some Behavioral and Neurobiological Constraints on Theories of Audiovisual Speech Integration: A Review and Suggestions for New Directions

Summerfield (1987) proposed several accounts of audiovisual speech perception, a field of research that has burgeoned in recent years. The proposed accounts included the integration of discrete phonetic features, vectors describing the values of independent acoustical and optical parameters, the filter function of the vocal tract, and articulatory dynamics of the vocal tract. The latter two accounts assume that the representations of audiovisual speech perception are based on abstract gestures, while the former two assume that the representations consist of symbolic or featural information obtained from visual and auditory modalities. Recent converging evidence from several different disciplines reveals that the general framework of Summerfield's feature-based theories should be expanded. An updated framework building upon the feature-based theories is presented. We propose a processing model arguing that auditory and visual brain circuits provide facilitatory information when the inputs are correctly timed, and that auditory and visual speech representations do not necessarily undergo translation into a common code during information processing. Future research on multisensory processing in speech perception should investigate the connections between auditory and visual brain regions, and utilize dynamic modeling tools to further understand the timing and information processing mechanisms involved in audiovisual speech integration.

Enhancement of Speech-Relevant Auditory Acuity in Absolute Pitch Possessors

Absolute pitch (AP) is the ability to identify the frequency or musical name of a specific tone, or to identify a tone without comparing it with any objective reference tone. While AP has recently been shown to be associated with morphological changes and neurophysiological adaptations in the planum temporale, a cortical area in the brain involved in speech perception processes, no behavioral evidence of speech-relevant auditory acuity in any AP possessors has hitherto been reported. In order to seek such evidence, in the present study, 15 professional musicians with AP and 14 without AP, all of whom had acquired Japanese as their first language, were asked to identify isolated Japanese syllables as quickly as possible after these syllables were presented auditorily. When the mean latency to the syllable identification was compared, it was significantly shorter in AP possessors than in non-AP possessors whether the presented syllables were those used as Japanese labels representing the 7 tones constituting an octave or not. The latency to hear the stimuli per se did not differ according to whether the participants were AP possessors or not. The results indicate the possibility that possessing AP provides one with extraordinarily enhanced acuity to individual syllables per se as fundamental units of a segmented word in the speech stream.

Quantitative Meta-Analysis on State and Trait Aspects of Auditory Verbal Hallucinations in Schizophrenia

Auditory verbal hallucinations (AVHs) have a high prevalence in schizophrenic patients. An array of studies have explored the neural correlates of AVHs by means of functional neuroimaging and have associated AVHs with diverse brain regions, some of which have been shown to be involved in speech generation, speech perception, and auditory stimulus processing. We divided these studies into "state" studies comparing periods of presence and absence of AVHs within-subject and "trait" studies comparing patients experiencing AVHs with patients without AVHs or healthy controls during tasks with verbal material. We set out to test the internal consistency and possible dissociations of the neural correlates of AVHs. We used activation likelihood estimation to perform quantitative meta-analyses on brain regions reported in state and trait studies on AVHs to assess significant concordance across studies. State studies were associated with activation in bilateral inferior frontal gyrus, bilateral postcentral gyrus, and left parietal operculum. Trait studies on the other hand showed convergence of decreases in hallucinating subjects in left superior temporal gyrus, left middle temporal gyrus, anterior cingulate cortex, and left premotor cortex activity. Based on the clear dissociation of brain regions that show convergence across state in comparison to trait studies, we conclude that the state of experiencing AVHs is primarily related brain regions that have been implicated in speech production ie, Broca's area, whereas the general trait that makes humans prone to AVHs seems to be related to brain areas involved in auditory stimuli processing and speech perception, ie, auditory cortex.

Temporal dynamics of sinusoidal and non‐sinusoidal amplitude modulation

Previous behavioural studies in human subjects have demonstrated the importance of amplitude modulations to the process of intelligible speech perception. In functional neuroimaging studies of amplitude modulation processing, the inherent assumption is that all sounds are decomposed into simple building blocks, i.e. sinusoidal modulations. The encoding of complex and dynamic stimuli is often modelled to be the linear addition of a number of sinusoidal modulations and so, by investigating the response of the cortex to sinusoidal modulation, an experimenter can probe the same mechanisms used to encode speech. The experiment described in this paper used magnetoencephalography to measure the auditory steady-state response produced by six sounds, all modulated in amplitude at the same frequency but which formed a continuum from sinusoidal to pulsatile modulation. Analysis of the evoked response shows that the magnitude of the envelope-following response is highly non-linear, with sinusoidal amplitude modulation producing the weakest steady-state response. Conversely, the phase of the steady-state response was related to the shape of the modulation waveform, with the sinusoidal amplitude modulation producing the shortest latency relative to the other stimuli. It is shown that a point in auditory cortex produces a strong envelope following response to all stimuli on the continuum, but the timing of this response is related to the shape of the modulation waveform. The results suggest that steady-state response characteristics are determined by features of the waveform outside of the modulation domain and that the use of purely sinusoidal amplitude modulations may be misleading, especially in the context of speech encoding.

Acoustic measurement of Canadian English and Canadian French interlingual homophones.

The purpose of this study was to examine how interlingual homophones of Canadian English (CE) and Canadian French (CF) differ along acoustic dimensions as a prelude to perceptual tests. Two male Canadian monolingual speakers of English and French produced sentences in which interlingual homophones were embedded (e.g., two /thu/ and /tu/ meaning all). Voiced onset times (VOTs), formant frequencies (F1 and F2), and vowel‐inherent spectral changes (VISCs) of each respective speaker were examined. First, it is expected that the CE monolingual speaker will have aspirated (long‐lag) voiceless stops, while the CF monolingual speaker will have unaspirated (short‐lag) voiceless stops regardless of the context. Second, it is expected that CF vowels will have lower F1 values for lax vowels and will be produced more peripherally (with respect to F2 values) than CE vowels (i.e., French front vowels will be more advanced, and French back vowels will be more posterior). Lastly, it is expected that CF vowels will have less vowel‐inherent spectral change than CE vowels. Results from this study will provide insight on the phonetic representation of interlingual homophones, enabling the understanding of acoustic‐phonetic influences on the processes of speech perception in bilinguals (CE and CF).

Auditory Spectral Integration in Nontraditional Speech Cues in Diotic and Dichotic Listening

Underlying auditory processes in speech perception were explored. Specifically of interest were the stages of auditory processing involved in the integration of dynamic information in nontraditional speech cues such as the virtual formant transitions. These signals utilize intensity ratio cues and changes in spectral center-of-gravity (instead of the actual formant frequency transitions) to produce perceived F3 glides. 6 men and 8 women (M age = 24.2 yr., SD = 2.1), recruited through posted materials from graduate students at The Ohio State University, participated in two experiments. The results for frequency-based formant transitions (Exp. 1) indicated that spectral cues to syllable identification are combined at more central levels of auditory processing. However, when the components of the virtual formant stimuli were divided between the ears in a dichotic listening task (Exp. 2), the results indicated that auditory spectral integration may occur above the auditory periphery but at stages more intermediate rather than central.

Effects of Early Auditory Experience on Word Learning and Speech Perception in Deaf Children With Cochlear Implants

That early word learning and speech perception skills have different sensitive periods and that very early implantation may affect later vocabulary outcomes more than speech perception outcomes. Several studies have found that deaf children who receive cochlear implants before 3 years of age tend to have better speech perception outcomes than children implanted later. Recent studies have not found age-at-implantation effects on speech perception or central auditory processing among children implanted younger than 2 years, suggesting that there may be a sensitive period for speech perception skills that closes by around 3 years of age. There has been very little work investigating possible sensitive periods for other language skills, such as the ability to learn words. Recent work suggests the possibility that the development of word-learning skills may have an earlier sensitive period than the development of speech perception skills. Assess speech perception and vocabulary outcomes in children implanted before 13 months of age and in children implanted between 16 and 23 months of age. Children implanted during the first year of life had better vocabulary outcomes than children implanted during the second year of life. However, earlier implanted children did not show better speech perception outcomes than later implanted children. There may be an earlier sensitive period for developing the ability to associate the sound patterns of words to their referents than for developing speech perception and central auditory processing skills.

Effects of Aging on The Processing of Speech Perception Examined by ERP

Effects of Aging on The Processing of Speech Perception Examined by ERP

Mapping of heard speech into articulation information and speech acquisition

Yuen et al. (1) find that hearing an “incongruent distractor” syllable shifts the tongue's place of articulation when pronouncing a second printed syllable. More specifically, hearing a /t/-initial phone in the distractor causes increased alveolar region tongue-palate contact when a subject seeks to pronounce a printed syllable with an initial /k/ or /s/. (The distractor/printed syllables were nonword stimuli such as /kib/ and /seeb/.) Yuen et al. (1) seek to interpret this effect as a speech phenomenon only in terms of potential speech perception processes. This interpretation overlooks the well-attested existence of a capability within the faculty of speech for overheard speech to be mapped directly and automatically into articulations independently of perception/comprehension.

Language-independent processing in speech perception: Identification of English intervocalic consonants by speakers of eight European languages

Processing speech in a non-native language requires listeners to cope with influences from their first language and to overcome the effects of limited exposure and experience. These factors may be particularly important when listening in adverse conditions. However, native listeners also suffer in noise, and the intelligibility of speech in noise clearly depends on factors which are independent of a listener’s first language. The current study explored the issue of language-independence by comparing the responses of eight listener groups differing in native language when confronted with the task of identifying English intervocalic consonants in three masker backgrounds, viz. stationary speech-shaped noise, temporally-modulated speech-shaped noise and competing English speech. The study analysed the effects of (i) noise type, (ii) speaker, (iii) vowel context, (iv) consonant, (v) phonetic feature classes, (vi) stress position, (vii) gender and (viii) stimulus onset relative to noise onset. A significant degree of similarity in the response to many of these factors was evident across all eight language groups, suggesting that acoustic and auditory considerations play a large rôle in determining intelligibility. Language-specific influences were observed in the rankings of individual consonants and in the masking effect of competing speech relative to speech-modulated noise.

Interactive processing in speech perception: The facts of feedback.

Context plays a critical role in human speech processing. Rapid, efficient integration of context and input is accomplished by interactive processing: bottom-up input information and top-down context information work together to constrain the percept. One domain where this is most clear is the effects of word context on perception of speech sounds. Behavioral experiments and computational model simulations show that interactive processing confers both benefits (robust perception in noisy environments and tuning in response to changes in input patterns) and costs (errors and delays in perception when the input is inconsistent with the context). Context effects are also prevalent is the constraining effect of global communication context on activation of different meanings of ambiguous words (i.e., homophones): when only highly imageable meanings are consistent with the context (a word-to-picture matching task), concrete noun meanings (e.g., the tree-related meaning of bark) become more activate than less imageable meanings (e.g., the dog-related meaning of bark). These findings are consistent with an interactive graded constraint satisfaction view of speech perception in which bottom-up input and top-down context simultaneously constrain the final percept.

Teaching and Learning Guide for: Speaking and Hearing Clearly: Talker and Listener Factors in Speaking Style Changes

Teaching and Learning Guide for: Speaking and Hearing Clearly: Talker and Listener Factors in Speaking Style Changes

Neural processing of asynchronous audiovisual speech perception

The temporal synchrony of auditory and visual signals is known to affect the perception of an external event, yet it is unclear what neural mechanisms underlie the influence of temporal synchrony on perception. Using parametrically varied levels of stimulus asynchrony in combination with BOLD fMRI, we identified two anatomically distinct subregions of multisensory superior temporal cortex (mSTC) that showed qualitatively distinct BOLD activation patterns. A synchrony-defined subregion of mSTC (synchronous>asynchronous) responded only when auditory and visual stimuli were synchronous, whereas a bimodal subregion of mSTC (auditory>baseline and visual>baseline) showed significant activation to all presentations, but showed monotonically increasing activation with increasing levels of asynchrony. The presence of two distinct activation patterns suggests that the two subregions of mSTC may rely on different neural mechanisms to integrate audiovisual sensory signals. An additional whole-brain analysis revealed a network of regions responding more with synchronous than asynchronous speech, including right mSTC, and bilateral superior colliculus, fusiform gyrus, lateral occipital cortex, and extrastriate visual cortex. The spatial location of individual mSTC ROIs was much more variable in the left than right hemisphere, suggesting that individual differences may contribute to the right lateralization of mSTC in a group SPM. These findings suggest that bilateral mSTC is composed of distinct multisensory subregions that integrate audiovisual speech signals through qualitatively different mechanisms, and may be differentially sensitive to stimulus properties including, but not limited to, temporal synchrony.

Are Changes in Pitch and Formants Enough to Influence Talker Normalization Processes in Children and Adults?

In the present study, we examined the influence of mean F0 and formant values on talker normalization. Initially, two speakers recorded an identical set of 10 isophonemic word lists in Hebrew, consisting of 10 words each. These recordings were then manipulated by means that affect F0 only, or both F0 and formant frequencies. Different degrees of manipulation were carried out in order to create a new set of lists which were perceived to have been obtained from multiple talkers. The original and manipulated lists were presented to a group of 12 adults and 12 children at 30 dB above pure-tone average thresholds (PTA). Half of the listeners was presented with the single-talker lists first and then the multiple-talker lists, and the other half was presented with the word lists in reversed order. Listeners were instructed to repeat the words they heard. Correct word recognition was scored. The results indicated: (1) lower word recognition scores for the multiple-talker lists than for the single-talker lists, for both children and adults, (2) larger inter-subject variability in performance for the multiple-talker compared to the single-talker lists for both age groups, and (3) that order of presentation influenced word recognition of the multiple-talker lists only, being worse when presented first. Our findings support the following: (1) manipulations of F0 and formants of a single talker are sufficient to influence talker normalization processes, (2) this influence varies between listeners suggesting that listeners do not use the same acoustic information in their speech perception process, and (3) even adults, who are proficient in the language, do not eliminate entirely the irrelevant talker-specific acoustic information in the speech perception process.

Lexicality drives audio-motor transformations in Broca’s area

Broca’s area is classically associated with speech production. Recently, Broca’s area has also been implicated in speech perception and non-linguistic information processing. With respect to the latter function, Broca’s area is considered to be a central area in a network constituting the human mirror system, which maps observed or heard actions onto motor programs to execute analogous actions. These mechanisms share some similarities with Liberman’s motor theory, where objects of speech perception correspond to listener’s intended articulatory gestures. The aim of the current series of behavioral, TMS and fMRI studies was to test if Broca’s area is indeed implicated in such audio-motor transformations. More specifically, using a classical phonological rhyme priming paradigm, we investigated whether the role of Broca’s area could be purely phonological or rather, is lexical in nature. In the behavioral baseline study, we found a large priming effect in word prime/target pairs (W–W) and no effect for pseudo-words (PW–PW). Online TMS interference of Broca’s area canceled the priming difference between W–W and PW–PW by enhancing the effects for PW–PW. Finally, the fMRI study showed activation of Broca’s area for W–W pairs, but not for PW–PW pairs. Our data show that Broca’s area plays a significant role in speech perception strongly linked to the lexicality of a stimulus.

Perceptual learning of co-articulation in speech

Four experiments investigated the novel issue of learning to accommodate the co-articulated nature of speech. Experiment 1 established a co-articulatory mismatch effect for a set of vowel–consonant (VC) syllables (reaction times were faster for co-articulation matching than for mismatching stimuli). A rhyme judgment training task on words (Experiment 2) or VC stimuli (Experiment 3) with mismatching information was followed by a phoneme monitoring task on a set of VC stimuli; training and test stimuli contained physically identical (same condition) or new (different condition) mismatching co-articulatory information (along with a set containing matching co-articulatory information). A third group received no training. A co-articulatory mismatch effect was found without training but not when the same mismatching tokens were used at training and test. Both word (Experiment 2) and syllable (Experiment 3) training stimuli eliminated the mismatch effect; overall reaction times were somewhat slower when the training stimuli were words. Perceptual learning generalized to new tokens only when the acoustic manifestation of the critical co-articulatory information in the training stimuli was sufficiently large (Experiments 3 and 4). The results are discussed in terms of speech processing and perceptual learning in speech perception.

Spatiotemporal Convergence of Semantic Processing in Reading and Speech Perception

Retrieval of word meaning from the semantic system and its integration with context are often assumed to be shared by spoken and written words. How is modality-independent semantic processing manifested in the brain, spatially and temporally? Time-sensitive neuroimaging allows tracking of neural activation sequences. Use of semantically related versus unrelated word pairs or sentences ending with a semantically highly or less plausible word, in separate studies of the auditory and visual modality, has associated lexical-semantic analysis with sustained activation at approximately 200-800 ms. Magnetoencephalography (MEG) studies have further identified the superior temporal cortex as a main locus of the semantic effect. Nevertheless, a direct comparison of the spatiotemporal neural correlates of visual and auditory word comprehension in the same brain is lacking. We used MEG to compare lexical-semantic analysis in the visual and auditory domain in the same individuals, and contrasted it with phonological analysis that, according to models of language perception, should occur at a different time with respect to semantic analysis in reading and speech perception. The stimuli were lists of four words that were either semantically or phonologically related, or with the final word unrelated to the preceding context. Superior temporal activation reflecting semantic processing occurred similarly in the two modalities, left-lateralized at 300-450 ms and thereafter bilaterally, generated in close-by areas. Effect of phonology preceded the semantic effect in speech perception but not in reading. The present data indicate involvement of the middle superior temporal cortex in semantic processing from approximately 300 ms onwards, regardless of input modality.

Absolute Pitch--Functional Evidence of Speech-Relevant Auditory Acuity

Absolute pitch (AP) has been shown to be associated with morphological changes and neurophysiological adaptations in the planum temporale, a cortical area involved in higher-order auditory and speech perception processes. The direct link between speech processing and AP has hitherto not been addressed. We provide first evidence that AP compared with relative pitch (RP) ability is associated with significantly different hemodynamic responses to complex speech sounds. By systematically varying the lexical and/or prosodic information of speech stimuli, we demonstrated consistent activation differences in AP musicians compared with RP musicians and nonmusicians. These differences relate to stronger activations in the posterior part of the middle temporal gyrus and weaker activations in the anterior mid-part of the superior temporal gyrus. Furthermore, this pattern is considerably modulated by the auditory acuity of AP. Our results suggest that the neural underpinnings of pitch processing expertise exercise a strong influence on propositional speech perception (sentence meaning).

Speech perception and cochlear signal processing [Life Sciences

Speech sounds are encoded by time-varying spectral patterns called acoustic cues. The processing and detection of these acoustic cues lead to events defined as the psychological correlates of the acoustic cues. Due to the similarity between the acoustic cues, speech sounds form natural confusion groups. When the feature of the sound within a group is masked by noise, one event can turn into another. A systematic psychoacoustic has been developed to explore the perceptual cues of stop consonants from naturally produced speech sounds. For each sound, our 3-D method measures the contribution of each subcomponent by time-truncating, high-pass/low-pass filtering, and masking with noise. The Al-gram, a visualization tool that simulates the auditory peripheral processing, is used to predict the audible components of the speech sound. The results are that the plosive consonants are defined by a short duration bursts characterized by their center frequency, as well as the delay to the onset of voicing. Fricatives are characterized by the duration and bandwidth of a noise-like feature. Pilot studies of hearing-impaired (HI) speech perception indicate that cochlear dead regions have a considerable impact on consonant identification. An HI listener may have problems understanding speech simply because he/she cannot hear certain sounds, since the events are missing due to either the hearing loss, or the masking effect introduced by the noise.