Auditory Speech Processing Research Articles

Transfer Effect of Speech-sound Learning on Auditory-motor Processing of Perceived Vocal Pitch Errors.

Speech perception and production are intimately linked. There is evidence that speech motor learning results in changes to auditory processing of speech. Whether speech motor control benefits from perceptual learning in speech, however, remains unclear. This event-related potential study investigated whether speech-sound learning can modulate the processing of feedback errors during vocal pitch regulation. Mandarin speakers were trained to perceive five Thai lexical tones while learning to associate pictures with spoken words over 5 days. Before and after training, participants produced sustained vowel sounds while they heard their vocal pitch feedback unexpectedly perturbed. As compared to the pre-training session, the magnitude of vocal compensation significantly decreased for the control group, but remained consistent for the trained group at the post-training session. However, the trained group had smaller and faster N1 responses to pitch perturbations and exhibited enhanced P2 responses that correlated significantly with their learning performance. These findings indicate that the cortical processing of vocal pitch regulation can be shaped by learning new speech-sound associations, suggesting that perceptual learning in speech can produce transfer effects to facilitating the neural mechanisms underlying the online monitoring of auditory feedback regarding vocal production.

FALA NIEZGODNOŚCI I KOMPONENTA N2B WYWOŁANE PRZEZ DŹWIĘKI TONALNE I MOWĘ U OSÓB Z DYSNOMIĄ: STUDIUM PRZYPADKU

BackgroundThe auditory processing impairments frequently observed in aphasia are being slowly clarified by using eventrelated potentials (ERPs), a method that allows brain processes to be observed at high temporal resolution. Mismatch negativity (MMN) and the N2b amplitude reflect aspects of echoic memory, attention, and phonological representation. This study evaluates the auditory processing of speech and pure tones in an anomic aphasia subject 6 years after a stroke, and investigates whether ERPs can detect possible neurophysiologic sequelae after recovery and rehabilitation.Material and MethodsA recovered subject with anomic aphasia, 6 years post-stroke, was compared with 6 healthy controls. Event-related potentials (MMN, N1, N2b) were obtained during two auditory oddball paradigms, one using pure tones and the other consonant–vowel (CV) stimuli.ResultsWhen compared to healthy subjects, the anomic aphasia subject had reduced MMN amplitude across the frontocentral electrode sites, particularly for speech stimuli. Average deviant waveform analysis revealed poor morphology of N2b to speech stimuli, which might relate to deficits in phonological representation.ConclusionsIn the presented case the neurophysiologic brain activity for processing of phonologic representations had not fully recovered 6 years post-stroke. MMN and N2b are highly sensitive ERPs for evaluating impairments in auditory processing and can be registered in the absence of attention and with no task requirements, features which makes it particularly suitable for investigating aphasic subjects.

The Musician Redefined: A Behavioral Assessment of Rhythm Perception in Professional Club DJs

Studies of musical training demonstrate functional advantages in rhythm tasks that result from enriched auditory experience. Anatomical correlates exist in brain areas involved in auditory perception, speech processing, motor control, attention, and emotion. However, these studies fail to include many classes of musicians that might undergo experience-related change. The current study examined rhythm processing in professional disc jockeys (DJs) who routinely engage in temporally-demanding tasks during practice and performance. In Experiment 1, DJs outperformed controls at detecting a deviation in a rhythmic pattern, and were no different than trained percussionists. In Experiment 2, participants receiving one week of DJ training trended toward outperforming untrained participants on this same measure. Across experiments, movement improved detection of rhythmic deviations, providing evidence of privileged auditory-motor connections, and underscoring the importance of motor areas to rhythm perception. It is clear that DJs show experience-dependent changes in perception that are comparable to more traditional musicians.

A causal test of the motor theory of speech perception: a case of impaired speech production and spared speech perception

The debate about the causal role of the motor system in speech perception has been reignited by demonstrations that motor processes are engaged during the processing of speech sounds. Here, we evaluate which aspects of auditory speech processing are affected, and which are not, in a stroke patient with dysfunction of the speech motor system. We found that the patient showed a normal phonemic categorical boundary when discriminating two non-words that differ by a minimal pair (e.g., ADA–AGA). However, using the same stimuli, the patient was unable to identify or label the non-word stimuli (using a button-press response). A control task showed that he could identify speech sounds by speaker gender, ruling out a general labelling impairment. These data suggest that while the motor system is not causally involved in perception of the speech signal, it may be used when other cues (e.g., meaning, context) are not available.

Auditory Processing of Speech and Non-Speech Stimuli in Children who Stutter: Electrophysiological Evidences

Objectives: Current scientific evidence supports the hypothesis that people who stutter have anomalous connections in auditory regions of the left hemisphere. Thus, it is reasonable to suppose that abnormal results in auditory evoked potentials may be related to this type of disorder. In the present study, Auditory Brainstem Responses (ABR) using stimuli of different complexities were recorded in order to investigate possible neural synchrony deficits in children who stutter (CWS). Methods: Ten CWS aged between seven and 11 years and their non-stuttering peers (CWNS) underwent electrophysiological (speech- and click-evoked ABR) assessment. Results: CWS showed greater variability in latency values, as well as a statistical trend towards significance regarding differences between right and left ears for the interpeak I-III in the click-evoked ABR. In the speech-evoked ABR, the latency values of wave C and the amplitude of VA complex were significantly higher in CWS. Conclusions: The results suggest that CWS present differences in neural processes related to the processing of acoustic information, when compared to typically developing children, especially when more complex stimuli, such as speech, are considered.

Speech processing in children with functional articulation disorders

This study explored auditory speech processing and comprehension abilities in 5–8-year-old monolingual Hungarian children with functional articulation disorders (FADs) and their typically developing peers. Our main hypothesis was that children with FAD would show co-existing auditory speech processing disorders, with different levels of these skills depending on the nature of the receptive processes. The tasks included (i) sentence and non-word repetitions, (ii) non-word discrimination and (iii) sentence and story comprehension. Results suggest that the auditory speech processing of children with FAD is underdeveloped compared with that of typically developing children, and largely varies across task types. In addition, there are differences between children with FAD and controls in all age groups from 5 to 8 years. Our results have several clinical implications.

How visual timing and form information affect speech and non-speech processing

Auditory speech processing is facilitated when the talker’s face/head movements are seen. This effect is typically explained in terms of visual speech providing form and/or timing information. We determined the effect of both types of information on a speech/non-speech task (non-speech stimuli were spectrally rotated speech). All stimuli were presented paired with the talker’s static or moving face. Two types of moving face stimuli were used: full-face versions (both spoken form and timing information available) and modified face versions (only timing information provided by peri-oral motion available). The results showed that the peri-oral timing information facilitated response time for speech and non-speech stimuli compared to a static face. An additional facilitatory effect was found for full-face versions compared to the timing condition; this effect only occurred for speech stimuli. We propose the timing effect was due to cross-modal phase resetting; the form effect to cross-modal priming.

The sound of your lips: electrophysiological cross-modal interactions during hand-to-face and face-to-face speech perception.

Recent magneto-encephalographic and electro-encephalographic studies provide evidence for cross-modal integration during audio-visual and audio-haptic speech perception, with speech gestures viewed or felt from manual tactile contact with the speaker’s face. Given the temporal precedence of the haptic and visual signals on the acoustic signal in these studies, the observed modulation of N1/P2 auditory evoked responses during bimodal compared to unimodal speech perception suggest that relevant and predictive visual and haptic cues may facilitate auditory speech processing. To further investigate this hypothesis, auditory evoked potentials were here compared during auditory-only, audio-visual and audio-haptic speech perception in live dyadic interactions between a listener and a speaker. In line with previous studies, auditory evoked potentials were attenuated and speeded up during both audio-haptic and audio-visual compared to auditory speech perception. Importantly, the observed latency and amplitude reduction did not significantly depend on the degree of visual and haptic recognition of the speech targets. Altogether, these results further demonstrate cross-modal interactions between the auditory, visual and haptic speech signals. Although they do not contradict the hypothesis that visual and haptic sensory inputs convey predictive information with respect to the incoming auditory speech input, these results suggest that, at least in live conversational interactions, systematic conclusions on sensory predictability in bimodal speech integration have to be taken with caution, with the extraction of predictive cues likely depending on the variability of the speech stimuli.

Multidimensional scaling and order asymmetry of the acoustic change complex to voiceless fricatives in Polish, English, and Finnish listeners

The acoustic change complex (ACC) is a P1-N1-P2 onset response to changes between sounds measured using EEG, which when measured at a central location, is not thought to be affected by the language experience of the listener. Recent research has shown that with a pair of stimuli, there is an asymmetry between the ACC recorded when they are presented in one order compared to the other. This study used eight voiceless fricatives to measure the ACC to all possible pairs from EEG recordings of 15 English, Finnish, and Polish listeners. The ACC magnitude was used to perform multidimensional scaling to create a two dimensional perceptual space for each language that is driven by the spectral characteristics of the stimuli. Asymmetry occurred in all three languages depending on how peaky the first and second stimuli were. This asymmetry effect was greater for the Polish group, indicating that the Polish listeners' language experience increases their sensitivity to the spectral peakiness of fricatives. These results show that the ACC is in fact susceptible to native language effects, enhancing its application as a test for auditory speech processing.

Haptic and visual information speed up the neural processing of auditory speech in live dyadic interactions

Speech can be perceived not only by the ear and by the eye but also by the hand, with speech gestures felt from manual tactile contact with the speaker׳s face. In the present electro-encephalographic study, early cross-modal interactions were investigated by comparing auditory evoked potentials during auditory, audio–visual and audio–haptic speech perception in dyadic interactions between a listener and a speaker. In line with previous studies, early auditory evoked responses were attenuated and speeded up during audio–visual compared to auditory speech perception. Crucially, shortened latencies of early auditory evoked potentials were also observed during audio–haptic speech perception. Altogether, these results suggest early bimodal interactions during live face-to-face and hand-to-face speech perception in dyadic interactions.

Sensory-to-motor integration during auditory repetition: a combined fMRI and lesion study

The aim of this paper was to investigate the neurological underpinnings of auditory-to-motor translation during auditory repetition of unfamiliar pseudowords. We tested two different hypotheses. First we used functional magnetic resonance imaging in 25 healthy subjects to determine whether a functionally defined area in the left temporo-parietal junction (TPJ), referred to as Sylvian-parietal-temporal region (Spt), reflected the demands on auditory-to-motor integration during the repetition of pseudowords relative to a semantically mediated nonverbal sound-naming task. The experiment also allowed us to test alternative accounts of Spt function, namely that Spt is involved in subvocal articulation or auditory processing that can be driven either bottom-up or top-down. The results did not provide convincing evidence that activation increased in either Spt or any other cortical area when non-semantic auditory inputs were being translated into motor outputs. Instead, the results were most consistent with Spt responding to bottom up or top down auditory processing, independent of the demands on auditory-to-motor integration. Second, we investigated the lesion sites in eight patients who had selective difficulties repeating heard words but with preserved word comprehension, picture naming and verbal fluency (i.e., conduction aphasia). All eight patients had white-matter tract damage in the vicinity of the arcuate fasciculus and only one of the eight patients had additional damage to the Spt region, defined functionally in our fMRI data. Our results are therefore most consistent with the neurological tradition that emphasizes the importance of the arcuate fasciculus in the non-semantic integration of auditory and motor speech processing.

Brain Prediction of Auditory Emphasis by Facial Expressions During Audiovisual Continuous Speech

The visual cues involved in auditory speech processing are not restricted to information from lip movements but also include head or chin gestures and facial expressions such as eyebrow movements. The fact that visual gestures precede the auditory signal implicates that visual information may influence the auditory activity. As visual stimuli are very close in time to the auditory information for audiovisual syllables, the cortical response to them usually overlaps with that for the auditory stimulation; the neural dynamics underlying the visual facilitation for continuous speech therefore remain unclear. In this study, we used a three-word phrase to study continuous speech processing. We presented video clips with even (without emphasis) phrases as the frequent stimuli and with one word visually emphasized by the speaker as the non-frequent stimuli. Negativity in the resulting ERPs was detected after the start of the emphasizing articulatory movements but before the auditory stimulus, a finding that was confirmed by the statistical comparisons of the audiovisual and visual stimulation. No such negativity was present in the control visual-only condition. The propagation of this negativity was observed between the visual and fronto-temporal electrodes. Thus, in continuous speech, the visual modality evokes predictive coding for the auditory speech, which is analysed by the cerebral cortex in the context of the phrase even before the arrival of the corresponding auditory signal.

Effective cerebral connectivity during silent speech reading revealed by functional magnetic resonance imaging.

Seeing the articulatory gestures of the speaker (“speech reading”) enhances speech perception especially in noisy conditions. Recent neuroimaging studies tentatively suggest that speech reading activates speech motor system, which then influences superior-posterior temporal lobe auditory areas via an efference copy. Here, nineteen healthy volunteers were presented with silent videoclips of a person articulating Finnish vowels /a/, /i/ (non-targets), and /o/ (targets) during event-related functional magnetic resonance imaging (fMRI). Speech reading significantly activated visual cortex, posterior fusiform gyrus (pFG), posterior superior temporal gyrus and sulcus (pSTG/S), and the speech motor areas, including premotor cortex, parts of the inferior (IFG) and middle (MFG) frontal gyri extending into frontal polar (FP) structures, somatosensory areas, and supramarginal gyrus (SMG). Structural equation modelling (SEM) of these data suggested that information flows first from extrastriate visual cortex to pFS, and from there, in parallel, to pSTG/S and MFG/FP. From pSTG/S information flow continues to IFG or SMG and eventually somatosensory areas. Feedback connectivity was estimated to run from MFG/FP to IFG, and pSTG/S. The direct functional connection from pFG to MFG/FP and feedback connection from MFG/FP to pSTG/S and IFG support the hypothesis of prefrontal speech motor areas influencing auditory speech processing in pSTG/S via an efference copy.

Music therapy in stroke rehabilitation

Music therapy is currently used in the rehabilitation of many diseases. It is worth mentioning the possibility of using it in the rehabilitation of stroke-related. Music is a multimodal stimulus that activates many brain structures related to sensory processing, attention, and memory, and can stimulate complex cognition and multisensory integration. Stroke represents one of the most costly and long-term disabling conditions in adulthood worldwide and there is a need to determine the effectiveness of rehabilitation programs. The brain has a large capacity for automatic simultaneous processing and integration of sensory information. Musical training has extensive effects on the brain. One aspect that may be relevant for stroke rehabilitation is that musicians have enhanced subcortical auditory and audiovisual processing of speech and music. Listening to pleasant music activates an interconnected network of subcortical and cortical brain regions. Previous studies have shown rapid functional improvements associated with plastic brain changes due to musical performance, which involved the auditory and integrative a Music Supported Therapy and Melodic Intonation Therapy were proposed to induce plastic changes in the brain in terms of functional connectivity and neural reorganization in the sensorimotor cortex uditory-sensorimotor cortices instead of restricted motor cortical areas.

Visual speech form influences the speed of auditory speech processing

An important property of visual speech (movements of the lips and mouth) is that it generally begins before auditory speech. Research using brain-based paradigms has demonstrated that seeing visual speech speeds up the activation of the listener’s auditory cortex but it is not clear whether these observed neural processes link to behaviour. It was hypothesized that the very early portion of visual speech (occurring before auditory speech) will allow listeners to predict the following auditory event and so facilitate the speed of speech perception. This was tested in the current behavioural experiments. Further, we tested whether the salience of the visual speech played a role in this speech facilitation effect (Experiment 1). We also determined the relative contributions that visual form (what) and temporal (when) cues made (Experiment 2). The results showed that visual speech cues facilitated response times and that this was based on form rather than temporal cues.

Neural correlates of speech processing in prelingually deafened children and adolescents with cochlear implants.

Prelingually deafened children with cochlear implants stand a good chance of developing satisfactory speech performance. Nevertheless, their eventual language performance is highly variable and not fully explainable by the duration of deafness and hearing experience. In this study, two groups of cochlear implant users (CI groups) with very good basic hearing abilities but non-overlapping speech performance (very good or very bad speech performance) were matched according to hearing age and age at implantation. We assessed whether these CI groups differed with regard to their phoneme discrimination ability and auditory sensory memory capacity, as suggested by earlier studies. These functions were measured behaviorally and with the Mismatch Negativity (MMN). Phoneme discrimination ability was comparable in the CI group of good performers and matched healthy controls, which were both better than the bad performers. Source analyses revealed larger MMN activity (155–225 ms) in good than in bad performers, which was generated in the frontal cortex and positively correlated with measures of working memory. For the bad performers, this was followed by an increased activation of left temporal regions from 225 to 250 ms with a focus on the auditory cortex. These results indicate that the two CI groups developed different auditory speech processing strategies and stress the role of phonological functions of auditory sensory memory and the prefrontal cortex in positively developing speech perception and production.

Auditory Processing Following Consecutive Right Temporal Lobe Resections: A Prospective Case Study

The role of the right temporal lobe in processing speech is not well understood. Although the left temporal lobe has long been recognized as critical for speech perception, there is growing evidence for right hemisphere involvement. To investigate whether the right temporal lobe is critical for auditory speech processing, we studied prospectively a normal-hearing patient who underwent consecutive right temporal lobe resections for treatment of medically intractable seizures. To test the hypothesis that the right temporal lobe is critical for auditory speech processing. We used a prospective, repeated-measure, single-case design. Auditory processing was evaluated using behavioral tests of speech recognition (words, sentences) under multiple listening conditions (e.g., quiet, background noise, etc.). Auditory processing of nonspeech sounds was measured by pitch pattern sequencing and environmental sound recognition tasks. Repeat behavioral testing was performed at four time points over a 2 yr period: before and after consecutive right temporal lobe resection surgeries. Before surgery, the patient demonstrated normal speech recognition in quiet and under real-world listening conditions (background noise, filtered speech). After the initial right anterior temporal resection, speech recognition scores declined under adverse listening conditions, especially for the left ear, but remained largely within normal limits. Following resection of the right superior temporal gyrus 1 yr later, speech recognition in quiet and nonspeech sound processing (pitch patterns, environmental sounds) remained intact. However, speech recognition under adverse listening conditions was severely impaired. The right superior temporal gyrus appears to be critical for auditory processing of speech under real-world listening conditions.

Effects of hearing loss on the subcortical representation of speech cues

Individuals with sensorineural hearing loss often report frustration with speech being loud but not clear, especially in background noise. Despite advanced digital technology, hearing aid users may resort to removing their hearing aids in noisy environments due to the perception of excessive loudness. In an animal model, sensorineural hearing loss results in greater auditory nerve coding of the stimulus envelope, leading to a relative deficit of stimulus fine structure. Based on the hypothesis that brainstem encoding of the temporal envelope is greater in humans with sensorineural hearing loss, speech-evoked brainstem responses were recorded in normal hearing and hearing impaired age-matched groups of older adults. In the hearing impaired group, there was a disruption in the balance of envelope-to-fine structure representation compared to that of the normal hearing group. This imbalance may underlie the difficulty experienced by individuals with sensorineural hearing loss when trying to understand speech in background noise. This finding advances the understanding of the effects of sensorineural hearing loss on central auditory processing of speech in humans. Moreover, this finding has clinical potential for developing new amplification or implantation technologies, and in developing new training regimens to address this relative deficit of fine structure representation.

Multivoxel Patterns Reveal Functionally Differentiated Networks Underlying Auditory Feedback Processing of Speech

The everyday act of speaking involves the complex processes of speech motor control. An important component of control is monitoring, detection, and processing of errors when auditory feedback does not correspond to the intended motor gesture. Here we show, using fMRI and converging operations within a multivoxel pattern analysis framework, that this sensorimotor process is supported by functionally differentiated brain networks. During scanning, a real-time speech-tracking system was used to deliver two acoustically different types of distorted auditory feedback or unaltered feedback while human participants were vocalizing monosyllabic words, and to present the same auditory stimuli while participants were passively listening. Whole-brain analysis of neural-pattern similarity revealed three functional networks that were differentially sensitive to distorted auditory feedback during vocalization, compared with during passive listening. One network of regions appears to encode an "error signal" regardless of acoustic features of the error: this network, including right angular gyrus, right supplementary motor area, and bilateral cerebellum, yielded consistent neural patterns across acoustically different, distorted feedback types, only during articulation (not during passive listening). In contrast, a frontotemporal network appears sensitive to the speech features of auditory stimuli during passive listening; this preference for speech features was diminished when the same stimuli were presented as auditory concomitants of vocalization. A third network, showing a distinct functional pattern from the other two, appears to capture aspects of both neural response profiles. Together, our findings suggest that auditory feedback processing during speech motor control may rely on multiple, interactive, functionally differentiated neural systems.

Auditory–Motor Interactions for the Production of Native and Non-Native Speech

During speech production, auditory processing of self-generated speech is used to adjust subsequent articulations. The current study investigated how the proposed auditory-motor interactions are manifest at the neural level in native and non-native speakers of English who were overtly naming pictures of objects and reading their written names. Data were acquired with functional magnetic resonance imaging and analyzed with dynamic causal modeling. We found that (1) higher activity in articulatory regions caused activity in auditory regions to decrease (i.e., auditory suppression), and (2) higher activity in auditory regions caused activity in articulatory regions to increase (i.e., auditory feedback). In addition, we were able to demonstrate that (3) speaking in a non-native language involves more auditory feedback and less auditory suppression than speaking in a native language. The difference between native and non-native speakers was further supported by finding that, within non-native speakers, there was less auditory feedback for those with better verbal fluency. Consequently, the networks of more fluent non-native speakers looked more like those of native speakers. Together, these findings provide a foundation on which to explore auditory-motor interactions during speech production in other human populations, particularly those with speech difficulties.