Perceptual Processing Of Speech Research Articles

Brain activity during shadowing of audiovisual cocktail party speech, contributions of auditory–motor integration and selective attention

Selective listening to cocktail-party speech involves a network of auditory and inferior frontal cortical regions. However, cognitive and motor cortical regions are differentially activated depending on whether the task emphasizes semantic or phonological aspects of speech. Here we tested whether processing of cocktail-party speech differs when participants perform a shadowing (immediate speech repetition) task compared to an attentive listening task in the presence of irrelevant speech. Participants viewed audiovisual dialogues with concurrent distracting speech during functional imaging. Participants either attentively listened to the dialogue, overtly repeated (i.e., shadowed) attended speech, or performed visual or speech motor control tasks where they did not attend to speech and responses were not related to the speech input. Dialogues were presented with good or poor auditory and visual quality. As a novel result, we show that attentive processing of speech activated the same network of sensory and frontal regions during listening and shadowing. However, in the superior temporal gyrus (STG), peak activations during shadowing were posterior to those during listening, suggesting that an anterior–posterior distinction is present for motor vs. perceptual processing of speech already at the level of the auditory cortex. We also found that activations along the dorsal auditory processing stream were specifically associated with the shadowing task. These activations are likely to be due to complex interactions between perceptual, attention dependent speech processing and motor speech generation that matches the heard speech. Our results suggest that interactions between perceptual and motor processing of speech relies on a distributed network of temporal and motor regions rather than any specific anatomical landmark as suggested by some previous studies.

Consistency and variation in recognition of text and speech interrupted at variable rates

Recent research indicates that recognition of interrupted text can predict speech intelligibility under adverse listening conditions. However, factors underlying the relationship between perceptual processing of speech and text are not fully understood. We examined contributions of underlying linguistic and perceptual structure by comparing recognition of printed and spoken sentences interrupted at different rates (0.5—64 Hz) in 14 normal-hearing adults. The interruption method approximated deletion and retention of rate-specific linguistic information across the two modalities by substituting white space for silent intervals. Results indicate a remarkably similar U-shaped pattern of cross-rate variation for both modalities, with minima at 2 Hz. Nevertheless, at high and low interruption rates text recognition exceeded speech recognition, while the reverse trend was observed at middle rates. Surprisingly, no significant correlations were obtained in recognition accuracy between text and speech conditions. These findings indicate a high degree of perceptual constancy in recognition of interrupted text and speech, which may rely on retention of rate-specific linguistic and perceptual information retained after the interruptions. On the other hand, results also indicate rate-specific variation in perceptual processing of text and speech, which may potentially affect the degree to which recognition accuracy in one modality is predictive of the other.

Recognition memory for foreign- and native-accented sentences

The acoustic-phonetic properties of speech signals affect not only their intelligibility, but also how well they are encoded in memory. Recognition memory can be improved, for example, when speakers intentionally speak clearly (Van Engen et al., 2012). One explanation for this result is that enhanced acoustic-phonetic cues reduce the cognitive effort associated with perceptual speech processing, thereby increasing the availability of processing resources for encoding speech content in memory. In the current study, we tested the hypothesis that, following the same logic, recognition memory would be reduced for foreign-accented speech, in which acoustic-phonetic cues deviate from native-language norms. Participants heard English sentences produced by a native speaker of English and a native speaker of Korean. Recognition memory was tested using an old-new judgment task in which participants heard those recordings again, along with an equal number of new items. Surprisingly, recognition memory was higher for Korean-accented English than for native-accented English. A follow-up study in which test sentences were presented visually showed no difference between foreign- and native-accented speech. These results are consistent with an acoustic distinctiveness account in which foreign-accented speech can in some cases be remembered more accurately due to its distinct acoustic features.

Speech sounds alter facial skin sensation

Interactions between auditory and somatosensory information are relevant to the neural processing of speech since speech processes and certainly speech production involves both auditory information and inputs that arise from the muscles and tissues of the vocal tract. We previously demonstrated that somatosensory inputs associated with facial skin deformation alter the perceptual processing of speech sounds. We show here that the reverse is also true, that speech sounds alter the perception of facial somatosensory inputs. As a somatosensory task, we used a robotic device to create patterns of facial skin deformation that would normally accompany speech production. We found that the perception of the facial skin deformation was altered by speech sounds in a manner that reflects the way in which auditory and somatosensory effects are linked in speech production. The modulation of orofacial somatosensory processing by auditory inputs was specific to speech and likewise to facial skin deformation. Somatosensory judgments were not affected when the skin deformation was delivered to the forearm or palm or when the facial skin deformation accompanied nonspeech sounds. The perceptual modulation that we observed in conjunction with speech sounds shows that speech sounds specifically affect neural processing in the facial somatosensory system and suggest the involvement of the somatosensory system in both the production and perceptual processing of speech.

Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing

Speech and language are considered uniquely human abilities: animals have communication systems, but they do not match human linguistic skills in terms of recursive structure and combinatorial power. Yet, in evolution, spoken language must have emerged from neural mechanisms at least partially available in animals. In this paper, we will demonstrate how our understanding of speech perception, one important facet of language, has profited from findings and theory in nonhuman primate studies. Chief among these are physiological and anatomical studies showing that primate auditory cortex, across species, shows patterns of hierarchical structure, topographic mapping and streams of functional processing. We will identify roles for different cortical areas in the perceptual processing of speech and review functional imaging work in humans that bears on our understanding of how the brain decodes and monitors speech. A new model connects structures in the temporal, frontal and parietal lobes linking speech perception and production.

Perceptual learning and expectations: Cognitive mechanisms in speech recognition.

There is substantial plasticity in adult perceptual processing of speech that cannot be accounted for by most theories. Even 1 h of training on low-intelligibility synthetic speech can improve recognition by 20 percentage points for novel words. Whether speech perception is based on auditory properties or articulatory properties, few theories acknowledge the role of cognitive processes, and some explicitly exclude these mechanisms. Evidence will be examined suggesting perceptual learning of phonetic information in adult listeners involves processes such as attention and working memory. In addition, evidence will be examined that speech perception can be influenced, if not typically guided, by listeners’ expectations about speech. In some cases, expectations may be derived by explicit instruction to listeners, and in others, from contextual information. For example, research on talker normalization and perception of sinewave speech has demonstrated that listeners’ expectations about the speech signal change perceptual processing. Evidence will be discussed suggesting mechanisms that mediate this effect of expectations and compare this to mechanisms involved in perceptual learning. Perceptual learning may depend on the same kinds of cognitive mechanisms involved in the top-down guidance of perception by expectations and these mechanisms may be necessary for stable phonetic perception. [Research supported by NIDCD.]

Enhanced perceptual processing of speech in autism

Theories of autism have proposed that a bias towards low-level perceptual information, or a featural/surface-biased information-processing style, may compromise higher-level language processing in such individuals. Two experiments, utilizing linguistic stimuli with competing low-level/perceptual and high-level/semantic information, tested processing biases in children with autism and matched controls. Whereas children with autism exhibited superior perceptual processing of speech relative to controls, and showed no evidence of either a perceptual or semantic processing bias, controls showed a tendency to process speech semantically. The data provide partial support to the perceptual theories of autism. It is additionally proposed that the pattern of results may reflect different patterns of attentional focusing towards single or multiple stimulus cues in speech between children with autism and controls.

Dissociating speech perception and comprehension at reduced levels of awareness.

We used functional MRI and the anesthetic agent propofol to assess the relationship among neural responses to speech, successful comprehension, and conscious awareness. Volunteers were scanned while listening to sentences containing ambiguous words, matched sentences without ambiguous words, and signal-correlated noise (SCN). During three scanning sessions, participants were nonsedated (awake), lightly sedated (a slowed response to conversation), and deeply sedated (no conversational response, rousable by loud command). Bilateral temporal-lobe responses for sentences compared with signal-correlated noise were observed at all three levels of sedation, although prefrontal and premotor responses to speech were absent at the deepest level of sedation. Additional inferior frontal and posterior temporal responses to ambiguous sentences provide a neural correlate of semantic processes critical for comprehending sentences containing ambiguous words. However, this additional response was absent during light sedation, suggesting a marked impairment of sentence comprehension. A significant decline in postscan recognition memory for sentences also suggests that sedation impaired encoding of sentences into memory, with left inferior frontal and temporal lobe responses during light sedation predicting subsequent recognition memory. These findings suggest a graded degradation of cognitive function in response to sedation such that "higher-level" semantic and mnemonic processes can be impaired at relatively low levels of sedation, whereas perceptual processing of speech remains resilient even during deep sedation. These results have important implications for understanding the relationship between speech comprehension and awareness in the healthy brain in patients receiving sedation and in patients with disorders of consciousness.

Is the Linguistic Content of Speech Less Salient than its Perceptual Features in Autism?

Open-ended tasks are rarely used to investigate cognition in autism. No known studies have directly examined whether increased attention to the perceptual level of speech in autism might contribute to a reduced tendency to process language meaningfully. The present study investigated linguistic versus perceptual speech processing preferences. Children with autism and controls were tested on a quasi-open-format paradigm, in which speech stimuli contained competing linguistic and perceptual information, and could be processed at either level. Relative to controls, children with autism exhibited superior perceptual processing of speech. However, whilst their tendency to preferentially process linguistic rather than perceptual information was weaker than that of controls, it was nevertheless their primary processing mode. Implications for language acquisition in autism are discussed.

Knowing a word affects the fundamental perception of the sounds within it.

Understanding spoken language is an exceptional computational achievement of the human cognitive apparatus. Theories of how humans recognize spoken words fall into two categories: Some theories assume a fully bottom-up flow of information, in which successively more abstract representations are computed. Other theories, in contrast, assert that activation of a more abstract representation (e.g., a word) can affect the activation of smaller units (e.g., phonemes or syllables). The two experimental conditions reported here demonstrate the top-down influence of word representations on the activation of smaller perceptual units. The results show that perceptual processes are not strictly bottom-up: Computations at logically lower levels of processing are affected by computations at logically more abstract levels. These results constrain and inform theories of the architecture of human perceptual processing of speech.

Phonemic organization does not occur: Hence no feedback

I agree with Norris et al. that feedback to a phonemic level is never necessary, but disagree strongly with their reason why this is true. I believe the available evidence indicates that there is no feedback because there is no phonemic level employed in the perceptual processing of speech.

Perceptual speech processing and phonetic feature mapping for robust vowel recognition

We propose perceptual speech processing and phonetic feature mapping, which are inspired by the human auditory perceptual characteristics. The proposed perceptual speech processing is based on three perceptual characteristics and consists of three independent processing steps: masking effect, minimum audible field renormalization, and mel-scale resampling. They remove unperceptible spectral components, and adjust the magnitude and frequency scales of speech spectra, respectively. We apply these three processing steps to the speech spectrum sequentially to generate a new speech signal representation called the perceptual spectrum. For Mandarin vowel recognition, nine representative vowels are selected as references and similarity measures to these reference spectra, called phonetic features, are then generated from the perceptual spectrum. These phonetic features then serve as speech parameters in a continuous HMM-based recognition, stage. With these two techniques, a high recognition accuracy on Mandarin vowel phonemes has been achieved. Further experiments confirm that significant improvement on recognition robustness with respect to speaker variation and noise contamination can also obtained.

Cerebellar Contributions to the Perception of Temporal Cues within the Speech and Nonspeech Domain

A previous study (Ackermann, Gräber, Hertrich, & Daum, 1997) reported impaired phoneme identification in cerebellar disorders, provided that categorization depended on temporal cues. In order to further clarify the underlying mechanism of the observed deficit, the present study performed a discrimination and identification task in cerebellar patients using two-tone sequences of variable pause length. Cerebellar dysfunctions were found to compromise the discrimination of time intervals extending in duration from 10 to 150 ms, a range covering the length of acoustic speech segments. In contrast, categorization of the same stimuli as a “short” or “long pause” turned out to be unimpaired. These findings, along with the data of the previous investigation, indicate, first, that the cerebellum participates in the perceptual processing of speech and nonspeech stimuli and, second, that this organ might act as a back-up mechanism, extending the storage capacities of the “auditory analyzer” extracting temporal cues from acoustic signals.

Recognition of consonant clusters

Perceptual processing of speech has been frequently studied using analyses of consonant recognition errors. Typically, nonsense CV or VC syllables are used as stimulus materials. Nonsense consonant clusters constitute a class of sounds that are largely devoid of semantic context, yet they represent a higher level of phonological organization. Consequently, the recognition of these sounds may reflect processing levels that are not involved in the perception of simpler stimuli. In the present study, consonant clusters (CCV or CCCV in /a/ vowel context), as well as their singletone members, were presented to listeners against a background noise at 10, 5, and 0 dB S/N. The results show systematic trends in terms of the frequency and the patterns of recognition errors. In general, the proportion of substitutions, additions, and deletions resemble the pattern of errors observed in production errors of consonant clusters. These and further results will be presented and discussed at the meeting.

Speech production measured of speech perception: replications and extensions

In the early 1960's, Chistovitch and her colleagues used rapid shadowing (vocal reaction time) and manual reaction time to explore the perceptual processing of speech. Two ingriguing aspects of their data were the relatively fast vocal reaction times and the subjects' tendency to begin articulatory responses before all of the acoustic cues for consonants were presented. These results, among others, support the classic notion of an intimate link between perceptual analysis of speech signals and the mechanisms of control in speech production. The study to be reported replicates and extends some of Chistovitch's paradigms using modifications designed to eliminate possible sources of confounding. Results tend to support the conclusions of the Russian researchers and point to the additional ways in which the modified procedure might be used in exploring phonetic decoding. [Supported by NIH and the Sloan Foundation.]