Auditory Word Processing Research Articles

Sensory stimulation induces tensor fields, which specifically transform brain activity

It was recently shown that brain activity can be represented as a stimulation-specific vector field. Since the vector field of brain activity is specifically transformed by sensory input, we suggested that a tensor field that transforms brain activity reflects sensory input. We calculated the tensor fields that transform brain activity between visual baseline and auditory word processing in PET data and between environmental sounds and auditory word processing in fMRI data. In the first comparison, significant clusters formed a distributed network over the brain cortex. In the second comparison, clusters were more localised in the temporo-frontal network of speech processing. Our study therefore demonstrated that tensor fields reflect the sensory input that specifically transforms brain activity.

Language non-selective activation of orthography during spoken word processing in Hindi–English sequential bilinguals: an eye tracking visual world study

Previous psycholinguistic studies have shown that bilinguals activate lexical items of both the languages during auditory and visual word processing. In this study we examined if Hindi–English bilinguals activate the orthographic forms of phonological neighbors of translation equivalents of the non target language while listening to words either in L1 or L2. We tracked participant’s eye movements as they looked at an array of written words that contained a phonological neighbor of the translation equivalent of a simultaneously presented spoken word. Participants quickly oriented their visual attention towards the phonological neighbor of the translation equivalent compared to the distractors, suggesting an activation of the spelling of the non-target lexicon via translation leading to further spreading activation of related words. Further, this parallel activation of the non target lexicon was seen in both L1–L2 and L2–L1 direction. These results suggest that different bilinguals can automatically activate the orthographic forms of the non-target lexicon via translation equivalents even when the languages in question do not share cognates and use different scripts.

Stable Modality-Specific Activity Flows As Reflected by the Neuroenergetic Approach to the fMRI Weighted Maps

This article uses the ideas of neuroenergetic and neural field theories to detect stimulation-driven energy flows in the brain during face and auditory word processing. In this analysis, energy flows are thought to create the stable gradients of the fMRI weighted summary images. The sources, from which activity spreads in the brain during face processing, were detected in the occipital cortex. The following direction of energy flows in the frontal cortex was described: the right inferior frontal = >the left inferior frontal = >the triangular part of the left inferior frontal cortex = >the left operculum. In the left operculum, a localized circuit was described. For auditory word processing, the sources of activity flows were detected bilaterally in the middle superior temporal regions, they were also detected in the left posterior superior temporal cortex. Thus, neuroenergetic assumptions may give a novel perspective for the analysis of neuroimaging data.

Schizophrenia, culture and neuropsychology: sensory deficits, language impairments and social functioning in Chinese-speaking schizophrenia patients

While 20% of schizophrenia patients worldwide speak tonal languages (e.g. Mandarin), studies are limited to Western-language patients. Western-language patients show tonal deficits that are related to impaired emotional processing of speech. However, language processing is minimally affected. In contrast, in Mandarin, syllables are voiced in one of four tones, with word meaning varying accordingly. We hypothesized that Mandarin-speaking schizophrenia patients would show impairments in underlying basic auditory processing that, unlike in Western groups, would relate to deficits in word recognition and social outcomes. Altogether, 22 Mandarin-speaking schizophrenia patients and 44 matched healthy participants were recruited from New York City. The auditory tasks were: (1) tone matching; (2) distorted tunes; (3) Chinese word discrimination; (4) Chinese word identification. Social outcomes were measured by marital status, employment and most recent employment status. Patients showed deficits in tone-matching, distorted tunes, word discrimination and word identification versus controls (all p<0.0001). Impairments in tone-matching across groups correlated with both word identification (p<0.0001) and discrimination (p<0.0001). On social outcomes, tonally impaired patients had 'lower-status' jobs overall when compared with tonally intact patients (p<0.005) and controls (p<0.0001). Our study is the first to investigate an interaction between neuropsychology and language among Mandarin-speaking schizophrenia patients. As predicted, patients were highly impaired in both tone and auditory word processing, with these two measures significantly correlated. Tonally impaired patients showed significantly worse employment-status function than tonally intact patients, suggesting a link between sensory impairment and employment status outcome. While neuropsychological deficits appear similar cross-culturally, their consequences may be language- and culture-dependent.

Neural Processing of What and Who Information in Speech

Human speech is composed of two types of information, related to content (lexical information, i.e., "what" is being said [e.g., words]) and to the speaker (indexical information, i.e., "who" is talking [e.g., voices]). The extent to which lexical versus indexical information is represented separately or integrally in the brain is unresolved. In the current experiment, we use short-term fMRI adaptation to address this issue. Participants performed a loudness judgment task during which single or multiple sets of words/pseudowords were repeated with single (repeat) or multiple talkers (speaker-change) conditions while BOLD responses were collected. As reflected by adaptation fMRI, the left posterior middle temporal gyrus, a crucial component of the ventral auditory stream performing sound-to-meaning computations ("what" pathway), showed sensitivity to lexical as well as indexical information. Previous studies have suggested that speaker information is abstracted during this stage of auditory word processing. Here, we demonstrate that indexical information is strongly coupled with word information. These findings are consistent with a plethora of behavioral results that have demonstrated that changes to speaker-related information can influence lexical processing.

Simultaneous interpreters as a model for neuronal adaptation in the domain of language processing

In the context of language processing, proficiency and age of acquisition have reliably been shown to have a strong influence on the functional and structural architecture of the human brain. The aim of the present EEG study was to examine the impact of language training as experienced by simultaneous interpreters (SI) on auditory word processing and to disentangle its influence from that of proficiency and age of acquisition. Eleven native German SI and controls matched in L2 proficiency and age of acquisition were asked to judge whether auditory presented disyllabic noun pairs both within and across the German (L1) and English (L2) languages were either semantically congruent or incongruent. We revealed enlarged N400 responses in SI while they detected incongruent trials both within the native (L1) and non-native (L2) language and also while they performed the task in the opposite direction as specifically trained (L1 to L2). These enlarged N400 responses in SI suggest a training-induced altered sensitivity to semantic processing within and across L1 and L2. The enlarged N400 responses we revealed in SI to congruent noun pairs during the German–English condition (L1 to L2) may indicate that SI could not benefit from an L1 prime when the target was a L2 word, suggesting additional processing resulting from long-term backwards (L2 to L1) training.

Electrophysiological evidence for incremental lexical-semantic integration in auditory compound comprehension

The present study investigated the time-course of semantic integration in auditory compound word processing. Compounding is a productive mechanism of word formation that is used frequently in many languages. Specifically, we examined whether semantic integration is incremental or is delayed until the head, the last constituent in German, is available. Stimuli were compounds consisting of three nouns, and the semantic plausibility of the second and the third constituent was manipulated independently (high vs. low). Participants’ task was to listen to the compounds and evaluate them semantically. Event-related brain potentials in response to the head constituents showed an increased N400 for less plausible head constituents, reflecting the lexical-semantic integration of all three compound constituents. In response to the second (less plausible) constituents, an increased N400 with a central-left scalp distribution was observed followed by a parietal positivity. The occurrence of this N400 effect during the presentation of the second constituents suggests that the initial two non-head constituents are immediately integrated. The subsequent positivity might be an instance of a P600 and is suggested to reflect the structural change of the initially constructed compound structure. The results suggest that lexical-semantic integration of compound constituents is an incremental process and, thus, challenge a recent proposal on the time-course of semantic processing in auditory compound comprehension.

Deep dysphasia as a phonetic input deficit: Evidence from a single case

Background: The syndrome of deep dysphasia is characterised by an inability to repeat pseudowords and the production of semantic errors in word repetition. Several single case studies revealed that phonological decoding might be outstandingly impaired. Recovery of deep dysphasia has only been illustrated in detail for patient NC (Martin & Saffran, 1992). Dell, Schwartz, Martin, Saffran, and Gagnon (1997) tried to simulate NC's repetition performance in their connectionist lexical activation model, but it did not fit his error pattern as it assumes perfect recognition of auditory input. With admiration, we dedicate this study to John Marshall who is one of the pioneers in discovering the difference between deep and surface processing of lexical information. We would like to thank Gary Dell with whom we discussed many details of the study. We also would like to thank an anonymous reviewer who helped us to revise and to extend the first version of the manuscript. Aims: In this new single case study on recovery of deep dysphasia, we intended to collect further evidence for the assumption that impaired input processing is the crucial cause of the impairment. Moreover, we aimed to explain impairment and psycholinguistic parameter effects in the connectionist semantic‐phonological model (Foygel & Dell, 2000) by adding a phonetic input level. Methods & Procedures: JR's performance was repeatedly assessed in the course of recovery. Errors in naming and repetition were classified according to the taxonomy of Dell et al. (1997). JR's error patterns were simulated in the semantic‐phonological model to determine the naming disorder and to predict word repetition. In addition, we established an error modality analysis to disentangle input and output impairments in repetition. Thus, the source of each error could be subclassified as belonging to either expressive or receptive components of repetition. Outcomes & Results: Initially there was a sharp contrast between severely impaired word and pseudoword repetition and almost unimpaired reading aloud. During recovery, performance in naming and word repetition improved a great deal, while repetition of pseudowords remained impossible. The evolvement of real word repetition was characterised by psycholinguistic parameter effects at different points in time: concreteness before length, before frequency. The connectionist model over‐predicted correct responses in word repetition as for NC. There were only few expressive repetition errors; regarding receptive errors, nonwords and null responses decreased significantly while formal errors became the dominant error type in the course of recovery. Conclusions: The development of psycholinguistic parameter effects, dissociations in performance, the computer simulations, and results from error modality analysis as well as changes of error pattern are ample evidence for the primary decoding disorder in JR. We argue that deep dysphasia can be explained by an impairment of phonetic–phonological connections in an extended version of the connectionist one‐route model of repetition with four rather than three levels of auditory word processing. The improved real word repetition despite persisting failure on pseudowords is accounted for by an increase of both phonetic–phonological and lexical–phonological connection weights.

How does the incrementality of auditory word perception interplay with episodic and semantic memory?

In contrast to visual word presentation, auditory words unfold over time and stimulus information reaches subsequent processing stages only incrementally. The influence of this incrementality was assessed in the present study. Target nouns were semantically primed by synonymous or non-synonymous preceding nouns and had to be categorized by speeded choice responses according to their synonymity. Long-term repetition was manipulated by presenting these prime-target pairs twice in two halves of the experiment and the effect of incrementality was assessed by sorting the target nouns according to three bins of word length. Event-related brain potentials (ERPs) were recorded to the target words. Both behavioural and ERP-results are in agreement with an incremental auditory word processing model, where long-term repetition and semantic priming affect separable processes, presumably episodic and semantic memory, respectively.

Cortical localisation of the visual and auditory word form areas: A reconsideration of the evidence

In this paper we examine the evidence for human brain areas dedicated to visual or auditory word form processing by comparing cortical activation for auditory word repetition, reading, picture naming, and environmental sound naming. Both reading and auditory word repetition activated left lateralised regions in the frontal operculum (Broca’s area), posterior superior temporal gyrus (Wernicke’s area), posterior inferior temporal cortex, and a region in the mid superior temporal sulcus relative to baseline conditions that controlled for sensory input and motor output processing. In addition, auditory word repetition increased activation in a lateral region of the left mid superior temporal gyrus but critically, this area is not specific to auditory word processing, it is also activated in response to environmental sounds. There were no reading specific activations, even in the areas previously claimed as visual word form areas: activations were either common to reading and auditory word repetition or common to reading and picture naming. We conclude that there is no current evidence for cortical sites dedicated to visual or auditory word form processing.

Electrophysiological estimates of the time course of semantic and phonological encoding during listening and naming

Current psycholinguistic models suggest that we know what we want to say before we decide how we are going to say it: in other words, for speaking, word meaning is activated prior to information about syntax and phonology. Listening likely involves the reverse order of processes: phonological processing before meaning activation. We examined the relative time courses of phonological and semantic processing during language production and comprehension using event-related brain potentials (ERPs). Participants viewed a series of pictures (with the instruction to covertly name the depicted item), or heard a series of words, and made dual choice Go/noGo decisions based on each item’s conceptual (whether the item was an animal or an object) and phonological features (whether the item’s German name started with a vowel or a consonant). During picture naming, the N200 component (related to response inhibition) indicated that conceptual processing preceded phonological processing by about 170 ms. During auditory word processing, on the other hand, the brain activity related to these two aspects of comprehension indicated some temporal overlap with the N200 to phonological processing preceding that to semantic processing by only about 85 ms. In sum, the data are compatible with current psycholinguistic models of speech production and comprehension and argue for serial or widely spaced cascaded processing during production but more parallel processing of information during comprehension.

The constraints functional neuroimaging places on classical models of auditory word processing.

Several previous functional imaging experiments have demonstrated that auditory presentation of speech, relative to tones or scrambled speech, activate the superior temporal sulci (STS) bilaterally. In this study, we attempted to segregate the neural responses to phonological, lexical, and semantic input by contrasting activation elicited by heard words, meaningless syllables, and environmental sounds. Inevitable differences between the duration and amplitude of each stimulus type were controlled with auditory noise bursts matched to each activation stimulus. Half the subjects were instructed to say "okay" in response to presentation of all stimuli. The other half repeated back the words and syllables, named the source of the sounds, and said "okay" to the control stimuli (noise bursts). We looked for stimulus effects that were consistent across task. The results revealed that central regions in the STS were equally responsive to speech (words and syllables) and familiar sounds, whereas the posterior and anterior regions of the left superior temporal gyrus were more active for speech. The effect of semantic input was small but revealed more activation in the inferior temporal cortex for words and familiar sounds than syllables and noise. In addition, words (relative to syllables, sounds, and noise) enhanced activation in the temporo-parietal areas that have previously been linked to modality independent semantic processing. Thus, in cognitive terms, we dissociate phonological (speech) and semantic responses and propose that word specificity arises from functional integration among shared phonological and semantic areas.

The Anatomy of Auditory Word Processing: Individual Variability

This study used functional magnetic resonance imaging (fMRI) to investigate the neural substrate underlying the processing of single words, comparing activation patterns across subjects and within individuals. In a word repetition task, subjects repeated single words aloud with instructions not to move their jaws. In a control condition involving reverse speech, subjects heard a digitally reversed speech token and said aloud the word “crime.” The averaged fMRI results showed activation in the left posterior temporal and inferior frontal regions and in the supplementary motor area, similar to previous PET studies. However, the individual subject data revealed variability in the location of the temporal and frontal activation. Although these results support previous imaging studies, demonstrating an averaged localization of auditory word processing in the posterior superior temporal gyrus (STG), they are more consistent with traditional neuropsychological data, which suggest both a typical posterior STG localization and substantial individual variability. By using careful head restraint and movement analysis and correction methods, the present study further demonstrates the feasibility of using overt articulation in fMRI experiments.

The anatomy of language: contributions from functional neuroimaging.

This article illustrates how functional neuroimaging can be used to test the validity of neurological and cognitive models of language. Three models of language are described: the 19th Century neurological model which describes both the anatomy and cognitive components of auditory and visual word processing, and 2 20th Century cognitive models that are not constrained by anatomy but emphasise 2 different routes to reading that are not present in the neurological model. A series of functional imaging studies are then presented which show that, as predicted by the 19th Century neurologists, auditory and visual word repetition engage the left posterior superior temporal and posterior inferior frontal cortices. More specifically, the roles Wernicke and Broca assigned to these regions lie respectively in the posterior superior temporal sulcus and the anterior insula. In addition, a region in the left posterior inferior temporal cortex is activated for word retrieval, thereby providing a second route to reading, as predicted by the 20th Century cognitive models. This region and its function may have been missed by the 19th Century neurologists because selective damage is rare. The angular gyrus, previously linked to the visual word form system, is shown to be part of a distributed semantic system that can be accessed by objects and faces as well as speech. Other components of the semantic system include several regions in the inferior and middle temporal lobes. From these functional imaging results, a new anatomically constrained model of word processing is proposed which reconciles the anatomical ambitions of the 19th Century neurologists and the cognitive finesse of the 20th Century cognitive models. The review focuses on single word processing and does not attempt to discuss how words are combined to generate sentences or how several languages are learned and interchanged. Progress in unravelling these and other related issues will depend on the integration of behavioural, computational and neurophysiological approaches, including neuroimaging.

Language functional neuro-imaging changes following focal left thalamic infarction.

The involvement of the left thalamus in language function has been largely demonstrated through the effects of thalamic lesion and/or stimulation upon language. However, the pathophysiological mechanisms underlying thalamic aphasias remain a matter of debate. We report here on changes in the pattern of brain activity in auditory word processing, verb generation and visual lexical decision in a patient who sustained left thalamic infarct following the surgery of intractable mesio-temporal epilepsy with left hippocampal sclerosis. To some extent, our findings exemplify the way the left thalamus brings on line the cortical networks involved in language processing, suggested in the term 'selective engagement'.

Hemispheric lateralization at different levels of human auditory word processing: a functional magnetic resonance imaging study

We used functional magnetic resonance imaging to disentangle the functional anatomy of brain systems involved in the processing of auditory word form and meaning. Three monitoring tasks on auditory stimuli, aimed at phonetic, lexical and semantic processing, were used. We found no lateralization of temporal lobe activations, when word processing was contrasted versus the complex phonetic task. Bilateral middle temporal activations (Brodmann Area [BA] 21) were attributed to processing of word-form. Areas specific to semantic processing were restricted to the left hemisphere: the posterior middle frontal (BA 9) and posterior parietal (BA 7/40) cortex, as well as an inferior temporal area (BA 20/21). Our data suggest, that left hemispheric dominance for auditory word comprehension occurred at the level of semantic processing.

Auditory and visual word processing studied with fMRI

Auditory and visual word processing studied with fMRI

Auditory and visual word processing studied with fMRI

Brain activations associated with semantic processing of visual and auditory words were investigated using functional magnetic resonance imaging (fMRI). For each form of word presentation, subjects performed two tasks: one semantic, and one nonsemantic. The semantic task was identical for both auditory and visual presentation: single words were presented and subjects determined whether the word was concrete or abstract. In the nonsemantic task for auditory words, subjects determined whether the word had one syllable or multiple syllables. In the nonsemantic task for visual words, subjects determined whether the word was presented in lower case or upper case. There was considerable overlap in where auditory and visual word semantic processing occurred. Visual and auditory semantic tasks both activated the left inferior frontal (BA 45), bilateral anterior prefrontal (BA 10, 46), and left premotor regions (BA 6) and anterior SMA (BA 6, 8). Left posterior temporal (middle temporal and fusiform gyrus) and predominantly right-sided cerebellar activations were observed during the auditory semantic task but were not above threshold during visual word presentation. The data, when averaged across subjects, did not show obligatory activation of left inferior frontal and temporal language areas during nonsemantic word tasks. Individual subjects showed differences in the activation of the inferior frontal region while performing the same task, even though they showed similar response latency and accuracy.

The Constraints Functional Anatomy places on Cognitive Models of Auditory Word Processing

The Constraints Functional Anatomy places on Cognitive Models of Auditory Word Processing

Different activation patterns in the visual cortex of late and congenitally blind subjects.

A key issue in developmental neuroscience is the role of activity-dependent mechanisms in the epigenetic induction of functional organization in visual cortex. Ocular blindness and ensuing visual deprivation is one of the rare models available for the investigation of experience-dependent cortical reorganization in man. In a PET study we demonstrate that congenitally blind subjects show task-specific activation of extrastriate visual areas and parietal association areas during Braille reading, compared with auditory word processing. In contrast, blind subjects who lost their sight after puberty show additional activation in the primary visual cortex with the same tasks. Studies in blind-raised monkeys show that crossmodal responses in extrastriate areas can be elicited by somatosensory stimulation. This is consistent with the crossmodal extrastriate activations elicited by tactile processing in our congenitally blind subjects. Since primary visual cortex does not show crossmodal responses in primate studies, the differential activation in late and congenitally blind subjects highlights the possibility of reciprocal activation by visual imagery in subjects with early visual experience.