Amodal System Research Articles

Musical Minds: Attentional blink reveals modality-specific restrictions

Background Formal musical training is known to have positive effects on attentional and executive functioning, processing speed, and working memory. Consequently, one may expect to find differences in the dynamics of temporal attention between musicians and non-musicians. Here we address the question whether that is indeed the case, and whether any beneficial effects of musical training on temporal attention are modality specific or generalize across sensory modalities. Methodology/Principal Findings When two targets are presented in close temporal succession, most people fail to report the second target, a phenomenon known as the attentional blink (AB). We measured and compared AB magnitude for musicians and non-musicians using auditory or visually presented letters and digits. Relative to non-musicians, the auditory AB was both attenuated and delayed in musicians, whereas the visual AB was larger. Non-musicians with a large auditory AB tended to show a large visual AB. However, neither a positive nor negative correlation was found in musicians, suggesting that at least in musicians, attentional restrictions within each modality are completely separate. Conclusion/Significance AB magnitude within one modality can generalize to another modality, but this turns out not to be the case for every individual. Formal musical training seems to have a domain-general, but modality-specific beneficial effect on selective attention. The results fit with the idea that a major source of attentional restriction as reflected in the AB lies in modality-specific, independent sensory systems rather than a central amodal system. The findings demonstrate that individual differences in AB magnitude can provide important information about the modular structure of human cognition.

Imagem mental e metáfora

This paper focuses on the nature of mental imagery as a component of human cognition. Imagery has been considered as mere epiphenomena and unable to alter belief. However, recent evidence in neuroscience and psycholinguistics suggest that the mental image can be accessed and interact with higher order processes of cognition in the determination of speaker meaning. If this is true, then mental imagery can play a role during the comprehension procedure for metaphoric utterances. To understand the reasons why mental images have been discarded from linguistic theories of utterance comprehension, past philosophical ideas about the mental image are discussed. Many have questioned the existence of the image, stating that they are, in fact, constituted by propositional components. These perspectives fit within a theory of the mind that conforms to an amodal symbol system in which the relation between perceptual input and mental representation is an arbitrary one. This paper is suggesting that the mind follows a perceptual symbol system, and that the relation is an analogous one. Therefore, through an extension of relevance theory, it is possible to flesh out the meaning of metaphors by incorporating this embodied view of the mind.

Three symbol ungrounding problems: Abstract concepts and the future of embodied cognition.

A great deal of research has focused on the question of whether or not concepts are embodied as a rule. Supporters of embodiment have pointed to studies that implicate affective and sensorimotor systems in cognitive tasks, while critics of embodiment have offered nonembodied explanations of these results and pointed to studies that implicate amodal systems. Abstract concepts have tended to be viewed as an important test case in this polemical debate. This essay argues that we need to move beyond a pretheoretical notion of abstraction. Against the background of current research and theory, abstract concepts do not pose a single, unified problem for embodied cognition but, instead, three distinct problems: the problem of generalization, the problem of flexibility, and the problem of disembodiment. Identifying these problems provides a conceptual framework for critically evaluating, and perhaps improving upon, recent theoretical proposals.

Musical minds: attentional blink reveals modality-specific restrictions.

BackgroundFormal musical training is known to have positive effects on attentional and executive functioning, processing speed, and working memory. Consequently, one may expect to find differences in the dynamics of temporal attention between musicians and non-musicians. Here we address the question whether that is indeed the case, and whether any beneficial effects of musical training on temporal attention are modality specific or generalize across sensory modalities.Methodology/Principal FindingsWhen two targets are presented in close temporal succession, most people fail to report the second target, a phenomenon known as the attentional blink (AB). We measured and compared AB magnitude for musicians and non-musicians using auditory or visually presented letters and digits. Relative to non-musicians, the auditory AB was both attenuated and delayed in musicians, whereas the visual AB was larger. Non-musicians with a large auditory AB tended to show a large visual AB. However, neither a positive nor negative correlation was found in musicians, suggesting that at least in musicians, attentional restrictions within each modality are completely separate.Conclusion/SignificanceAB magnitude within one modality can generalize to another modality, but this turns out not to be the case for every individual. Formal musical training seems to have a domain-general, but modality-specific beneficial effect on selective attention. The results fit with the idea that a major source of attentional restriction as reflected in the AB lies in modality-specific, independent sensory systems rather than a central amodal system. The findings demonstrate that individual differences in AB magnitude can provide important information about the modular structure of human cognition.

Sticking your neck out and burying the hatchet: what idioms reveal about embodied simulation.

Idioms are used in conventional language twice as frequently as metaphors, but most research, particularly recent work on embodiment has focused on the latter. However, idioms have the potential to significantly deepen our understanding of embodiment because their meanings cannot be derived from their component words. To determine whether sensorimotor states could activate idiomatic meaning, participants were instructed to engage in postures/actions reflecting various idioms (e.g., sticking your neck out) relative to non-idiomatic control postures/actions while reading and responding to statements designed to assess idiomatic meaning. The results showed that statements were generally more strongly endorsed after idiom embodiment than control conditions, indicating that the meaning of idiomatic expressions may not be as disconnected from perceptual and motor experiences than previously thought. These findings are discussed in terms of the mirror neuron system and the necessity of pluralistic contributions from both sensorimotor and amodal linguistic systems to fully account for the representation and processing of idioms and other figurative expressions.

Representational Similarity Analysis Reveals Commonalities and Differences in the Semantic Processing of Words and Objects

Understanding the meanings of words and objects requires the activation of underlying conceptual representations. Semantic representations are often assumed to be coded such that meaning is evoked regardless of the input modality. However, the extent to which meaning is coded in modality-independent or amodal systems remains controversial. We address this issue in a human fMRI study investigating the neural processing of concepts, presented separately as written words and pictures. Activation maps for each individual word and picture were used as input for searchlight-based multivoxel pattern analyses. Representational similarity analysis was used to identify regions correlating with low-level visual models of the words and objects and the semantic category structure common to both. Common semantic category effects for both modalities were found in a left-lateralized network, including left posterior middle temporal gyrus (LpMTG), left angular gyrus, and left intraparietal sulcus (LIPS), in addition to object- and word-specific semantic processing in ventral temporal cortex and more anterior MTG, respectively. To explore differences in representational content across regions and modalities, we developed novel data-driven analyses, based on k-means clustering of searchlight dissimilarity matrices and seeded correlation analysis. These revealed subtle differences in the representations in semantic-sensitive regions, with representations in LIPS being relatively invariant to stimulus modality and representations in LpMTG being uncorrelated across modality. These results suggest that, although both LpMTG and LIPS are involved in semantic processing, only the functional role of LIPS is the same regardless of the visual input, whereas the functional role of LpMTG differs for words and objects.

The effects of inter-stimulus interval and prime modality in a semantic priming task

Background: Semantic priming studies are employed in order to examine how various semantic contexts can influence visual word recognition processes. Although research has shown numerous factors can have an influence on the magnitude of the semantic priming effects found in lexical decision tasks, the majority of these factors have been related to the prime–target relationship itself. However, other factors have also been shown to alter the priming effect. Two such factors are the inter-stimulus interval (ISI) and presentation modality. The inter-stimulus interval may be used to independently assess automatic and strategic processes, whereas presentation modality is typically used to assess the different processing time courses that occur with spoken or written words. These factors have not been adequately investigated in the normal population. A more in-depth understanding of the relationships between prime modality, inter-stimulus interval, and word recognition processes in a sample of individuals with typical language abilities may provide valuable when examining lexical access and storage in language disordered populations such as those with aphasia. Aims: The focus of the present study was to examine the impact of relatively short and long inter-stimulus intervals on processing time in a visual and cross-modal lexical decision task. Previous research has not fully addressed whether convergent processes occur during cross-modal tasks or if an amodal semantic system exists. The utilisation of slow and fast inter-stimulus intervals should allow for a clearer distinction relative to processing. Methods & Procedures: A series of four lexical decision tasks was used to investigate reaction time and accuracy. The four tasks resulted from the combination of the independent variables ISI (0 ms or 400 ms) and prime modality (auditory or visual). Outcomes & Results: Results indicated that participants exhibited a larger priming effect when stimuli were presented in the 0 ms condition. Results also indicated that participants responded more accurately when the target word was presented auditorily. Conclusions: It was concluded that automatic spreading activation occurred in both the visual and auditory modalities, providing further evidence of an amodal semantic system. These observations are helpful in developing a clearer understanding of lexical storage and access in language impairments such as aphasia.

Restricted Attentional Capacity within but Not between Sensory Modalities: An Individual Differences Approach

BackgroundMost people show a remarkable deficit to report the second of two targets when presented in close temporal succession, reflecting an attentional blink (AB). An aspect of the AB that is often ignored is that there are large individual differences in the magnitude of the effect. Here we exploit these individual differences to address a long-standing question: does attention to a visual target come at a cost for attention to an auditory target (and vice versa)? More specifically, the goal of the current study was to investigate a) whether individuals with a large within-modality AB also show a large cross-modal AB, and b) whether individual differences in AB magnitude within different modalities correlate or are completely separate.Methodology/Principal FindingsWhile minimizing differential task difficulty and chances for a task-switch to occur, a significant AB was observed when targets were both presented within the auditory or visual modality, and a positive correlation was found between individual within-modality AB magnitudes. However, neither a cross-modal AB nor a correlation between cross-modal and within-modality AB magnitudes was found.Conclusion/SignificanceThe results provide strong evidence that a major source of attentional restriction must lie in modality-specific sensory systems rather than a central amodal system, effectively settling a long-standing debate. Individuals with a large within-modality AB may be especially committed or focused in their processing of the first target, and to some extent that tendency to focus could cross modalities, reflected in the within-modality correlation. However, what they are focusing (resource allocation, blocking of processing) is strictly within-modality as it only affects the second target on within-modality trials. The findings show that individual differences in AB magnitude can provide important information about the modular structure of human cognition.

The amodal system for conscious word and picture identification in the absence of a semantic task

Previous studies using explicit semantic tasks, such as category or similarity judgments, have revealed considerable neuroanatomical overlap between processing of the meaning of words and pictures. This result may have been influenced by the semantic executive control required by such tasks. We examined the degree of overlap while minimizing semantic executive demands. In a first fMRI experiment (n=28), we titrated word (35.3 ms, SD=9.6) and picture presentation duration (50.7 ms, SD=15.8) such that conscious stimulus identification became a stochastic process, with a 50% chance of success. Subjects had to indicate by key press whether or not they had been able to identify the stimulus. In a second fMRI experiment (n=19), the identification runs were followed by a surprise forced-choice recognition task and events were sorted on the basis of subsequent memory retrieval success rather than a subjective consciousness report. For both words and pictures, when stimulus processing exceeded the conscious identification threshold, the left occipitotemporal sulcus (OTS), intraparietal sulcus, inferior frontal junction, and middle third of the inferior frontal sulcus (IFS) were more active than when subjects had been unable to identify the stimulus. For both words and pictures, activity in two of these regions, IFS and OTS, predicted subsequent memory retrieval success. A Bayesian comparison revealed that the effective connectivity between IFS and the word- or picture-specific systems was mainly mediated via its connections with OTS. The amodal nature of left OTS and IFS involvement in word and picture processing extends to tasks with minimal semantic executive demands.

感動詞「エー」を表出した表情と音声に対するマルチモーダルな感情認知

Human communication of emotions is achieved through both facial and vocal information. The purpose of this study is to investigate the dominant sensory modality in recognition of emotions to the multi-modal expression. In Experiment 1, expressions of happiness, surprise, sadness, or aversion was presented vocally, facially, or in both modalities through the expression of an interjectory word “eh”. Participants were required to judge the emotion that was expressed. In Experiment 2, recognition of conflicted emotions between modalities was investigated by combining different emotions between facial and vocal expression. Results of the two experiments indicated that the observers predominantly recognized happiness and surprise that was expressed facially rather than vocally. Furthermore, the expression of happiness was often mistaken as surprise, and the expression of sadness was often mistaken as aversion. Importantly, however, the reverse of these mistakes was little observed. Such the asymmetries of confusion were consistently obtained in every modality including bimodal presentations. This evidence is suggestive that an amodal processing system exists in multi-modal recognition of emotions.

Multisensory numerosity judgments for visual and tactile stimuli

To date, numerosity judgments have been studied only under conditions of unimodal stimulus presentation. It is therefore unclear whether the same limitations on correctly reporting the number of unimodal visual or tactile stimuli presented in a display might be expected under conditions in which participants have to count stimuli presented simultaneously in two or more different sensory modalities. In Experiment 1, we investigated numerosity judgments using both unimodal and bimodal displays consisting of one to six vibrotactile stimuli (presented over the body surface) and one to six visual stimuli (seen on the body via mirror reflection). Participants had to count the number of stimuli regardless of their modality of presentation. Bimodal numerosity judgments were significantly less accurate than predicted on the basis of an independent modality-specific resources account, thus showing that numerosity judgments might rely on a unitary amodal system instead. The results of a second experiment demonstrated that divided attention costs could not account for the poor performance in the bimodal conditions of Experiment 1. We discuss these results in relation to current theories of cross-modal integration and to the cognitive resources and/or common higher order spatial representations possibly accessed by both visual and tactile stimuli.

Homeostasis and Religious Behaviour

AbstractThe classic dichotomy between bodily and cognitive processes has survived in scientific thought and can still be seen undiminished in the classic paradigm of the cognitive sciences. Lately, however, the notion of amodal cognitive systems has been challenged by new insights. More and more findings from neurobiological and perceptual studies support the assumption that cognitive processes solely rely on the brain's modal activities – even the most complex ones. Discarding amodal systems, however, would have profound effects not only on the cognitive sciences, but also on the broader study of human behaviour. During the last two decades, cognitivist theories have made their entrance into humanistic studies, including the study of religion. Thus, in discarding one of the basic premises of the cognitive paradigm, as I argue in this article, could lead to a radical rethinking of the science of religion. I will argue that the body and its sensory-motor modalities are not only the mind's representational base, but also encompass the intentionality behind human behaviour in general and religious behaviour in particular. I will call this intentionality homeostatic intentionality, as it has to do with the self-regulation and optimization of the organism. Ignoring physiological activities, including homeostatic ones, in the mind's complex cognitive processes is unfortunately a characteristic of classic cognitivist theories, and this negligence, it seems to me, rules out a satisfying and realistic account of religious behaviour.

Influence of a visual spatial attention task on auditory early and late Nd and P300

Singhal et al. [Psychophysiology 39 (2002) 236] reported that the amplitude of the late negative difference (Nd) and P300 components, generated by a dichotic listening task, were decreased in parallel by a simultaneously performed visual flying task. In contrast, early Nd was unaffected. To investigate the role of covert spatial attention in this pattern of results, we substituted Posner’s attention switching paradigm for the flying task. Fourteen subjects performed the dichotic listening and attention tasks alone and in combination. The results showed that only the invalid/valid condition of the Posner task reduced the amplitude of the late Nd, while the early Nd was unaffected. Contrary to the earlier study, P300 dissociated from the late Nd and its amplitude decreased in both the invalid/valid and neutral conditions of the Posner task. We conclude that the late Nd is sensitive to some aspect of attention switching and that the late Nd and P300 share sensitivity to an amodal system to which the early Nd is insensitive. Within this system, the late Nd is differentiated from P300 by its narrow sensitivity to attentional processing whereas P300 is sensitive to a broader range of processing. These similarities and differences are explained in the context of a modified version of Kok’s model [Psychophysiology 38 (2001) 557] of the determinants of P300 amplitude.

Auditory repetition priming is impaired in pure alexic patients

Alexia without agraphia, or “pure” alexia, is an acquired impairment in reading that leaves writing skills intact. Repetition priming for visually presented words is diminished in pure alexia. However, it is not possible to verify whether this priming deficit is modality-specific or modality independent because reading abilities are compromised. Hence, auditory repetition priming was assessed with lexical decision and word stem completion tasks in pure alexic patients with lesions in left inferior temporal–occipital cortex and the splenium. Perceptually based, modality-specific priming models predict intact auditory priming, since auditory association cortex is spared in the patients. Alternatively, modality-independent models, which suggest that priming reflects the temporary modification of an amodal system, might predict impairments. Baseline performance was matched in the patients and controls, although lexical decision priming measures showed an interaction between group and repetition lag. The patients showed intact immediate priming but significantly less priming than controls at longer delays. Furthermore, word stem completion priming was abolished in the patients. One explanation for the deficit is that left inferior temporal–occipital cortex supports amodal aspects of priming, as suggested by recent neuroimaging results. Another possibility is that long-term auditory priming relies on covert orthographic representations which were unavailable in the patients. The results provide support for interactive models of word identification.

Graded modality-specific specialisation in semantics: A computational account of optic aphasia

A long-standing debate regarding the representation of semantic knowledge is whether such knowledge is represented in a single, amodal system or whether it is organised into multiple subsystems based on modality of input or type of information. The current paper presents a distributed connectionist model of semantics that constitutes a middle ground between these unitary- versus multiple-semantics accounts. In the model, semantic representations develop under the pressure of learning to mediate between multiple input and output modalities in performing various tasks. The system has a topographic bias on learning that favours short connections, leading to a graded degree of modality-specific functional specialisation within semantics. The model is applied to the specific empirical phenomena of optic aphasia--a neuropsychological disorder in which patients exhibit a selective deficit in naming visually presented objects that is not attributable to more generalised impairments in object recognition (visual agnosia) or naming (anomia). As a result of the topographic bias in the model, as well as the relative degrees of systematicity among tasks, damage to connections from vision to regions of semantics near phonology impairs visual object naming far more than visual gesturing or tactile naming, as observed in optic aphasia. Moreover, as in optic aphasia, the system is better at generating the name of an action associated with an object than at generating the name of the object itself, because action naming receives interactive support from the activation of action representations. The ability of the model to account for the pattern of performance observed in optic aphasia across the full range of severity of impairment provides support for the claim that semantic representations exhibit graded functional specialisation rather than being entirely amodal or modality-specific.

EEG changes during word and figure categorization

Objective: To analyze whether the EEG changes observed during figure and word categorization are compatible with either the dual, the common amodal, or the alternative model (modality-specific codes for words and pictures, where meaning is represented for both in a higher-order amodal system) for semantic knowledge. Methods: EEG was recorded during word and figure categorization of animals or non-animals in a group of 28 children 8–10 years old. Computation of EEG sources in the frequency domain using variable resolution electrical tomography (VARETA) and their statistical evaluation by statistical parametric mapping were carried out. Results: At all frequencies, there were significant changes between EEG segments prior to the presentation of the stimuli and EEG segments recorded after the stimuli. Post-segments showed more power from 1.56 to 7.02 Hz, and less power than pre-segments from 8 to 12.48 Hz. EEG changes were only observed in the word task at: 3.9 (left occipital), 4.68, 5.46, and 6.24 Hz (temporo-occipital regions). These changes may be associated with visual encoding of words. Frequencies 7.8 and 17.94 Hz increased in prefrontal, anterior cingulate, and anterior temporal regions only during figure categorization. The prefrontal region may be related to object working memory. Thus, these frequencies might be related to figure codification. No significant differences between tasks were observed at 3.12 and 7.02 Hz in very wide brain areas (all lobes except occipital), suggesting that the amodal semantic system storage could be the model compatible with figure and word categorization. Conclusions: Thus, our results support the modified amodal semantic hypothesis, which advocates that the meanings of both kinds of stimuli are represented in a conceptual memory that receives input from the logogen and iconogen systems.

Creativity of metaphor in perceptual symbol systems

A metaphor can often create novel features in an object or a situation. This phenomenon has been particularly hard to account for using amodal symbol systems: although highlighting and downplaying can explain the shift of focus, it cannot explain how entirely new features can come about. We suggest here that the dynamism of perceptual symbol systems, particularly the notion of simulator, provides an elegant account of the creativity of metaphor. The elegance lies in the idea that the creation of new features by a metaphor proceeds in the same way as the creation of regular features in a perceptual symbol.

Perceptual symbols in language comprehension: Can an empirical case be made?

Perceptual symbol systems form a theoretically plausible alternative to amodal symbol systems. At this point it is unclear whether there is any truly diagnostic empirical evidence to decide between these systems. We outline some possible avenues of research in the domain of language comprehension that might yield such evidence. Language comprehension will be an important arena for tests of the two types of symbol systems.

Woorden en Objecten

The semantic system has a central position in the language processing system as the intermediate between language production and language comprehension. The system itself may be separated into distinctive components: visual and lexical semantics. Their is much discussion about the interference between the visual and lexical semantic system and about the quality of the processing routes. Some authors propose a unitary amodal system, other authors plead for modality-specific semantic systems. The stage of perceptual categorization is considered both as optional and as obligatory. The objective of this study is: to investigate visual and lexical semantic processing in aphasic patients (n=74) (control groups: right- hemispheric patients (n=10) and normals (n=96)) (i), to examine the relation between semantic deficits and aphasia type and severity (ii) and to explore the relation between presemantic and semantic visual processing. Instruments to measure presemantic and semantic processing: Object Decision (Riddoch & Humphreys, 1987) and the verbal and visual Semantic Association Test (Visch-Brink e.a.., 1993). Results: aphasie patients as a group were significantly impaired both in Object Decision and visual and verbal semantic processing. Some patients appeared to have a selective deficit in visual or verbal semantic processing (1). No correlation was found between the performance on the visual or verbal Semantic Association Test and the aphasia type or severity (ii). In some patients a dissociation was found between presemantic and semantic visual processing (iii). Conclusions: Visual and lexical semantic processing in aphasia may selectively disturbed, which pleads rather for a multiple than for a unitary semantic processing system. Aphasia type and the severity of aphasia do not function as indicators for the presence of a visual and/or lexical semantic disorder. For the interpretation of the meaning of a picture, the stage of perceptual categorization can be bypassed.