Cross-modal Priming Paradigm Research Articles

Neural processing of Mandarin and Cantonese lexical tone alternations: implications for the nature of phonological representations

This study provides neurocognitive evidence to shed further insight into the architecture of phonological representations. We tap into these representations by focusing on the neural processing of phonological alternations. Traditional generative accounts postulate abstract representations which are transformed by phonological processes into different alternants. Recent psycholinguistically-based frameworks posit that different alternants are listed in parallel in the abstract representation. Exemplar-based theories posit that alternation is a manifestation of frequency effects among concrete phonetic forms stored in memory. However, it is unclear the extent to which phonological representations of different levels of abstractedness are neurocognitive realities, beyond formal entities for linguistic analysis. This study tests the hypothesis that phonological representations are neurocognitive primitives that modulate speech processing and lexical access. We examined the processing of two surface-similar, but distributionally distinct lexical tone alternation patterns in Mandarin and Cantonese. With a cross-modal priming paradigm, differential neurophysiological components (LPC vs. N400) associated with the processing of alternation violations were identified cross-linguistically. Results support our hypothesis, suggesting that cross-linguistically distinct abstract phonological representations differentially modulate phonotactic detection, lexical access, and phonological restructuring processes. Results also lend theoretical insight into the multiform nature of the phonological representation, which encompasses both abstract and concrete levels of representations.

The Processing of Lexical Ambiguity: Evidence from Child and Adult Greek

The aim of the study is to examine the effect of sentential context on lexical ambiguity resolution in Greek adults and typically developing children. Context and word frequency are factors that can affect lexical processing, however, the role of them has not been thoroughly examined in Greek. To this aim, we assessed sentence context effects in homonym meaning activation in monolingual speakers of Greek, children and adults, using a cross-modal priming paradigm. Additionally, all participants conducted a verbal working memory task and an inhibition task so as to examine whether the use of sentential context for lexical ambiguity resolution relates to age and/or cognitive processing capacity. The data analysis showed (a) major processing differences between adults and children due to ambiguity and sentential context, (b) children’s processing times affected by cognitive skills while adults’ processing unaffected, and (c) visual word recognition intact for all participants.

The Effect of Face-Voice Gender Consistency on Impression Evaluation.

Face and voice are important information cues of interpersonal interaction. Most previous studies have investigated the cross-modal perception of face and voice from the perspective of cognitive psychology, but few empirical studies have focused on the effect of gender consistency of face and voice on the impression evaluation of the target from the perspective of social cognition. Based on the two-stage model of stereotype activation and the stereotype content model, this research examined the effects of face-voice gender consistency on impression evaluation (gender categorization and warmth competence evaluation) by using a cross-modal priming paradigm (Study 1, 20 males and 23 females, Mage = 21.00, SDage = 2.59), a sequential presentation task (Study 2a, 57 males and 70 females, Mage = 18.54, SDage = 1.54; Study 2b, 52 males and 51 females, Mage = 18.54, SDage = 1.36), and a simultaneous presentation task (Study 3, 51 males and 55 females, Mage = 23.58, SDage = 3.20), respectively. The results showed that: (1) there was a face-voice gender consistency preference in gender categorization, and the response of face-voice consistent condition was faster than that of inconsistent condition; (2) compared with the face-voice gender-inconsistent individuals, the participants showed a higher and more stable evaluation of the warmth and competence of the gender-consistent individuals, indicating the effect of matching preference of the face-voice gender consistency on the impression evaluation; (3) people paid more attention to the gender information of faces in the impression evaluation, and the female face could improve people's evaluation on the target's warmth and competence; (4) males were more intolerant of face-voice gender inconsistency when presented sequentially; the "voice needs to match face" effect was stronger for females when presented simultaneously. These findings, on the one hand, enrich and expand previous theories and research on cross-modal processing of face and voice from the perspective of social cognitive impression evaluation; on the other hand, these findings have important practical implications for impression management and decision-making in social interaction.

The online processing of the ambiguous null object in Mandarin among patients with Broca’s aphasia

ABSTRACT Background: Research on how patients with Broca’s aphasia and concomitant agrammatism process dependency structures online offers a window into the understanding of how language is organized in the mind. Previous studies have shown that patients evince differential performances on dependency structures generated at different levels of linguistic representation. The present study extends this line of research to investigate patients’ online processing of referential dependencies involving ambiguous pronominal interpretations. Aims: Mandarin, unlike English, allows the object to be missing (null) in some contexts, rendering these null pronominal forms as ambiguous between two interpretations (strict vs. sloppy), which are generated in different levels of linguistic representation. This study examined Mandarin-speaking healthy and agrammatic aphasic participants’ processing of the two interpretations of the null pronominal form such as Zhangsan qile ta de ma, Lisi ye qile e “literally: Zhangsan rode his horse and Lisi rode, too”, where Lisi rode either Zhangsan’s horse (strict) or Lisi’s horse (sloppy). Methods & Procedures: Two experiments, employing a Cross-Modal Picture Priming paradigm, manipulated three probe picture conditions (strict, sloppy, control) and two probe positions (at the verb phrase offset and 400 ms afterwards). Experiment 1 tested neurologically unimpaired controls in order to establish the time course of processing as a standard of comparison. Experiment 2 tested patients with agrammatic aphasia. Outcomes and Results: Immediately at the verb phrase offset where the null object is structurally licensed, unimpaired individuals primed both the strict and the sloppy interpretations, with no significant difference in reaction times between the two conditions. At the position 400 ms after the verb phrase offset; however, unimpaired participants did not show priming of either of the two interpretations. Patients showed the opposite pattern: at verb phrase offset, they did not show threshold-level priming of either of the two interpretations; 400 ms afterwards, they showed priming of both the strict and sloppy interpretations, with the strict interpretation eliciting significantly longer reaction times than the sloppy interpretation. Conclusions: Patients’ delay in priming both the strict and the sloppy interpretations shows that patients evince a temporal delay when processing dependency structures, which may arise from patients’ deficit with lexical integration. Additionally, patients spent significantly longer reaction times on the strict interpretation relative to the sloppy interpretation, which confirms the prediction that that the former, which is generated in the level of discourse, is computationally more costly than the latter, which is generated in the level of semantics.

Do infants represent human actions cross-modally? An ERP visual-auditory priming study

Recent findings indicate that 7-months-old infants perceive and represent the sounds inherent to moving human bodies. However, it is not known whether infants integrate auditory and visual information in representations of specific human actions. To address this issue, we used ERPs to investigate infants’ neural sensitivity to the correspondence between sounds and images of human actions. In a cross-modal priming paradigm, 7-months-olds were presented with the sounds generated by two types of human body movement, walking and handclapping, after watching the kinematics of those actions in either a congruent or incongruent manner. ERPs recorded from frontal, central and parietal electrodes in response to action sounds indicate that 7-months-old infants perceptually link the visual and auditory cues of human actions. However, at this age these percepts do not seem to be integrated in cognitive multimodal representations of human actions.

Differential Neurobehavioral Effects of Cross-Modal Selective Priming on Phonetic and Emotional Prosodic Information in Late Second Language Learners.

Purpose Spoken language is inherently multimodal and multidimensional in natural settings, but very little is known about how second language (L2) learners undertake multilayered speech signals with both phonetic and affective cues. This study investigated how late L2 learners undertake parallel processing of linguistic and affective information in the speech signal at behavioral and neurophysiological levels. Method Behavioral and event-related potential measures were taken in a selective cross-modal priming paradigm to examine how late L2 learners (N = 24, M age = 25.54 years) assessed the congruency of phonetic (target vowel: /a/ or /i/) and emotional (target affect: happy or angry) information between the visual primes of facial pictures and the auditory targets of spoken syllables. Results Behavioral accuracy data showed a significant congruency effect in affective (but not phonetic) priming. Unlike a previous report on monolingual first language (L1) users, the L2 users showed no facilitation in reaction time for congruency detection in either selective priming task. The neurophysiological results revealed a robust N400 response that was stronger in the phonetic condition but without clear lateralization and that the N400 effect was weaker in late L2 listeners than in monolingual L1 listeners. Following the N400, late L2 learners showed a weaker late positive response than the monolingual L1 users, particularly in the left central to posterior electrode regions. Conclusions The results demonstrate distinct patterns of behavioral and neural processing of phonetic and affective information in L2 speech with reduced neural representations in both the N400 and the later processing stage, and they provide an impetus for further research on similarities and differences in L1 and L2 multisensory speech perception in bilingualism.

Electrophysiological Differentiation of the Effects of Stress and Accent on Lexical Integration in Highly Fluent Bilinguals

Individuals who acquire a second language (L2) after infancy often retain features of their native language (L1) accent. Cross-language priming studies have shown negative effects of L1 accent on L2 comprehension, but the role of specific speech features, such as lexical stress, is mostly unknown. Here, we investigate whether lexical stress and accent differently modulate semantic processing and cross-language lexical activation in Welsh–English bilinguals, given that English and Welsh differ substantially in terms of stress realisation. In an L2 cross-modal priming paradigm, we manipulated the stress pattern and accent of spoken primes, whilst participants made semantic relatedness judgments on visual word targets. Event-related brain potentials revealed a main effect of stress on target integration, such that stimuli with stress patterns compatible with either the L1 or L2 required less processing effort than stimuli with stress incompatible with both Welsh and English. An independent cross-language phonological overlap manipulation revealed an interaction between accent and L1 access. Interestingly, although it increased processing effort, incorrect stress did not significantly modulate semantic priming effects or covert access to L1 phonological representations. Our results are consistent with the concept of language-specific stress templates, and suggest that accent and lexical stress affect speech comprehension mechanisms differentially.

Cross-modality interference effects of phonetic and prosodic information in second language learners

Successful communication relies on efficient use of information from multiple modalities and dimensions. Our previous research using a cross-modal priming paradigm showed that congruent visual primes facilitate faster recognition of phonetic and emotional prosodic information in speech. Event-related potential (ERP) data further revealed distinct brain mechanisms in the N400 and late positive response (LPR) components for processing linguistic and paralinguistic congruency. The current study extended the same paradigm to English-as-a-second-language (ESL) learners to examine possible interference between the two informational dimensions as a function of language experience. Participants were sixteen normal adult ESL learners. Monosyllables /bab/ and /bib/ in a happy or angry tone were used as the auditory stimuli, and pictures of the speaker articulating vowel /a/ and /i/ with a happy or angry facial expression were used as the visual primes. Compared to native English speakers, ESL learners showed significantly longer reaction time with inconsistent congruency effects in both conditions. But their behavioral accuracy data mirrored those of the native speakers, and native-like N400 and LPR components were also reliably elicited in the ESL group in both conditions. Together, these results indicate stronger cross-modal interference between phonetic and emotional prosodic information in speech for second language learners.

A biphasic effect of cross-modal priming on visual shape recognition

We used a cross-modal priming paradigm to evoke a biphasic effect in visual short-term memory. Participants were required to match the memorandum (a visual shape, either spiky or curvy) to a delayed probe (a shape belonging to the same category). In two-thirds of trials the sequence of shapes was accompanied by a task-irrelevant sound (either tzk or upo, cross-modally correspondent to spiky and curvy shape categories, respectively). The biphasic effect occurred when a congruent vs. incongruent sound was presented 200ms after the memorandum, while it did not occur when the sound was presented 200ms before or simultaneously with it. The biphasic pattern of recognition sensitivities was revealed by an interaction between cross-modal congruency and probe delay, such that sensitivity was higher for visual shapes paired with a congruent rather than incongruent sound with a 300-ms delay, while the opposite was true with a 1300-ms delay. We suggest that this biphasic pattern of recognition sensitivities was dependent on the task-irrelevant sound activating different levels of shape processing as a function of the relative timing of sound, memorandum, and probe.

Misattribution of musical arousal increases sexual attraction towards opposite-sex faces in females.

Several theories about the origins of music have emphasized its biological and social functions, including in courtship. Music may act as a courtship display due to its capacity to vary in complexity and emotional content. Support for music’s reproductive function comes from the recent finding that only women in the fertile phase of the reproductive cycle prefer composers of complex melodies to composers of simple ones as short-term sexual partners, which is also in line with the ovulatory shift hypothesis. However, the precise mechanisms by which music may influence sexual attraction are unknown, specifically how music may interact with visual attractiveness cues and affect perception and behaviour in both genders. Using a crossmodal priming paradigm, we examined whether listening to music influences ratings of facial attractiveness and dating desirability of opposite-sex faces. We also tested whether misattribution of arousal or pleasantness underlies these effects, and explored whether sex differences and menstrual cycle phase may be moderators. Our sample comprised 64 women in the fertile or infertile phase (no hormonal contraception use) and 32 men, carefully matched for mood, relationship status, and musical preferences. Musical primes (25 s) varied in arousal and pleasantness, and targets were photos of faces with neutral expressions (2 s). Group-wise analyses indicated that women, but not men, gave significantly higher ratings of facial attractiveness and dating desirability after having listened to music than in the silent control condition. High-arousing, complex music yielded the largest effects, suggesting that music may affect human courtship behaviour through induced arousal, which calls for further studies on the mechanisms by which music affects sexual attraction in real-life social contexts.

The Sensory-Dependent Nature of Audio-Visual Interactions for Semantic Knowledge

The Sensory-Dependent Nature of Audio-Visual Interactions for Semantic Knowledge Guillaume Vallet (guillaume.vallet@univ-lyon2.fr) Universit´e Lumi`ere Lyon 2. Laboratoire EMC, 5 avenue Pierre Mend`es France, 69676, Bron cedex, France & Laval University, School of Psychololgy, 2325 rue des Biblioth`eques Quebec City (Quebec), G1V 0A6 Canada Benoit Riou (benoit.riou@univ-lyon2.fr), R´emy Versace (remy.versace@univ-lyon2.fr) Universit´e Lumi`ere Lyon 2. Laboratoire EMC, 5 avenue Pierre Mend`es France, 69676, Bron cedex, France Martine Simard (martine.simard@psy.ulaval.ca) Laval University, School of Psychololgy, 2325 rue des Biblioth`eques Quebec City (Quebec), G1V 0A6 Canada Abstract The nature of audio-visual interactions is poorly understood for meaningful objects. These interactions would be indirect through semantic memory according to the amodal nature of knowledge, whereas these interactions would be direct accord- ing to the modal nature of knowledge. This question, cen- tral for both memory and multisensory frameworks, was as- sessed using a cross-modal priming paradigm from auditory to visual modalities tested on familiar objects. For half of the sound primes, a visual abstract mask was simultaneously pre- sented to the participants. The results showed a cross-modal priming effect for semantically congruent objects compared to semantically incongruent objects presented without the mask. The mask interfered in the semantically congruent condition, but had no effect in the semantically incongruent condition. The semantic specificity of the mask effect demonstrates a memory-related effect. The results suggest that audio-visual interactions are direct. The data support the modal approach of knowledge and the grounded cognition theory. Keywords: Memory; Perception; Audio-visual; Masking; Priming; Grounded Cognition. Introduction Our environment is filled with meaningful objects repre- senting semantic knowledge. These objects are perceptually processed using several sensory channels in which the auditory and visual modalities dominate the other senses in Human (for a review see Spence, 2007). The sensory infor- mation is mainly integrated on the basis of the temporal and spatial relationships between the stimuli (Calvert & Thesen, 2004), and also on the basis of the semantic relationships existing between them (Laurienti, Kraft, Maldjian, Burdette, & Wallace, 2004). Yet it remains uncertain how semantic memory aspects are involved in multisensory perception (for a review see Doehrmann & Naumer, 2008). This issue depends on the perceptual or semantic nature of cross-modal interactions, and thus questions the modal or amodal nature of knowledge (Vallet, Brunel, & Versace, 2010). The present study therefore aims at assessing the nature of audio-visual interactions using an innovative masking procedure. Communication between different modalities is called in- teraction (or interplay). If this interaction involves a repre- sentation of higher level, this interaction is called integration (Driver & Noesselt, 2008). An integrated object is a represen- tation that is more than the sum of its part. Previous research in the multisensory perception theoretical framework princi- pally studied the neural basis of the integration mechanism using meaningless stimuli (for review see Calvert & Thesen, 2004; Koelewijn, Bronkhorst, & Theeuwes, 2010). Fewer studies were conducted with meaningful stimuli, and the goal of these studies was also to determine the brain substrates of multisensory integration (Doehrmann & Naumer, 2008). The semantic constraint is generally assessed by manipulating the semantic congruency. A congruent trial is when the prime and the target refer to the same semantic object (meowing sound - cat’s picture). Semantic congruent stimuli usually fa- cilitate information processing (Chen & Spence, 2010), and may enhance memory performances in semantic (Laurienti et al., 2004) and episodic tasks (Lehmann & Murray, 2005). In the memory theoretical framework, cross-modal inter- actions tested on meaningful stimuli are generally studied by inserting a delay between the stimuli. The most famous paradigm in this field is the cross-modal priming paradigm. The cross-modal priming effect is the facilitation of the pro- cessing of one stimulus in one modality (the target) by the previous presentation of another stimulus in another modal- ity (the prime). The cross-modal priming effect may be ob- served between different modalities, such as the haptic and vi- sual modalities (Easton, Srinivas, & Greene, 1997), but most of the studies were realized between the auditory and visual modalities (for a review see Schneider, Engel, & Debener, 2008). The increasing number of studies on the audio-visual interactions involving meaningful stimuli are aimed at a bet- ter understanding of these effects. Nevertheless, the nature of audio-visual interactions, which is the central issue underly- ing these effects, remains poorly understood. The nature of these interactions depends on the nature

Cortical processing of phonetic and emotional information in speech: A cross-modal priming study

The current study employed behavioral and electrophysiological measures to investigate the timing, localization, and neural oscillation characteristics of cortical activities associated with phonetic and emotional information processing of speech. The experimental design used a cross-modal priming paradigm in which the normal adult participants were presented a visual prime followed by an auditory target. Primes were facial expressions that systematically varied in emotional content (happy or angry) and mouth shape (corresponding to /a/ or /i/ vowels). Targets were spoken words that varied by emotional prosody (happy or angry) and vowel (/a/ or /i/). In both the phonetic and prosodic conditions, participants were asked to judge congruency status of the visual prime and the auditory target. Behavioral results showed a congruency effect for both percent correct and reaction time. Two ERP responses, the N400 and late positive response (LPR), were identified in both conditions. Source localization and inter-trial phase coherence of the N400 and LPR components further revealed different cortical contributions and neural oscillation patterns for selective processing of phonetic and emotional information in speech. The results provide corroborating evidence for the necessity of differentiating brain mechanisms underlying the representation and processing of co-existing linguistic and paralinguistic information in spoken language, which has important implications for theoretical models of speech recognition as well as clinical studies on the neural bases of language and social communication deficits.

Connections between Fingerspelling and Print: The Impact of Working Memory and Temporal Dynamics on Lexical Activation

Recently there has been a renewed interest in characterizing the role of fingerspelling for deaf readers. The present study takes a step back and creates a theoretical foundation for investigating similarities between fingerspelling and print decoding in hearing signers. In this way, we can probe the constraints of temporal processing and memory on L1 orthography and the processing of L2 fingerspelling. Using a cross-modal priming paradigm, the role of orthography and phonology in print and fingerspelling word recognition was investigated. Results indicate significant inhibition in target retrieval when the prime was fingerspelled but not when it was presented in print. It was hypothesized that inhibition was due to either recoding or prime temporal dynamics. Hearing nonsigners were tested with serially or simultaneously presented print to determine the role of recoding and temporal dynamics. The results suggest that: (1) difficulties in processing fingerspelling for L2 learners might arise from recoding back into an L1 orthographic representation; (2) working memory abilities may reduce inhibition caused by recoding in L2 learners; (3) serial presentation of an orthographic code, either manual or visual, reduces priming effects; and (4) letter position differences provide evidence of depletion of activation over time.

Laterality effects in cross-modal affective priming

ABSTRACTThe present study pursued M. P. Bryden's legacy by investigating how contextual factors can affect laterality effects. Specifically, a cross-modal affective priming paradigm was used in two experiments to determine whether priming with facial expressions would affect responses to emotional sounds. Experiment 1 established that cross-modal priming could be obtained when presenting the emotional sounds binaurally by showing more accurate responses when prime and target were congruent than when they were incongruent, although this extended to response time only for the happy emotion. This priming effect justified Experiment 2, in which the priming paradigm was integrated into a dichotic listening task. The central finding of Experiment 2 was a congruency by ear interaction on number of correct reports, showing that presentation of a facial emotion congruent with a left target produced a large left ear advantage that was reduced when a right ear congruent prime or an incongruent pairing was used. Implications of these findings for emotion processing in the context of Bryden's legacy are discussed.

The perceptual nature of audiovisual interactions for semantic knowledge in young and elderly adults

Audiovisual interactions for familiar objects are at the core of perception. The nature of these interactions depends on the amodal – sensory abstracted – or modal – sensory-dependent – approach of knowledge. According to these approaches, the interactions should be respectively semantic and indirect or perceptual and direct. This issue is therefore a central question to memory and perception, yet the nature of these interactions remains unexplored in young and elderly adults. We used a cross-modal priming paradigm combined with a visual masking procedure of half of the auditory primes. The data demonstrated similar results in the young and elderly adult groups. The mask interfered with the priming effect in the semantically congruent condition, whereas the mask facilitated the processing of the visual target in the semantically incongruent condition. These findings indicate that audiovisual interactions are perceptual, and support the grounded cognition theory.

Proficiency modulates early orthographic and phonological processing in L2 spoken word recognition

The present study investigated orthographic and phonological processing in L2 French spoken word recognition by Finnish learners of French, using the masked cross-modal priming paradigm. Experiment 1 showed a repetition effect in L2 within-language priming that was most pronounced for high proficiency learners and a significant effect for French pseudohomophones. In the between-language Experiment 2, high proficiency learners showed significant facilitation from L1 Finnish to L2 French shared orthography in the absence of phonological and semantic overlap. This effect was not observed in the lower intermediate group, which showed a significant benefit of L1 pseudohomophones instead. The orthographic effect in the high proficiency group was modulated by subjective familiarity showing facilitation for less familiar but not for highly familiar words. The results suggest that with L2 learners, the extent to which orthographic information affects L2 spoken word recognition depends on their L2 proficiency.

Syntactic structural parallelisms influence processing of positive stimuli: Evidence from cross-modal ERP priming

Language can strongly influence the emotional state of the recipient. In contrast to the broad body of experimental and neuroscientific research on semantic information and prosodic speech, the emotional impact of grammatical structure has rarely been investigated. One reason for this might be, that measuring effects of syntactic structure involves the use of complex stimuli, for which the emotional impact of grammar is difficult to isolate.In the present experiment we examined the emotional impact of structural parallelisms, that is, repetitions of syntactic features, on the emotion-sensitive “late positive potential” (LPP) with a cross-modal priming paradigm. Primes were auditory presented nonsense sentences which included grammatical–syntactic parallelisms. Visual targets were positive, neutral, and negative faces, to be classified as emotional or non-emotional by the participants. Electrophysiology revealed diminished LPP amplitudes for positive faces following parallel primes. Thus, our findings suggest that grammatical structure creates an emotional context that facilitates processing of positive emotional information.

Effects of mode, consonance, and register in visual and word-evaluation affective priming experiments

An affective cross-modal priming paradigm was used in three experiments to test the effects of mode, consonance, and register in picture-evaluation and word-evaluation tasks. In experiment 1, participants heard major mode/minor mode, high-register/low-register chords (three tones) as primes and then they had to categorize a ‘happy’ or ‘sad’ word as a target. Participants evaluated target words faster if the words were preceded by a similarly valenced chord as opposed to affectively incongruent chord-word pairs. In experiments 2 and 3, target words were replaced by target affective pictures. In experiment 2, the primes were consonant/dissonant, high register/low-register chords. Register influenced picture evaluation whereas consonance was not effective for the affective priming. In experiment 3, the primes were major mode/minor mode, high-register/low-register chords. Register induced a faster recognition of targets while mode was not effective. Register is suggested to be a more powerful structural aspect of music than mode or consonance in influencing ongoing cognitive activities.

Cross-modal priming effect based on short-term experience of ecologically unrelated audio-visual information: An event-related potential study

In the present study, we aimed to identify whether cross-modal priming effect based on short-term experience of ecologically unrelated audio-visual information existed. In the experiment, we employed ecologically unrelated pictures and sounds as visual and auditory stimuli. Participants were required to learn the associations between pictures and sounds in the study phase, and then finished the following experiment adopting a cross-modal priming paradigm. The result showed that the processing of sound was facilitated by a priming picture if an audio-visual pair was presented in the study phase. Enhanced induced gamma-band activity (GBA) was also found under the Match condition, and we suggested that induced GBA reflected the association between auditory and visual information to form supra-modal representation, which is top-down modulated by short-term experience of audio-visual information. Event-related potential (ERP) analysis revealed an N400 effect under the Mismatch condition compared to the Match condition, and source reconstruction of N400 effect showed that the biggest difference of activity between two conditions was localized in middle temporal gyrus (MTG), suggesting that MTG played an important role in the mapping process of auditory information onto a temporal semantic network.

Crossmodal transfer of arousal, but not pleasantness, from the musical to the visual domain.

Arousal and valence (pleasantness) are considered primary dimensions of emotion. However, the degree to which these dimensions interact in emotional processing across sensory modalities is poorly understood. We addressed this issue by applying a crossmodal priming paradigm in which auditory primes (Romantic piano solo music) varying in arousal and/or pleasantness were sequentially paired with visual targets (IAPS pictures). In Experiment 1, the emotion spaces of 120 primes and 120 targets were explored separately in addition to the effects of musical training and gender. Thirty-two participants rated their felt pleasantness and arousal in response to primes and targets on equivalent rating scales as well as their familiarity with the stimuli. Musical training was associated with elevated familiarity ratings for high-arousing music and a trend for elevated arousal ratings, especially in response to unpleasant musical stimuli. Males reported higher arousal than females for pleasant visual stimuli. In Experiment 2, 40 nonmusicians rated their felt arousal and pleasantness in response to 20 visual targets after listening to 80 musical primes. Arousal associated with the musical primes modulated felt arousal in response to visual targets, yet no such transfer of pleasantness was observed between the two modalities. Experiment 3 sought to rule out the possibility of any order effect of the subjective ratings, and responses of 14 nonmusicians replicated results of Experiment 2. This study demonstrates the effectiveness of the crossmodal priming paradigm in basic research on musical emotions.