Cross-modal Task Research Articles

Unimodal and multimodal regions for logographic language processing in left ventral occipitotemporal cortex

The human neocortex appears to contain a dedicated visual word form area (VWFA) and an adjacent multimodal (visual/auditory) area. However, these conclusions are based on functional magnetic resonance imaging (fMRI) of alphabetic language processing, languages that have clear grapheme-to-phoneme correspondence (GPC) rules that make it difficult to disassociate visual-specific processing from form-to-sound mapping. In contrast, the Chinese language has no clear GPC rules. Therefore, the current study examined whether native Chinese readers also have the same VWFA and multimodal area. Two cross-modal tasks, phonological retrieval of visual words and orthographic retrieval of auditory words, were adopted. Different task requirements were also applied to explore how different levels of cognitive processing modulate activation of putative VWFA-like and multimodal-like regions. Results showed that the left occipitotemporal sulcus (LOTS) responded exclusively to visual inputs and an adjacent region, the left inferior temporal gyrus (LITG), showed comparable activation for both visual and auditory inputs. Surprisingly, processing levels did not significantly alter activation of these two regions. These findings indicated that there are both unimodal and multimodal word areas for non-alphabetic language reading, and that activity in these two word-specific regions are independent of task demands at the linguistic level.

Constraints on the Transfer of Perceptual Learning in Accented Speech

The perception of speech sounds can be re-tuned through a mechanism of lexically driven perceptual learning after exposure to instances of atypical speech production. This study asked whether this re-tuning is sensitive to the position of the atypical sound within the word. We investigated perceptual learning using English voiced stop consonants, which are commonly devoiced in word-final position by Dutch learners of English. After exposure to a Dutch learner’s productions of devoiced stops in word-final position (but not in any other positions), British English (BE) listeners showed evidence of perceptual learning in a subsequent cross-modal priming task, where auditory primes with devoiced final stops (e.g., “seed”, pronounced [si:th]), facilitated recognition of visual targets with voiced final stops (e.g., SEED). In Experiment 1, this learning effect generalized to test pairs where the critical contrast was in word-initial position, e.g., auditory primes such as “town” facilitated recognition of visual targets like DOWN. Control listeners, who had not heard any stops by the speaker during exposure, showed no learning effects. The generalization to word-initial position did not occur when participants had also heard correctly voiced, word-initial stops during exposure (Experiment 2), and when the speaker was a native BE speaker who mimicked the word-final devoicing (Experiment 3). The readiness of the perceptual system to generalize a previously learned adjustment to other positions within the word thus appears to be modulated by distributional properties of the speech input, as well as by the perceived sociophonetic characteristics of the speaker. The results suggest that the transfer of pre-lexical perceptual adjustments that occur through lexically driven learning can be affected by a combination of acoustic, phonological, and sociophonetic factors.

Sensitivity to the acoustic correlates of lexical stress and their relationship to reading in skilled readers.

The role of suprasegmental information in reading processes is a growing area of interest, and sensitivity to lexical stress has been shown to explain unique variance in reading development. However, less is known about its role in skilled reading. This study aimed to investigate the acoustic features of suprasegmental information using a same/different cross-modal matching task. Sixty-four adult participants completed standardized measures of reading accuracy, reading speed, and comprehension and performed an experimental task. The experimental task required the participants to identify whether non-speech acoustic sequences matched the characteristics of written words. The findings indicated differences in responses depending on where the lexical stress was required for the word. Moreover, evidence was found to support the view that amplitude information is part of the word knowledge retrieval process in skilled reading. The findings are discussed relative to models of reading and the role of lexical stress in lexical access.

P2-14: Noise Effect to Cross-Modality Stop Signal Task in Patients with Attention-Deficit / Hyperactivity Disorder

Background:Response inhibition has been proposed as a core element of attention-deficit / hyperactivity disorder (ADHD). ADHD has two subtypes: the hyperactivity and combined subtype (ADHDcom) was considered to have more inhibition deficit, while the inattentive subtype (ADHDin) was not (Barkley, 1997). The stochastic resonance model proposed that noise exerts a positive effect on cognitive performance for ADHD (Soderlund et al., 2007). The current study addressed the issue of the facilitation / interference effect of noise on patients with ADHD by adopting a cross-modality stop-signal task.Method:The cross-modality stop signal task contained a visual discrimination task and an auditory stop signal. Participants were asked to respond to the visual stimuli, and they were asked to withhold their response when the stop signal was presented (25% of trials). The stop signal was a pure tone embedded background noise with various sound levels (no noise, 35 dB, 55 dB). The stop signal reaction time (SSRT) was est...

Effect of Perceptual Load on Semantic Access by Speech in Children

To examine whether semantic access by speech requires attention in children. Children (N = 200) named pictures and ignored distractors on a cross-modal (distractors: auditory-no face) or multimodal (distractors: auditory-static face and audiovisual-dynamic face) picture word task. The cross-modal task had a low load, and the multimodal task had a high load (i.e., respectively naming pictures displayed on a blank screen vs. below the talker's face on his T-shirt). Semantic content of distractors was manipulated to be related vs. unrelated to the picture (e.g., picture "dog" with distractors "bear" vs. "cheese"). If irrelevant semantic content manipulation influences naming times on both tasks despite variations in loads, Lavie's (2005) perceptual load model proposes that semantic access is independent of capacity-limited attentional resources; if, however, irrelevant content influences naming only on the cross-modal task (low load), the perceptual load model proposes that semantic access is dependent on attentional resources exhausted by the higher load task. Irrelevant semantic content affected performance for both tasks in 6- to 9-year-olds but only on the cross-modal task in 4- to 5-year-olds. The addition of visual speech did not influence results on the multimodal task. Younger and older children differ in dependence on attentional resources for semantic access by speech.

Effects of left and right hemisphere damage on sensitivity to global context during lexical ambiguity resolution

Background: A great deal of research suggests that both the left hemisphere (LH) and right hemisphere (RH) contribute to the resolution of lexically ambiguous words. If this is indeed the case, then patients with left (LHD) or right hemisphere brain damage (RHD) should exhibit deficits in ambiguity resolution. Although several studies have explored this issue, there is still little consensus regarding the exact nature of the patients' deficits. Moreover, the focus of most studies has been on ambiguity resolution in short, single-sentence contexts, so less is known about the impact of LH and RH damage on ambiguity resolution in larger contexts. Aims: The goal of the present study was to investigate sensitivity to a global (i.e., two-sentence) context during lexical ambiguity resolution in LHD and RHD patients relative to a group of control participants with no neurological impairment. Control participants were expected to activate only contextually appropriate meanings. In contrast, LHD patients were expected to have difficulty activating ambiguous word meanings or selecting contextually appropriate meanings. RHD patients were expected to have difficulty using context or suppressing contextually inappropriate meanings. Methods & Procedures: Three groups of participants took part in this study: 10 LHD patients with nonfluent aphasia, 8 RHD patients, and 9 age-matched control participants. A cross-modal sentence priming task was employed where participants first listened to a biased two-sentence passage ending in an ambiguous word. After each passage a visual target word, related to either the first or second meaning of the ambiguous word, was presented for lexical decision. Reaction time data were analysed using a repeated-measures ANOVA. Priming effects were tested by way of planned comparisons. Outcomes & Results: The performance of both patient groups differed markedly from that of the control group, who only showed priming for contextually appropriate meanings. The LHD patients did not exhibit any significant priming of ambiguous word meanings, whereas the RHD patients only showed priming for the most frequent or dominant meaning of ambiguous words (e.g., the “animal” meaning as opposed to the “emblem” meaning of seal) regardless of context. Conclusions: Both LH and RH damage can disrupt the process of ambiguity resolution in a global context. Specifically, LH damage is argued to impair the degree of activation of ambiguous word meanings or the time course of meaning activation, while RH damage seems to impair the use of context, leading to frequency-based meaning activation.

Crossmodal interactions and multisensory integration in the perception of audio-visual motion — A free-field study

Motion perception can be altered by information received through multiple senses. So far, the interplay between the visual and the auditory modality in peripheral motion perception is scarcely described. The present free-field study investigated audio-visual motion perception for different azimuthal trajectories in space. To disentangle effects related to crossmodal interactions (the influence of one modality on signal processing in another modality) and multisensory integration (binding of bimodal streams), we manipulated the subjects’ attention in two experiments on a single set of moving audio-visual stimuli. Acoustic and visual signals were either congruent or spatially and temporally disparate at motion offset. (i) Crossmodal interactions were studied in a selective attention task. Subjects were instructed to attend to either the acoustic or the visual stream and to indicate the perceived final position of motion. (ii) Multisensory integration was studied in a divided attention task in which subjects were asked to report whether they perceived unified or separated audio-visual motion offsets. The results indicate that crossmodal interactions in motion perception do not depend on the integration of the audio-visual stream. Furthermore, in the crossmodal task, both visual and auditory motion perception were susceptible to modulation by irrelevant streams, provided that temporal disparities did not exceed a critical range. Concurrent visual streams modulated auditory motion perception in the central field, whereas concurrent acoustic streams attracted visual motion information in the periphery. Differential abilities between the visual and auditory system when attempting to accurately track positional information along different trajectories account for the observed biasing effects.

Visual mismatch negativity elicited by facial expressions: new evidence from the equiprobable paradigm

BackgroundConverging evidence revealed that facial expressions are processed automatically. Recently, there is evidence that facial expressions might elicit the visual mismatch negativity (MMN), expression MMN (EMMN), reflecting that facial expression could be processed under non-attentional condition. In the present study, using a cross modality task we attempted to investigate whether there is a memory-comparison-based EMMN.Methods12 normal adults were instructed to simultaneously listen to a story and pay attention to a non-patterned white circle as a visual target interspersed among face stimuli. In the oddball block, the sad face was the deviant with a probability of 20% and the neutral face was the standard with a probability of 80%; in the control block, the identical sad face was presented with other four kinds of face stimuli with equal probability (20% for each). Electroencephalogram (EEG) was continuously recorded and ERPs (event-related potentials) in response to each kind of face stimuli were obtained. Oddball-EMMN in the oddball block was obtained by subtracting the ERPs elicited by the neutral faces (standard) from those by the sad faces (deviant), while controlled-EMMN was obtained by subtracting the ERPs elicited by the sad faces in the control block from those by the sad faces in the oddball block. Both EMMNs were measured and analyzed by ANOVAs (Analysis of Variance) with repeated measurements. sLORETA (standardized low-resolution brain electromagnetic tomography) was used to investigate the cortical generators of controlled-EMMN.ResultsBoth the oddball-EMMN in deviant-standard comparison and the controlled-EMMN in deviant-control comparison were observed at occipital-temporal regions with right hemisphere predominance. The oddball-EMMN was bigger and earlier than the controlled-EMMN because, besides the memory-based comparison, the former included a difference of refractoriness due to the distinction of presented probability between the deviant and standard face stimuli. The source analysis of controlled-EMMN indicated a current source primarily involved in posterior areas including superior temporal gyrus, postcentral gyrus, inferior parietal lobule as well as the insula.ConclusionsThe valid EMMN properly reflecting the memory-based comparison of facial expressions could be obtained, i.e., the controlled-EMMN.

Audiovisual crossmodal correspondences in Autism Spectrum Disorders (ASDs)

The label ‘crossmodal correspondences’ has been used to define the nonarbitrary associations that appear to exist between different basic physical stimulus attributes in different sensory modalities. For instance, it has been consistently shown in the neurotypical population that higher pitched sounds are more frequently matched with visual patterns which are brighter, smaller, and sharper than those associated to lower pitched sounds. Some evidence suggests that patients with ASDs tend not to show this crossmodal preferential association pattern (e.g., curvilinear shapes and labial/lingual consonants vs. rectilinear shapes and plosive consonants). In the present study, we compared the performance of children with ASDs (6–15 years) and matched neurotypical controls in a non-verbal crossmodal correspondence task. The participants were asked to indicate which of two bouncing visual patterns was making a centrally located sound. In intermixed trials, the visual patterns varied in either size, surface brightness, or shape, whereas the sound varied in pitch. The results showed that, whereas the neurotypical controls reliably matched the higher pitched sound to a smaller and brighter visual pattern, the performance of participants with ASDs was at chance level. In the condition where the visual patterns differed in shape, no inter-group difference was observed. Children’s matching performance cannot be attributed to intensity matching or difficulties in understanding the instructions, which were controlled. These data suggest that the tendency to associate congruent visual and auditory features vary as a function of the presence of ASDs, possibly pointing to poorer capabilities to integrate auditory and visual inputs in this population.

The informational constraints of behavioral distraction by unexpected sounds: The role of event information.

Sounds deviating from an otherwise repeated stream of task-irrelevant auditory stimuli (deviant sounds among standard sounds) are known to capture attention and impact negatively on ongoing behavioral performance (behavioral oddball distraction). Traditional views consider such distraction as the ineluctable consequence of the deviant sounds' low probability of occurrence relative to that of the standard. Contrary to this contention, recent evidence demonstrates that distraction by deviant sounds is not obligatory and occurs only when sounds (standards and deviants), though to be ignored, act as useful warning cues by providing information as to whether and when a target stimulus is to be presented (Parmentier, Elsley, & Ljungberg, 2010). The present study aimed to extend this finding by disentangling the roles of event information (target's probability of occurrence) and temporal information (target's time of occurrence). Comparing performance in a cross-modal oddball task where standard and deviant sounds provided temporal information, event information, both, or none, we found that distraction by deviant sounds emerged when sounds conveyed event information. These results suggest that unexpected changes in a stream of sounds yield behavioral distraction to the extent that standards and deviants carry relevant goal-directed information, specifically, the likelihood of occurrence of an upcoming target.

On Chinese EFL Learners’ Homonym Processing in Relation to Their Organization of L2 Mental Lexicon

Based on the models of lexical ambiguity resolution and the organization of L2 mental lexicon proposed by previous researchers in recent years, this paper aimed at investigating the correlation between the Chinese EFL learners’ homonym processing in relation to their organization of L2 mental lexicon through cross-modal sentence-priming lexical decision tasks and word association tasks completed by both English major and non-English major students. The results showed that there is a positive correlation between the learners’ organization of L2 mental lexicon and their performance on English homonym processing. Inferential statistic analysis indicates that learners whose L2 mental lexicon is more syntagmatically related can process the English homonym more effectively. They can use contextual cues (semantic and syntactic information) to deactivate inappropriate meanings while low proficiency learners may be slower and less effective in using the sentence context cues needed for disambiguation. Therefore, it has been tentatively indicated that effective spreading activation of a word during reading and listening is important and helpful for learners’ comprehension, and pedagogically suggested that consciously categorizing syntagmatically related words in L2 mental lexicon during teaching and learning new English words can help reconstruct lexical networks so as to correctly and quickly retrieve the words from the mental lexicon in a certain context and better understand the meaning of the whole context in reading and listening.

Automatic Selective Attention as a Function of Sensory Modality in Aging

It was recently hypothesized that age-related differences in selective attention depend on sensory modality (Guerreiro, M. J. S., Murphy, D. R., & Van Gerven, P. W. M. (2010). The role of sensory modality in age-related distraction: A critical review and a renewed view. Psychological Bulletin, 136, 975-1022. doi:10.1037/a0020731). So far, this hypothesis has not been tested in automatic selective attention. The current study addressed this issue by investigating age-related differences in automatic spatial cueing effects (i.e., facilitation and inhibition of return [IOR]) across sensory modalities. Thirty younger (mean age = 22.4 years) and 25 older adults (mean age = 68.8 years) performed 4 left-right target localization tasks, involving all combinations of visual and auditory cues and targets. We used stimulus onset asynchronies (SOAs) of 100, 500, 1,000, and 1,500 ms between cue and target. The results showed facilitation (shorter reaction times with valid relative to invalid cues at shorter SOAs) in the unimodal auditory and in both cross-modal tasks but not in the unimodal visual task. In contrast, there was IOR (longer reaction times with valid relative to invalid cues at longer SOAs) in both unimodal tasks but not in either of the cross-modal tasks. Most important, these spatial cueing effects were independent of age. The results suggest that the modality hypothesis of age-related differences in selective attention does not extend into the realm of automatic selective attention.

The interplay of cue modality and response latency in brain areas supporting crossmodal motor preparation: an event-related fMRI study

Crossmodal (auditory, visual) motor facilitation can be defined as a cue in one sensory modality eliciting speeded responses to targets in a different sensory modality. We used event-related functional magnetic resonance imaging (fMRI) to isolate brain activity underlying crossmodal motor preparation. Our predictions were that interactions between input modality and processes underlying response selection would be indexed by distinct spatiotemporal brain dynamics. A crossmodal response selection task was designed in which a central, nonspatial cue indicated the response rule (compatible or incompatible) to a lateralized target. Cues and targets appeared in auditory and visual modalities and were separated by a lengthy delay period in which cue-related brain activity could be dissociated. We found faster reaction times to auditory compared with visual cues. Next, we correlated brain activity with behavioural performance using multivariate spatiotemporal partial least squares. We identified a distinct, significant brain-behaviour pattern in which faster reaction times to auditory cues were correlated with higher blood oxygenation level-dependent percent signal change in medial visual, frontoparietal (inferior parietal lobule, superior frontal gyrus and premotor cortex) and subcortical (thalamus and cerebellum) areas. For visual cues, quicker responses were linked to greater activity in the same frontoparietal and subcortical but not medial visual areas. Our results show that both modality-dependent and modality-independent brain areas with different brain-behaviour relationships are implicated in crossmodal motor preparation.

The effects of inter-stimulus interval and prime modality in a semantic priming task

Background: Semantic priming studies are employed in order to examine how various semantic contexts can influence visual word recognition processes. Although research has shown numerous factors can have an influence on the magnitude of the semantic priming effects found in lexical decision tasks, the majority of these factors have been related to the prime–target relationship itself. However, other factors have also been shown to alter the priming effect. Two such factors are the inter-stimulus interval (ISI) and presentation modality. The inter-stimulus interval may be used to independently assess automatic and strategic processes, whereas presentation modality is typically used to assess the different processing time courses that occur with spoken or written words. These factors have not been adequately investigated in the normal population. A more in-depth understanding of the relationships between prime modality, inter-stimulus interval, and word recognition processes in a sample of individuals with typical language abilities may provide valuable when examining lexical access and storage in language disordered populations such as those with aphasia. Aims: The focus of the present study was to examine the impact of relatively short and long inter-stimulus intervals on processing time in a visual and cross-modal lexical decision task. Previous research has not fully addressed whether convergent processes occur during cross-modal tasks or if an amodal semantic system exists. The utilisation of slow and fast inter-stimulus intervals should allow for a clearer distinction relative to processing. Methods & Procedures: A series of four lexical decision tasks was used to investigate reaction time and accuracy. The four tasks resulted from the combination of the independent variables ISI (0 ms or 400 ms) and prime modality (auditory or visual). Outcomes & Results: Results indicated that participants exhibited a larger priming effect when stimuli were presented in the 0 ms condition. Results also indicated that participants responded more accurately when the target word was presented auditorily. Conclusions: It was concluded that automatic spreading activation occurred in both the visual and auditory modalities, providing further evidence of an amodal semantic system. These observations are helpful in developing a clearer understanding of lexical storage and access in language impairments such as aphasia.

Developmental changes in the inferior frontal cortex for selecting semantic representations

Functional magnetic resonance imaging (fMRI) was used to examine the neural correlates of semantic judgments to Chinese words in a group of 10–15 year old Chinese children. Two semantic tasks were used: visual–visual versus visual–auditory presentation. The first word was visually presented (i.e. character) and the second word was either visually or auditorily presented, and the participant had to determine if these two words were related in meaning. Different from English, Chinese has many homophones in which each spoken word corresponds to many characters. The visual–auditory task, therefore, required greater engagement of cognitive control for the participants to select a semantically appropriate answer for the second homophonic word. Weaker association pairs produced greater activation in the mid-ventral region of left inferior frontal gyrus (BA 45) for both tasks. However, this effect was stronger for the visual–auditory task than for the visual–visual task and this difference was stronger for older compared to younger children. The findings suggest greater involvement of semantic selection mechanisms in the cross-modal task requiring the access of the appropriate meaning of homophonic spoken words, especially for older children.

Necessity is the Mother of Invention: Reconstructing Missing Sensory Information in Multiple, Concurrent Reference Frames for Eye–Hand Coordination

When aligning the hand to grasp an object, the CNS combines multiple sensory inputs encoded in multiple reference frames. Previous studies suggest that when a direct comparison of target and hand is possible via a single sensory modality, the CNS avoids performing unnecessary coordinate transformations that add noise. But when target and hand do not share a common sensory modality (e.g., aligning the unseen hand to a visual target), at least one coordinate transformation is required. Similarly, body movements may occur between target acquisition and manual response, requiring that egocentric target information be updated or transformed to external reference frames to compensate. Here, we asked subjects to align the hand to an external target, where the target could be presented visually or kinesthetically and feedback about the hand was visual, kinesthetic, or both. We used a novel technique of imposing conflict between external visual and gravito-kinesthetic reference frames when subjects tilted the head during an instructed memory delay. By comparing experimental results to analytical models based on principles of maximum likelihood, we showed that multiple transformations above the strict minimum may be performed, but only if the task precludes a unimodal comparison of egocentric target and hand information. Thus, for cross-modal tasks, or when head movements are involved, the CNS creates and uses both kinesthetic and visual representations. We conclude that the necessity of producing at least one coordinate transformation activates multiple, concurrent internal representations, the functionality of which depends on the alignment of the head with respect to gravity.

Examining lateralized lexical ambiguity processing using dichotic and cross-modal tasks

The individual roles played by the cerebral hemispheres during the process of language comprehension have been extensively studied in tasks that require individuals to read text (for review see Jung-Beeman, 2005). However, it is not clear whether or not some aspects of the theorized laterality models of semantic comprehension are a result of the modality of presentation. Extending earlier work examining lateralized semantic processing using lexically ambiguous words, the current experiments utilized two modified lexical-decision tasks (one fully auditory and one cross-modal) with dichotically presented target stimuli. When targets were presented to the right ear/left hemisphere there was a distinct advantage for detecting words that are associated with the dominant meaning of the ambiguous word over the subordinate meaning. In contrast, for left ear/right hemisphere trials, there was either no difference between the pattern of semantic access for dominant and subordinate meaning (dichotic only) or a processing advantage for the subordinate meaning of the ambiguous word (with cross-modal presentation). These data suggest that the complimentary hemispheric strategies that allow for semantic access are not modality specific and instead characterize how the hemispheres each contribute to comprehension for both speech and text. Thus, dichotic presentation does seem to allow for the study of subtle hemispheric difference in meaning comprehension.

Influence of the body on crossmodal interference effects between tactile and two-dimensional visual stimuli

We investigated how tactile discrimination performance was interfered with by irrelevant two-dimensional visual stimuli using the crossmodal interference task. Participants made speeded discrimination responses to the location of vibrotactile targets presented to either tip or base of their forefinger, while trying to ignore simultaneously presented visual distractors presented to either side of the central fixation on a front display. The array of visual distractors was presented at four different angles, and the participants rested their stimulated hand on a desk in either a forward-pointing or inward-pointing posture. Although there was apparently no specific spatial relationship between the tactile and two-dimensional visual stimuli arrays and the spatial response requirement was controlled, visuotactile interference effects occurred between them. Moreover, we found that the spatial relationships between the arrays depended on the potential range of movement and the current posture of the vibrotactile-stimulated hand and possibly the stored orientation of our hand representation, even without any explicit cue referring to hands. Our results suggest that the visuotactile spatial interactions involve multiple mechanisms regarding our bodily perception and our internal body representation.

Multisensory Perceptual Learning of Temporal Order: Audiovisual Learning Transfers to Vision but Not Audition

BackgroundAn outstanding question in sensory neuroscience is whether the perceived timing of events is mediated by a central supra-modal timing mechanism, or multiple modality-specific systems. We use a perceptual learning paradigm to address this question.Methodology/Principal FindingsThree groups were trained daily for 10 sessions on an auditory, a visual or a combined audiovisual temporal order judgment (TOJ). Groups were pre-tested on a range TOJ tasks within and between their group modality prior to learning so that transfer of any learning from the trained task could be measured by post-testing other tasks. Robust TOJ learning (reduced temporal order discrimination thresholds) occurred for all groups, although auditory learning (dichotic 500/2000 Hz tones) was slightly weaker than visual learning (lateralised grating patches). Crossmodal TOJs also displayed robust learning. Post-testing revealed that improvements in temporal resolution acquired during visual learning transferred within modality to other retinotopic locations and orientations, but not to auditory or crossmodal tasks. Auditory learning did not transfer to visual or crossmodal tasks, and neither did it transfer within audition to another frequency pair. In an interesting asymmetry, crossmodal learning transferred to all visual tasks but not to auditory tasks. Finally, in all conditions, learning to make TOJs for stimulus onsets did not transfer at all to discriminating temporal offsets. These data present a complex picture of timing processes.Conclusions/SignificanceThe lack of transfer between unimodal groups indicates no central supramodal timing process for this task; however, the audiovisual-to-visual transfer cannot be explained without some form of sensory interaction. We propose that auditory learning occurred in frequency-tuned processes in the periphery, precluding interactions with more central visual and audiovisual timing processes. Functionally the patterns of featural transfer suggest that perceptual learning of temporal order may be optimised to object-centered rather than viewer-centered constraints.

Is two better than one? A cross-modal oddball paradigm reveals greater sensitivity of the P300 to emotional face-voice associations

Objective Studies exploring neurophysiological correlates of main psychiatric disorders have commonly used event-related potentials (ERP) during a visual or an auditory oddball task. The main results concern modulations of the P3b amplitude and/or latency. The present study aims to increase the clinical sensitivity of these P3b modulations by using a more ecological oddball design, using synchronized pairs of audio-visual emotional stimuli. Method Two groups of healthy participants, one composed of controls and the other of students displaying anxious and depressive tendencies completed visual, auditory and audio-visual (cross-modal) oddball tasks, in which they had to detect deviant happy and sad stimuli among neutral ones as quickly as possible. Behavioral performance and P3b ERP data were analyzed. Results Subjects displaying anxious and depressive tendencies exhibited lower P3b amplitude than the controls, but only in the cross-modal oddball task. Conclusions Although the two groups of subjects differed in their levels of co-morbid anxiety and depression, unimodal visual and auditory oddball tasks did not allow us to detect this difference by P3b amplitude modulations, but the cross-modal task did. Significance These results suggest that a cross-modal oddball design should be used in future studies to increase the sensitivity of the P300 amplitude differences between healthy participants and those with clinical symptoms.