Audiovisual Stimulation Research Articles

Revealing Action Representation Processes in Audio Perception Using Fractal EEG Analysis

Electroencephalogram (EEG) recordings, and especially the Mu-rhythm over the sensorimotor cortex that relates to the activation of the mirror neuron system (MNS), were acquired from two subject groups (orchestral musicians and nonmusicians), in order to explore action representation processes involved in the perception and performance of musical pieces. Two types of stimuli were used, i.e., an auditory one consisting of an excerpt of Beethoven's fifth symphony and a visual one presenting a conductor directing an orchestra performing the same excerpt of the piece. Three tasks were conducted including auditory stimulation, audiovisual stimulation, and visual stimulation only, and the acquired signals were processed using fractal [time-dependent fractal dimension (FD) estimation] and statistical analysis (analysis of variance, Mann-Whitney). Experimental results showed significant differences between the two groups while desychronization of the Mu-rhythm, which can be linked to MNS activation, was observed during all tasks for the musicians' group, as opposed to the nonmusicians' group who exhibited similar response only when the visual stimulus was present. The mobility of the conductor was also correlated to the estimated FD signals, showing significantly higher correlation for the case of musicians compared to nonmusicians' one. The present study sheds light upon the difference in action representation in auditory perception between musicians and nonmusicians and paves the way for better comprehension of the underlying mechanisms of the MNS.

Early Cortical Connective Network Relating to Audiovisual Stimulation by Partial Directed Coherence Analysis

Cross-modal sensory integration between the auditory and visual systems has long been a topic of interest in cognitive neuroscience, but spatiotemporal cortical interdependence under audiovisual stimulation is still poorly understood. In this letter, parametric spectral analysis, specifically partial directed coherence (PDC), is used to study the cortical connective causal-interdependence network under audiovisual cognitive processes within the time window of the N1 component, i.e., 96--145 ms poststimulus in the event-related potential (ERP). A total of 13 subjects (age: 22.9+/-0.8 years; male/female = 9/4) performed an audiovisual recognition task. Multichannel EEG signals were analyzed using an adaptive multivariate autoregressive model to identify causal influences between different cortical areas. Three different cortical connective networks were detected, and by comparing the cortical PDC network in the early cognitive process of audiovisual (VA) perception with that in unimodal situations (V or A), it was found that the left temporal cortex (TC) might be responsible for audiovisual integration. Such electrophysiological findings were highly consistent with previous functional neuroimaging results. Moreover, the results presented here show the dynamics of lateral causal influences during audiovisual integration.

Meaningful auditory information enhances perception of visual biological motion

Robust perception requires efficient integration of information from our various senses. Much recent electrophysiology points to neural areas responsive to multisensory stimulation, particularly audiovisual stimulation. However, psychophysical evidence for functional integration of audiovisual motion has been ambiguous. In this study we measure perception of an audiovisual form of biological motion, tap dancing. The results show that the audio tap information interacts with visual motion information, but only when in synchrony, demonstrating a functional combination of audiovisual information in a natural task. The advantage of multimodal combination was better than the optimal maximum likelihood prediction.

Gesture and metaphor comprehension: Electrophysiological evidence of cross-modal coordination by audiovisual stimulation

In recent years, studies have suggested that gestures influence comprehension of linguistic expressions, for example, eliciting an N400 component in response to a speech/gesture mismatch. In this paper, we investigate the role of gestural information in the understanding of metaphors. Event related potentials (ERPs) were recorded while participants viewed video clips of an actor uttering metaphorical expressions and producing bodily gestures that were congruent or incongruent with the metaphorical meaning of such expressions. This modality of stimuli presentation allows a more ecological approach to meaning integration. When ERPs were calculated using gesture stroke as time-lock event, gesture incongruity with metaphorical expression modulated the amplitude of the N400 and of the late positive complex (LPC). This suggests that gestural and speech information are combined online to make sense of the interlocutor’s linguistic production in an early stage of metaphor comprehension. Our data favor the idea that meaning construction is globally integrative and highly context-sensitive.

Does Audiovisual Stimulation With Music and Nature Sights (MuViCure) Reduce Pain and Discomfort During Placement of a Femoral Nerve Block?

MuViCure (Photobia ApS, Copenhagen, Denmark) is a new program for audiovisual stimulation. We hypothesized that audiovisual stimulation would reduce pain and discomfort and improve patients' well-being during placement of a femoral nerve block. Fifty-five outpatients scheduled for anterior cruciate ligament reconstruction were randomly allocated into three groups: the first group received audiovisual stimulation (MuViCure), the second group received audio stimulation (MusiCure, Gefion Records ApS, Virum, Denmark), and the third group received no intervention (control). Ten of the 55 patients underwent a qualitative in-depth interview 1-2 days after surgery. Pain and discomfort during the procedure were more prominent in the MuViCure group when compared with the other two groups. Despite these negative results, 14 of the 19 patients in the MuViCure group answered that MuViCure had a positive effect on their well-being. The qualitative interviews revealed that a number of factors other than the audiovisual stimulation had a significant impact on the patients' experience. The use of MuViCure may be more appropriate in other settings.

Phase Synchronization in Human EEG During Audio-Visual Stimulation

Synchrony of EEG data recorded under influence of audio-visual stimulation was investigated. Instantaneous phases were derived from Wavelet transform method. Phase synchronization is assessed quantitatively via measure derived from uniformity of phase difference distribution. EEG data came from 6 healthy volunteers repeatedly exposed to 20 min AVS program with intervals with stable stimulation at 4 and 17 Hz. Phase synchronization during AVS significantly increased in comparison to non stimulation conditions in all examined cortex locations. The lowest increases in synchronization occurred in the frontal areas. In central region no dumping in synchronization was recognizable in comparison to backward locations where visual processing centers are situated.

Audio-visual stimulation improves oculomotor patterns in patients with hemianopia

Patients with visual field disorders often exhibit impairments in visual exploration and a typical defective oculomotor scanning behaviour. Recent evidence [Bolognini, N., Rasi, F., Coccia, M., & Làdavas, E. (2005b). Visual search improvement in hemianopic patients after audio-visual stimulation. Brain, 128, 2830–2842] suggests that systematic audio-visual stimulation of the blind hemifield can improve accuracy and search times in visual exploration, probably due to the stimulation of Superior Colliculus (SC), an important multisensory structure involved in both the initiation and execution of saccades. The aim of the present study is to verify this hypothesis by studying the effects of multisensory training on oculomotor scanning behaviour. Oculomotor responses during a visual search task and a reading task were studied before and after visual (control) or audio-visual (experimental) training, in a group of 12 patients with chronic visual field defects and 12 controls subjects. Eye movements were recorded using an infra-red technique which measured a range of spatial and temporal variables. Prior to treatment, patients’ performance was significantly different from that of controls in relation to fixations and saccade parameters; after Audio-Visual Training, all patients reported an improvement in ocular exploration characterized by fewer fixations and refixations, quicker and larger saccades, and reduced scanpath length. Overall, these improvements led to a reduction of total exploration time. Similarly, reading parameters were significantly affected by the training, with respect to specific impairments observed in both left- and right-hemianopia readers. Our findings provide evidence that Audio-Visual Training, by stimulating the SC, may induce a more organized pattern of visual exploration due to an implementation of efficient oculomotor strategies. Interestingly, the improvement was found to be stable at a 1 year follow-up control session, indicating a long-term persistence of treatment effects on the oculomotor system.

Observing Virtual Arms that You Imagine Are Yours Increases the Galvanic Skin Response to an Unexpected Threat

Multi-modal visuo-tactile stimulation of the type performed in the rubber hand illusion can induce the brain to temporarily incorporate external objects into the body image. In this study we show that audio-visual stimulation combined with mental imagery more rapidly elicits an elevated physiological response (skin conductance) after an unexpected threat to a virtual limb, compared to audio-visual stimulation alone. Two groups of subjects seated in front of a monitor watched a first-person perspective view of slow movements of two virtual arms intercepting virtual balls rolling towards the viewer. One group was instructed to simply observe the movements of the two virtual arms, while the other group was instructed to observe the virtual arms and imagine that the arms were their own. After 84 seconds the right virtual arm was unexpectedly “stabbed” by a knife and began “bleeding”. This aversive stimulus caused both groups to show a significant increase in skin conductance. In addition, the observation-with-imagery group showed a significantly higher skin conductance (p<0.05) than the observation-only group over a 2-second period shortly after the aversive stimulus onset. No corresponding change was found in subjects' heart rates. Our results suggest that simple visual input combined with mental imagery may induce the brain to measurably temporarily incorporate external objects into its body image.

Temporal Characteristics of Audiovisual Information Processing

In complex natural environments, auditory and visual information often have to be processed simultaneously. Previous functional magnetic resonance imaging (fMRI) studies focused on the spatial localization of brain areas involved in audiovisual (AV) information processing, but the temporal characteristics of AV information flow in these regions remained unclear. In this study, we used fMRI and a novel information-theoretic approach to study the flow of AV sensory information. Subjects passively perceived sounds and images of objects presented either alone or simultaneously. Applying the measure of mutual information, we computed for each voxel the latency in which the blood oxygenation level-dependent signal had the highest information content about the preceding stimulus. The results indicate that, after AV stimulation, the earliest informative activity occurs in right Heschl's gyrus, left primary visual cortex, and the posterior portion of the superior temporal gyrus, which is known as a region involved in object-related AV integration. Informative activity in the anterior portion of superior temporal gyrus, middle temporal gyrus, right occipital cortex, and inferior frontal cortex was found at a later latency. Moreover, AV presentation resulted in shorter latencies in multiple cortical areas compared with isolated auditory or visual presentation. The results provide evidence for bottom-up processing from primary sensory areas into higher association areas during AV integration in humans and suggest that AV presentation shortens processing time in early sensory cortices.

Changes in effective connectivity of human superior parietal lobule under multisensory and unisensory stimulation

Previous event-related potential (ERP) studies have identified the superior parietal lobule (SPL) as actively multisensory. This study compares effective, or contextually active, connections to this region under unisensory and multisensory conditions. Effective connectivity, the influence of one brain region over another, during unisensory visual, unisensory auditory and multisensory audiovisual stimulation was investigated. ERPs were recorded from subdural electrodes placed over the parietal lobe of three patients while they conducted a rapid reaction-time task. A generative model of interacting neuronal ensembles for ERPs was inverted in a scheme allowing investigation of the connections from and to the SPL, a multisensory processing area. Important features of the ensemble model include inhibitory and excitatory feedback connections to pyramidal cells and extrinsic input to the stellate cell pool, with extrinsic forward and backward connections delineated by laminar connection differences between ensembles. The framework embeds the SPL in a plausible connection of distinct neuronal ensembles mirroring the integrated brain regions involved in the response task. Bayesian model comparison was used to test competing feed-forward and feed-backward models of how the electrophysiological data were generated. Comparisons were performed between multisensory and unisensory data. Findings from three patients show differences in summed unisensory and multisensory ERPs that can be accounted for by a mediation of both forward and backward connections to the SPL. In particular, a negative gain in all forward and backward connections to the SPL from other regions was observed during the period of multisensory integration, while a positive gain was observed for forward projections that arise from the SPL.

Attention rivalry under irrelevant audiovisual stimulation

Audiovisual integration has been known to enhance perception; nevertheless, another fundamental audiovisual interaction, i.e. attention rivalry, has not been well investigated. This paper studied the attention rivalry under irrelevant audiovisual stimulation using event-related potential (ERP) and behavioral analysis, and tested the existence of a vision dominated rivalry model. Participants need respond to the target in a bi- or unimodal audiovisual stimulation paradigm. The enhanced amplitude of central P300 under visual target bimodal stimulus indicated that vision demanded more cognitive resources, and the significant amplitude of frontal P200 under bimodal stimulus with non-target auditory stimulus implied that the brain mostly restrained the process of the non-target auditory information. ERP results, together with the analysis of the behavioral data and the subtraction waves, indicated a vision dominated attention rivalry model involved in audiovisual interaction. Furthermore, the latencies of P200 and P300 components implied that audiovisual attention rivalry occurred within the first 300ms after stimulus onset, i.e. significant differences were found in P200 latencies among three target bimodal stimuli, while no difference existed in P300 latencies. Attention shifting and re-directing might be the cause of such early audiovisual rivalry.

T07-P-02 Penile color-Doppler ultrasonography after audio-visual sexual stimulation and vardenafil administration. Results of single blind, prospective study

Introduction Dynamic Penile color-Doppler ultrasonography (CDS) is to be considered a second-level invasive test for the etiological diagnosis of erectile dysfunction. Some scientific evidences underline the possibility of replacing the drug-injection with a PDE-5 Inhibitors. The purpose of the study was to evaluate the possible relationship between the basal and dynamic results of penile Doppler ultrasound, complications and patient's compliance, after intra-cavernosal injection (ICI) or after vardenafil administration and audio visual sexual stimulation (AVSS). Materials and methods Between January an May 2007, 40 ED patients, median age 52 years (range 45-58), without contra-indications for using PDE-5 inhibitors, were included in the study. Each patient underwent two CDS in basal and dynamic condition using ICI (PGE 10 MCG), or, after 2 weeks, using vardenafil 20 mg and AVSS. The same operator, blinded performed all the tests. A satisfaction questionnaire was administrated to the patients after each study session. The differences were considered statistically significant for values of P Result After CDS and ICI there were 30 patients (75%) with nonvasculogenic ED, 2 patients (5%) with arteriogenic ED, 8 patients (20%) with venogenic ED. All the diagnosis were confirmed after 2 weeks with CDS and vardenafil, except in 4 patients (2 with arteriogenic and 2 with venogenic ED) vardenafil non-responders. Most part of patients (35/40) would prefer, in the need of procedure repetition, vardenafil instead of ICI. Conclusions The results of this study demonstrate that the two techniques are comparable in terms of diagnostic results. Patients’preferences highlight the conclusion that vardenafil can be regarded as an alternative to FIC during CDS evaluation.

Effects of trauma-related audiovisual stimulation on cerebrospinal fluid norepinephrine and corticotropin-releasing hormone concentrations in post-traumatic stress disorder

Although elevated concentrations of both corticotropin-releasing hormone (CRH) and norepinephrine are present in the cerebrospinal fluid (CSF) of patients with post-traumatic stress disorder (PTSD), the effects of exposure to traumatic stimuli on these stress-related hormones in CSF are unknown. A randomized, within-subject, controlled, cross-over design was used, in which patients with war-related PTSD underwent 6-h continuous lumbar CSF withdrawal on two occasions per patient (6-9 weeks apart). During one session the patients watched a 1-h film containing combat footage (traumatic film) and in the other a 1-h film on how to oil paint (neutral film). At 10-min intervals, we quantified CRH and norepinephrine in CSF, and ACTH and cortisol in plasma, before, during, and after symptom provocation. Subjective anxiety and mood were monitored using 100-mm visual analog scales. Blood pressure and heart rate were obtained every 10min from a left leg monitor. Eight of 10 patients completed two CSF withdrawal procedures each. A major drop in mood and increases in anxiety and blood pressure occurred during the traumatic relative to the neutral videotape. CSF norepinephrine rose during the traumatic film relative to the neutral videotape; this rise directly correlated with magnitude of mood drop. In contrast, CSF CRH concentrations declined during the trauma-related audiovisual stimulus, both absolutely and relative to the neutral stimulus; the magnitude of CRH decline correlated with degree of subjective worsening of anxiety level and mood. Plasma cortisol concentrations were lower and ACTH levels similar during the stress compared with the neutral videotape. CSF concentrations of the stress hormones norepinephrine and CRH differentially change after exposure to 1h of trauma-related audiovisual stimulation in chronic, combat-related PTSD. While the CSF norepinephrine increase was postulated, the decline in CSF CRH levels is surprising and could be due to audiovisual stress-induced increased uptake of CSF CRH into brain tissue, increased CRH utilization, increased CRH degradation, or to an acute stress-related inhibition or suppression of CRH secretion.

Brain Science: From the Very Small to the Very Large

We still lack a clear understanding of how brain imaging signals relate to neuronal activity. Recent work shows that the simultaneous activity of neuronal ensembles strongly correlates with local field potentials and imaging measurements.

Audiovisual integration of emotional signals in voice and face: An event-related fMRI study

In a natural environment, non-verbal emotional communication is multimodal (i.e. speech melody, facial expression) and multifaceted concerning the variety of expressed emotions. Understanding these communicative signals and integrating them into a common percept is paramount to successful social behaviour. While many previous studies have focused on the neurobiology of emotional communication in the auditory or visual modality alone, far less is known about multimodal integration of auditory and visual non-verbal emotional information. The present study investigated this process using event-related fMRI. Behavioural data revealed that audiovisual presentation of non-verbal emotional information resulted in a significant increase in correctly classified stimuli when compared with visual and auditory stimulation. This behavioural gain was paralleled by enhanced activation in bilateral posterior superior temporal gyrus (pSTG) and right thalamus, when contrasting audiovisual to auditory and visual conditions. Further, a characteristic of these brain regions, substantiating their role in the emotional integration process, is a linear relationship between the gain in classification accuracy and the strength of the BOLD response during the bimodal condition. Additionally, enhanced effective connectivity between audiovisual integration areas and associative auditory and visual cortices was observed during audiovisual stimulation, offering further insight into the neural process accomplishing multimodal integration. Finally, we were able to document an enhanced sensitivity of the putative integration sites to stimuli with emotional non-verbal content as compared to neutral stimuli.

Pre-stimulus alpha rhythms are correlated with post-stimulus sensorimotor performance in athletes and non-athletes: A high-resolution EEG study

Objective In this study, we tested the hypothesis that a pre-stimulus brief (1 min) 10-Hz audio-visual flickering stimulation modulates alpha EEG rhythms and cognitive-motor performance in elite athletes and in non-athletes during visuo-spatial demands. Methods Electroencephalographic (EEG) data were recorded (56 channels; EB-Neuro) in 14 elite fencing athletes and in 14 non-athletes during visuo-spatial-motor demands (i.e. subjects had to react to pictures of fencing and karate attacks). The task was performed after pre-stimulus 15- (placebo) or 10-Hz (experimental) flickering audio-visual stimulation lasting 1 min and after no stimulation (baseline). Results With reference to the baseline condition, only the 10-Hz stimulation induced a negative correlation between pre-stimulus alpha power and reaction time in the fencing athletes and non-athletes as a single group. The higher the enhancement of alpha power before the pictures, the stronger the improvement of the reaction time. The maximum effects were observed in right posterior parietal area (P4 electrode) overlying sensorimotor integrative cortex. Similar results were obtained in a control experiment in which eight elite karate subjects had to react to pictures of karate and basket attacks. Conclusions The present results suggest that a preliminary 10-Hz sensory stimulation can modulate EEG alpha rhythms and sensorimotor performance in both elite athletes and non-athletes engaged in visuo-spatial-motor demands. Significance Identification of the EEG state of sporting experts prior to their performance provides a plausible rationale for the modulation of alpha rhythms to enhance sporting performance in athletes and sensorimotor performance in patients to be rehabilitated.

A Cartesian Time–Frequency Approach to Reveal Brain Interaction Dynamics

The study of large-scale interactions from magnetoencephalographic data based on the magnitude of the complex coherence computed at channel level is a widely used method to track the coupling between neural signals. Traditionally, a measure based on the magnitude of the complex coherence estimated by Fourier analysis, has been used under the assumption that the neural signals are stationary. Here, we split the complex coherence in its real and imaginary parts and focus on the latter with the advantage that the imaginary part is insensitive to spurious connectivity resulting from volume conducted "self interaction". Furthermore, interacting sources alone contribute to a non-vanishing imaginary part of the complex coherence whereas the contribute of non-interacting sources is also mapped from the magnitude of the complex coherence. Since it has been extensively shown that non-stationary stochastic processes contribute to the generation of neural signals, it is fundamental to be able to define interaction measures that are able to follow the temporal variations in the coupling between neural signals. To this purpose time-frequency domain techniques to estimate the magnitude of the complex coherence have been developed in the past decades. Similarly, we extend the analysis of the imaginary part of complex coherence to the time-frequency domain, by using the short-time Fourier transform to analyze the complex coherence as a function of time. In this way, it is possible to get an indication about the dynamic of the underlying source interaction pattern by looking at channel level interactions without the bias introduced by artifactual self-interaction by volume conduction or by the contribute of non-interacting sources. Furthermore, the corresponding imaginary part of the cross-spectrogram can be used to estimate interactions on a source level by localizing pools of sources interacting at a given frequency and by characterizing their dynamics. The method has been applied to magnetoencephalographic data from a cross-modal visual auditory stimulation and provided evidence for the involvement of temporal and occipital areas in the integrated information processing for simultaneous audio-visual stimulation. Furthermore, the source interaction pattern shows a variation in time that reflects a dynamical synchronization of the involved brain sources in the frequency bands of interest.

A test of the efficacy of the MC Square device for improving verbal memory, learning and attention

Cognitive enhancement devices have been supported by positive anecdotal reports, but generally have not undergone rigorous testing. In the following report we tested one such device, the MC Square, which uses Audio-Visual Stimulation (AVS) (synchronised pulsed tones and flickering lights set at an alpha or theta frequency) to entrain neural activity. Its effect on three key cognitive functions (verbal learning, memory, and attention) was tested following a regimen of training with the device. A double blind, placebo controlled (sham device), and crossover design was utilised with pre- and post-testing on the cognitive measures occurring during each phase of the crossover. The primary hypothesis was that after training with the MC Square there would be improvement in verbal memory, associative learning, working memory and attention/concentration. Results showed a statistically reliable improvement on the measure of attention/concentration, the Digit Span Forwards test, following MC Square training. The data suggest the MC Square device provides modest enhancement in the ability to focus, attend, and report information over the short term.

The Effects of Spinal Cord Injury on Psychogenic Sexual Arousal in Males

We determined if the degree of preservation of sensory function in the T11-L2 dermatomes could be used to determine the potential for psychogenic erectile responses in men with spinal cord injury. Subjects included 45 men with spinal cord injury and 16 able-bodied control subjects. A 78-minute laboratory based analysis was done of subject subjective arousal, penile circumference, blood pressure, and heart rate responses to audiovisual erotic and audiovisual erotic combined with manual penile stimulation. Able-bodied subjects generally had significantly greater penile circumferences than spinal cord injured subjects during the stimulation periods. The degree of preservation of combined pinprick and light touch sensation in the T11-L2 dermatomes distinguished those who did and did not have a significant increase in penile circumference with audiovisual stimulation. Blood pressure and heart rate readings were generally higher in able-bodied than spinal cord injured subjects throughout the experimental protocol. However, all readings were within normal limits. Results support the hypothesis that psychogenic erection depends on the sympathetic nervous system. Findings underscore a possible parallel in neurological control of sexual responses between the sexes.

Human brain activation elicited by the localization of sounds delivering at attended or unattended positions: an fMRI/MEG study

The visual meridian effect, as a consequence of programming the necessary ocular movements for visual object localization (Rizzolatti et al. 1987), was interpreted as an evidence of a supramodal network for visual attention to space, which may be critical for reorienting the focus of attention toward stimuli appearing at unattended locations. This hypothesis was confirmed in a behavioural study also for auditory and for bimodal audio–visual stimulation (Ferlazzo et al. 2001). The present study is aimed at observing: the cerebral circuit underling the auditory meridian effect; whether this circuit is related with the network involved in reorienting the focus of attention toward stimuli appearing at an unattended location; whether a functional temporal dynamics can be defined in these areas.