Auditory Dominance Research Articles

Auditory dominance in motor-sensory temporal recalibration.

Perception of synchrony between one’s own action (e.g. a finger tap) and the sensory feedback thereof (e.g. a flash or click) can be shifted after exposure to an induced delay (temporal recalibration effect, TRE). It remains elusive, however, whether the same mechanism underlies motor-visual (MV) and motor-auditory (MA) TRE. We examined this by measuring crosstalk between MV- and MA-delayed feedbacks. During an exposure phase, participants pressed a mouse at a constant pace while receiving visual or auditory feedback that was either delayed (+150 ms) or subjectively synchronous (+50 ms). During a post-test, participants then tried to tap in sync with visual or auditory pacers. TRE manifested itself as a compensatory shift in the tap–pacer asynchrony (a larger anticipation error after exposure to delayed feedback). In experiment 1, MA and MV feedback were either both synchronous (MV-sync and MA-sync) or both delayed (MV-delay and MA-delay), whereas in experiment 2, different delays were mixed across alternating trials (MV-sync and MA-delay or MV-delay and MA-sync). Exposure to consistent delays induced equally large TREs for auditory and visual pacers with similar build-up courses. However, with mixed delays, we found that synchronized sounds erased MV-TRE, but synchronized flashes did not erase MA-TRE. These results suggest that similar mechanisms underlie MA- and MV-TRE, but that auditory feedback is more potent than visual feedback to induce a rearrangement of motor-sensory timing.

Semantic congruency and the (reversed) Colavita effect in children and adults

When presented with auditory, visual, or bimodal audiovisual stimuli in a discrimination task, adults tend to ignore the auditory component in bimodal stimuli and respond to the visual component only (i.e., Colavita visual dominance effect). The same is true for older children, whereas young children are dominated by the auditory component of bimodal audiovisual stimuli. This suggests a change of sensory dominance during childhood. The aim of the current study was to investigate, in three experimental conditions, whether children and adults show sensory dominance when presented with complex semantic stimuli and whether this dominance can be modulated by stimulus characteristics such as semantic (in)congruency, frequency of bimodal trials, and color information. Semantic (in)congruency did not affect the magnitude of the auditory dominance effect in 6-year-olds or the visual dominance effect in adults, but it was a modulating factor of the visual dominance in 9-year-olds (Conditions 1 and 2). Furthermore, the absence of color information (Condition 3) did not affect auditory dominance in 6-year-olds and hardly affected visual dominance in adults, whereas the visual dominance in 9-year-olds disappeared. Our results suggest that (a) sensory dominance in children and adults is not restricted to simple lights and sounds, as used in previous research, but can be extended to semantically meaningful stimuli and that (b) sensory dominance is more robust in 6-year-olds and adults than in 9-year-olds, implying a transitional stage around this age.

Multisensory Competition Is Modulated by Sensory Pathway Interactions with Fronto-Sensorimotor and Default-Mode Network Regions.

Multisensory information competes for preferential access to consciousness. It remains unknown what neural processes cause one particular modality to win multisensory competition and eventually dominate behavior. Thus, in a paradigm in which human participants sought to make simultaneous auditory and visual detection responses, we sought to identify prestimulus and poststimulus neural signals that were associated with auditory and visual dominance on each trial. Behaviorally, visual detection responses preceded auditory responses more frequently than vice versa. Even when visual responses were preceded by auditory responses, they recovered more quickly from previous responses, indicating the dominance of vision over audition. Neurally, visual precedence was associated with increased prestimulus activity in the prefrontal cortex and reduced prestimulus activity in the default-mode network, and increased poststimulus connectivity between the prefrontal cortex and the visual system. Moreover, the dorsal visual stream showed not only increased activity in post-perceptual phases but also enhanced connectivity with the sensorimotor cortex, indicating the functional role of the dorsal visual stream in prioritizing the flow of visual information into the motor system. In contrast, auditory precedence was associated with increased prestimulus activity in the auditory cortex and increased poststimulus neural coupling between the auditory and the sensorimotor cortex. Finally, whenever one modality lost multisensory competition, the corresponding sensory cortex showed enhanced connectivity with the default-mode network. Overall, the outcome of audiovisual competition depended on dynamic interactions between sensory systems and both the fronto-sensorimotor and the default-mode network. Together, these results revealed both the neural causes and the neural consequences of visual and auditory dominance during multisensory competition.

Hedonics and the “Selective Associations” Biological Constraint on Learning

Research concerned with visual dominance in appetitive and auditory dominance in aversive learning situations (selective associations) is reviewed. The present analysis stresses that the dominant sensory modality of stimulus control is determined by the relative affective valence acquired by a compound auditory-visual stimulus through reinforcement contingencies, rather than by whether the primary reinforcer is appetitive or aversive. For example, take two groups of rats or pigeons on exactly the same shock-avoidance contingency in a tone-light compound (TL), but with different contingencies when the compound is absent (TL). Responding came predominantly under (1) auditory control when conditions in TL were hedonically negative relative to those in (TL), and (2) visual control when conditions in (TL) made TL relatively positive. Selective associations here are a product of the relative hedonic state, positive or negative, established to the auditory-visual compound. Therefore, this constraint reflects a high level of functioning by a hedonic comparator -- with TL’s hedonic value contextually determined by the totality of the events encountered, and reinforcement contingencies, operating in its world. The physical particulars of the reinforcer in TL here, shock avoidance, clearly were not responsible for the hedonic psychological state TL produced. Weiss, Panlilio, and Schindler (1993a, 1993b) went on to show that these proclivities can be (1) reversed, and (2) overcome by a blocking design when the biologically-contingency-disadvantaged stimulus is first pretrained on its own. Relating the “hedonic model” to evolution is speculative. But, the hedonic model is scientifically integrative by relating this biological constraint to a variety of phenomena that involve incentive-motivational states. These include choice behavior, conditioned preference, behavioral contrast and appetitive-aversive interactions.

Evidence of multisensory plasticity: Asymmetrical medial geniculate body in people with one eye.

The medial geniculate body (MGB) plays a central role in auditory processing with both efferent and afferent tracts to primary auditory cortex. People who have lost one eye early in life have enhanced sound localization, lack visual over auditory dominance and integrate auditory and visual information optimally, similar to controls, despite taking longer to localize unimodal visual stimuli. Compared to controls, people with one eye have decreased lateral geniculate nuclei (LGN) volume as expected given the 50% deafferentation of the visual system. However, LGN volume is larger than predicted contralateral to the remaining eye, indicating altered structural development likely through recruitment of deafferented LGN cells.Purpose: the current study investigated whether structural MGB changes are also present in this group given the changes they exhibit in auditory processing.Methods: MGB volumes were measured in adults who had undergone early unilateral eye enucleation and were compared to binocularly intact controls.Results: unlike controls, people with one eye had a significant asymmetry with a larger left compared to right MGB, independent of eye of enucleation. MGB volume correlated positively with LGN volume in people with one eye.Conclusions: volume asymmetry in the MGB in people with one eye may represent increased interactions between the left MGB and primary auditory cortex. This interaction could contribute to increased auditory and other left hemisphere-dominant processing, including language, as compensation for the loss of one half of visual inputs early in life. The positive correlation between MGB and LGN volume is not due to space constraints but rather indicates increased plasticity in both auditory and visual sensory systems following early eye enucleation.

Crossmodal attention switching: Auditory dominance in temporal discrimination tasks

Visual stimuli are often processed more efficiently than accompanying stimuli in another modality. In line with this “visual dominance”, earlier studies on attentional switching showed a clear benefit for visual stimuli in a bimodal visual–auditory modality-switch paradigm that required spatial stimulus localization in the relevant modality. The present study aimed to examine the generality of this visual dominance effect. The modality appropriateness hypothesis proposes that stimuli in different modalities are differentially effectively processed depending on the task dimension, so that processing of visual stimuli is favored in the dimension of space, whereas processing auditory stimuli is favored in the dimension of time. In the present study, we examined this proposition by using a temporal duration judgment in a bimodal visual–auditory switching paradigm. Two experiments demonstrated that crossmodal interference (i.e., temporal stimulus congruence) was larger for visual stimuli than for auditory stimuli, suggesting auditory dominance when performing temporal judgment tasks. However, attention switch costs were larger for the auditory modality than for visual modality, indicating a dissociation of the mechanisms underlying crossmodal competition in stimulus processing and modality-specific biasing of attentional set.

Audition dominates vision in duration perception irrespective of salience, attention, and temporal discriminability.

Whereas the visual modality tends to dominate over the auditory modality in bimodal spatial perception, the auditory modality tends to dominate over the visual modality in bimodal temporal perception. Recent results suggest that the visual modality dominates bimodal spatial perception because spatial discriminability is typically greater for the visual than for the auditory modality; accordingly, visual dominance is eliminated or reversed when visual-spatial discriminability is reduced by degrading visual stimuli to be equivalent or inferior to auditory spatial discriminability. Thus, for spatial perception, the modality that provides greater discriminability dominates. Here, we ask whether auditory dominance in duration perception is similarly explained by factors that influence the relative quality of auditory and visual signals. In contrast to the spatial results, the auditory modality dominated over the visual modality in bimodal duration perception even when the auditory signal was clearly weaker, when the auditory signal was ignored (i.e., the visual signal was selectively attended), and when the temporal discriminability was equivalent for the auditory and visual signals. Thus, unlike spatial perception, where the modality carrying more discriminable signals dominates, duration perception seems to be mandatorily linked to auditory processing under most circumstances.

Crossmodal Binding Rivalry: An Alternative Hypothesis for the Double Flash Illusion

Extensive research on multisensory processing has established that temporal and/or spatial proximity of sensory information lead to the percept of a unified multisensory event, both at a behavioral and neuronal level (e.g., Stein, Huneycutt, & Meredith, 1988). Binding of multiple sensory inputs has also been demonstrated for stimuli presented with a certain degree of temporal disparity (e.g., Vatakis & Spence, 2010). A classical example of crossmodal interaction is the well-known sound induced flash illusion (SIFI; whereby a brief flash paired with two auditory beeps is actually perceived as two distinct flashes; Shams, Kamitani, & Shimojo, 2000). SIFI is considered an example of auditory dominance, where auditory stimulation modulates visual perception for audiovisual presentations that fall within the temporal window of integration. Studies on the SIFI (Andersen, Tiipana, & Sams, 2004; Shams, Kamitani, & Shimojo, 2002) have shown diminished performance in 1 flash-2 beeps (SIFI illusion) and 2 flash-1 beep presentations (considered different from SIFI but as yet not elucidated), while the performance in 1 flash-1 beep is excellent. That is, the close in time-space presence of 2 versus 1 input from different sensory modalities affects participant performance, while this is not the case for presentations of equal number of sensory stimulus inputs. We claim that the diminished performance in 1 flash-2 beeps and 2 flash-1 beep conditions are not two different illusions but they both represent examples of the crossmodal binding rivalry between the unequal number of sensory inputs presented. That is, presentations of multiple sensory inputs in close spatial and temporal proximity lead to a rivalry between the sensory inputs that are to be integrated. This rivalry will be weaker or stronger depending on a number of findings related to multisensory integration.As has been previously shown, a unified multisensory percept is more robust if the visual input is presented slightly before or in synchrony with the auditory input (Keetels & Vroomen, 2012; van Wassenhove, Grant, & Poeppel, 2007; Vatakis & Spence, 2007, 2008). In cases where the auditory input precedes the visual, binding is weaker leading to a less integrated percept. Moreover, binding is highly dependent on timing with temporally proximal presentations taking precedence over distal presentations (e.g., Vatakis & Spence, 2010). Thus, presentations of asynchronous stimuli even if presented within the temporal window of integration represent binding types of different strength with synchronous presentations being the ones leading to higher binding. These findings drive our crossmodal binding rivalry hypothesis and we support that the rivalry between the unequal number of sensory inputs will vary according to their binding robustness. So, for inputs where the visual is in synchrony or leading the auditory input, the binding is robust leading to a stronger rivalry with the spare stimulus. This rivalry results to a lower percent of illusory percepts and slower reaction times. On the other hand, if the binding between the auditory and visual inputs is weak, then the rivalry between them and the spare stimulus is less intense, thus resulting in quicker responding and higher illusory experiences. Generally, during illusion conditions, we expect to have slower reaction times than in conditions with equal number of visual and auditory inputs and in bimodal conditions (equal number of inputs) more accurate responses than in unimodal conditions.We have tested directly the rivalry hypothesis by utilizing the classical SIFI but with multiple timing presentations (never tested before in one experimental set-up). More specifically, we have used 0, 25, 50, and 100ms onset asynchronies of auditory beep before and after the visual flash. Illusion conditions and test conditions were intermixed in order to avoid biased responding (in terms of the number of flashes) and to be sure that the task is not too difficult for the participants to carry out. The proposed project will allow us to evaluate the rivalry hypothesis for multiple audiovisual inputs, which will provide a common explanation for both 1 flash-2 beeps and 2 flash-1 beep presentations, while at the same time it will allow the revisiting of the role of auditory dominance in the double flash illusion.

When Audition Dominates Vision

Presenting information to multiple sensory modalities sometimes facilitates and sometimes interferes with processing of this information. Research examining interference effects shows that auditory input often interferes with processing of visual input in young children (i.e., auditory dominance effect), whereas visual input often interferes with auditory processing in adults (i.e., visual dominance effect). The current study used a cross-modal statistical learning task to examine modality dominance in adults. Participants ably learned auditory and visual statistics when auditory and visual sequences were presented unimodally and when auditory and visual sequences were correlated during training. However, increasing task demands resulted in an important asymmetry: Increased task demands attenuated visual statistical learning, while having no effect on auditory statistical learning. These findings are consistent with auditory dominance effects reported in young children and have important implications for our understanding of how sensory modalities interact while learning the structure of cross-modal information.

Vision Dominates at the Preresponse Level and Audition Dominates at the Response Level in Cross-modal Interaction: Behavioral and Neural Evidence

There are ongoing debates on the direction of sensory dominance in cross-modal interaction. In the present study, we demonstrate that the specific direction of sensory dominance depends on the level of processing: vision dominates at earlier stages, whereas audition dominates at later stages of cognitive processing. Moreover, these dominances are subserved by different neural networks. In three experiments, human participants were asked to attend to either visual or auditory modality while ignoring simultaneous stimulus inputs from the other modality. By manipulating three levels of congruency between the simultaneous visual and auditory inputs, congruent (C), incongruent at preresponse level (PRIC), and incongruent at response level (RIC), we differentiated the cross-modal conflict explicitly into preresponse (PRIC > C) and response (RIC > PRIC) levels. Behavioral data in the three experiments consistently suggested that visual distractors caused more interference to auditory processing than vice versa (i.e., the typical visual dominance) at the preresponse level, but auditory distractors caused more interference to visual processing than vice versa (i.e., the typical auditory dominance) at the response level regardless of experimental tasks, types of stimuli, or differential processing speeds in different modalities. Dissociable neural networks were revealed, with the default mode network being involved in the visual dominance at the preresponse level and the prefrontal executive areas being involved in the auditory dominance at the response level. The default mode network may be attracted selectively by irrelevant visual, rather than auditory, information via enhanced neural coupling with the ventral visual stream, resulting in visual dominance at the preresponse level.

Neural interactions in unilateral colliculus and between bilateral colliculi modulate auditory signal processing

In the auditory pathway, the inferior colliculus (IC) is a major center for temporal and spectral integration of auditory information. There are widespread neural interactions in unilateral (one) IC and between bilateral (two) ICs that could modulate auditory signal processing such as the amplitude and frequency selectivity of IC neurons. These neural interactions are either inhibitory or excitatory, and are mostly mediated by γ-aminobutyric acid (GABA) and glutamate, respectively. However, the majority of interactions are inhibitory while excitatory interactions are in the minority. Such unbalanced properties between excitatory and inhibitory projections have an important role in the formation of unilateral auditory dominance and sound location, and the neural interaction in one IC and between two ICs provide an adjustable and plastic modulation pattern for auditory signal processing.

Changes in Sensory Dominance During Childhood: Converging Evidence From the Colavita Effect and the Sound‐Induced Flash Illusion

In human adults, visual dominance emerges in several multisensory tasks. In children, auditory dominance has been reported up to 4years of age. To establish when sensory dominance changes during development, 41 children (6-7, 9-10, and 11-12years) were tested on the Colavita task (Experiment 1) and 32 children (6-7, 9-10, and 11-12years) were tested on the sound-induced flash illusion (Experiment 2). In both experiments, an auditory dominance emerged in 6- to 7-year-old children compared to older children. Adult-like visual dominance started to emerge from 9 to 10years of age, and consolidated in 11- to 12-year-old children. These findings show that auditory dominance persists up to 6years, but switches to visual dominance during the first school years.

Perceptual learning in temporal discrimination: asymmetric cross-modal transfer from audition to vision

This study assessed possible cross-modal transfer effects of training in a temporal discrimination task from vision to audition as well as from audition to vision. We employed a pretest-training-post-test design including a control group that performed only the pretest and the post-test. Trained participants showed better discrimination performance with their trained interval than the control group. This training effect transferred to the other modality only for those participants who had been trained with auditory stimuli. The present study thus demonstrates for the first time that training on temporal discrimination within the auditory modality can transfer to the visual modality but not vice versa. This finding represents a novel illustration of auditory dominance in temporal processing and is consistent with the notion that time is primarily encoded in the auditory system.

Synchronization with competing visual and auditory rhythms: bouncing ball meets metronome

Synchronization of finger taps with periodically flashing visual stimuli is known to be much more variable than synchronization with an auditory metronome. When one of these rhythms is the synchronization target and the other serves as a distracter at various temporal offsets, strong auditory dominance is observed. However, it has recently been shown that visuomotor synchronization improves substantially with moving stimuli such as a continuously bouncing ball. The present study pitted a bouncing ball against an auditory metronome in a target-distracter synchronization paradigm, with the participants being auditory experts (musicians) and visual experts (video gamers and ball players). Synchronization was still less variable with auditory than with visual target stimuli in both groups. For musicians, auditory stimuli tended to be more distracting than visual stimuli, whereas the opposite was the case for the visual experts. Overall, there was no main effect of distracter modality. Thus, a distracting spatiotemporal visual rhythm can be as effective as a distracting auditory rhythm in its capacity to perturb synchronous movement, but its effectiveness also depends on modality-specific expertise.

Re-evaluating visual and auditory dominance through modality switching costs and congruency analyses

Competition between the senses can lead to modality dominance, where one sense influences multi-modal processing to a greater degree than another. Modality dominance can be influenced by task demands, speeds of processing, contextual influence and practice. To resolve previous discrepancies in these factors, we assessed modality dominance in an audio-visual paradigm controlling for the first three factors while manipulating the fourth. Following a uni-modal task in which auditory and visual processing were equated, participants completed a pre-practice selective attention bimodal task in which the congruency relationship and task-relevant modality changed across trials. Participants were given practice in one modality prior to completing a post-practice selective attention bimodal task similar to the first. The effects of practice were non-specific as participants were speeded post-practice relative to pre-practice. Congruent stimuli relative to incongruent stimuli, also led to increased processing efficiency. RT data tended to reveal symmetric modality switching costs whereas the error rate data tended to reveal asymmetric modality switching costs in which switching from auditory to visual processing was particularly costly. The data suggest that when a number of safeguards are put in place to equate auditory and visual responding as far as possible, evidence for an auditory advantage can arise.

Preschool-age children and adults flexibly shift their preferences for auditory versus visual modalities but do not exhibit auditory dominance

The goal of this study was to evaluate the claim that young children display preferences for auditory stimuli over visual stimuli. This study was motivated by concerns that the visual stimuli employed in prior studies were considerably more complex and less distinctive than the competing auditory stimuli, resulting in an illusory preference for auditory cues. Across three experiments, preschool-age children and adults were trained to use paired audio–visual cues to predict the location of a target. At test, the cues were switched so that auditory cues indicated one location and visual cues indicated the opposite location. In contrast to prior studies, preschool-age children did not exhibit auditory dominance. Instead, children and adults flexibly shifted their preferences as a function of the degree of contrast within each modality, with high contrast leading to greater use.

Development of Visuo-Auditory Integration in Space and Time

Adults integrate multisensory information optimally (e.g., Ernst and Banks, 2002) while children do not integrate multisensory visual-haptic cues until 8–10 years of age (e.g., Gori et al., 2008). Before that age strong unisensory dominance occurs for size and orientation visual-haptic judgments, possibly reflecting a process of cross-sensory calibration between modalities. It is widely recognized that audition dominates time perception, while vision dominates space perception. Within the framework of the cross-sensory calibration hypothesis, we investigate visual-auditory integration in both space and time with child-friendly spatial and temporal bisection tasks. Unimodal and bimodal (conflictual and not) audio-visual thresholds and PSEs were measured and compared with the Bayesian predictions. In the temporal domain, we found that both in children and adults, audition dominates the bimodal visuo-auditory task both in perceived time and precision thresholds. On the contrary, in the visual-auditory spatial task, children younger than 12 years of age show clear visual dominance (for PSEs), and bimodal thresholds higher than the Bayesian prediction. Only in the adult group did bimodal thresholds become optimal. In agreement with previous studies, our results suggest that also visual-auditory adult-like behavior develops late. We suggest that the visual dominance for space and the auditory dominance for time could reflect a cross-sensory comparison of vision in the spatial visuo-audio task and a cross-sensory comparison of audition in the temporal visuo-audio task.

Auditory signal dominates visual in the perception of emotional social interactions

Multimodal perception of emotions has been typically examined using displays of a solitary character (e.g., the face–voice and/or body–sound of one actor). We extend investigation to more complex, dyadic point-light displays combined with speech. A motion and voice capture system was used to record twenty actors interacting in couples with happy, angry and neutral emotional expressions. The obtained stimuli were validated in a pilot study and used in the present study to investigate multimodal perception of emotional social interactions. Participants were required to categorize happy and angry expressions displayed visually, auditorily, or using emotionally congruent and incongruent bimodal displays. In a series of cross-validation experiments we found that sound dominated the visual signal in the perception of emotional social interaction. Although participants’ judgments were faster in the bimodal condition, the accuracy of judgments was similar for both bimodal and auditory-only conditions. When participants watched emotionally mismatched bimodal displays, they predominantly oriented their judgments towards the auditory rather than the visual signal. This auditory dominance persisted even when the reliability of auditory signal was decreased with noise, although visual information had some effect on judgments of emotions when it was combined with a noisy auditory signal. Our results suggest that when judging emotions from observed social interaction, we rely primarily on vocal cues from the conversation, rather then visual cues from their body movement.

Reversing the Colavita visual dominance effect

Many researchers have taken the Colavita effect to represent a paradigm case of visual dominance. Broadly defined, the effect occurs when people fail to respond to an auditory target if they also have to respond to a visual target presented at the same time. Previous studies have revealed the remarkable resilience of this effect to various manipulations. In fact, a reversal of the Colavita visual dominance effect (i.e., auditory dominance) has never been reported. Here, we present a series of experiments designed to investigate whether it is possible to reverse the Colavita effect when the target stimuli consist of repetitions embedded in simultaneously presented auditory and visual streams of stimuli. In line with previous findings, the Colavita effect was still observed for an immediate repetition task, but when an n-1 repetition detection task was used, a reversal of visual dominance was demonstrated. These results suggest that masking from intervening stimuli between n-1 repetition targets was responsible for the elimination and reversal of the Colavita visual dominance effect. They further suggest that varying the presence of a mask (pattern, conceptual, or absent) in the repetition detection task gives rise to different patterns of sensory dominance (i.e., visual dominance, an elimination of the Colavita effect, or even auditory dominance).

Familiarity of objects affects susceptibility to the sound-induced flash illusion

Audition is accepted as more reliable (thus dominant) than vision when temporal discrimination is required by the task. However, it is not known whether the characteristics of the visual stimulus, for example its familiarity to the perceiver, affect auditory dominance. In this study we manipulated familiarity of the visual stimulus in a well-established multisensory phenomenon, i.e., the sound-induced flash illusion. This illusion occurs when, for example, one brief visual stimulus (e.g., a flash) is presented in close temporal proximity with two brief sounds; participants perceive two flashes instead of one. We found that when the visual stimuli (faces or buildings) were familiar, participants were less susceptible to the illusion than when they were unfamiliar. As the illusion has been ascribed to early cross-sensory interactions between vision and audition, the present work offers behavioural evidence that high level processing of objects’ characteristics such as familiarity, affects early temporal multisensory integration. Possible mechanisms underlying the effect of familiarity are discussed.