Simultaneous Visual Stimuli Research Articles

A Treatment Evaluation of Successive and Simultaneous Visual Stimulus Presentation During Tact Training with Children with Autism

A Treatment Evaluation of Successive and Simultaneous Visual Stimulus Presentation During Tact Training with Children with Autism

Multisensory-driven facilitation within the peripersonal space is modulated by the expectations about stimulus location on the body

Compelling evidence from human and non-human studies suggests that responses to multisensory events are fastened when stimuli occur within the space surrounding the bodily self (i.e., peripersonal space; PPS). However, some human studies did not find such effect. We propose that these dissonant voices might actually uncover a specific mechanism, modulating PPS boundaries according to sensory regularities. We exploited a visuo-tactile paradigm, wherein participants provided speeded responses to tactile stimuli and rated their perceived intensity while ignoring simultaneous visual stimuli, appearing near the stimulated hand (VTNear) or far from it (VTFar; near the non-stimulated hand). Tactile stimuli could be delivered only to one hand (unilateral task) or to both hands randomly (bilateral task). Results revealed that a space-dependent multisensory enhancement (i.e., faster responses and higher perceived intensity in VTNear than VTFar) was present when highly predictable tactile stimulation induced PPS to be circumscribed around the stimulated hand (unilateral task). Conversely, when stimulus location was unpredictable (bilateral task), participants showed a comparable multisensory enhancement in both bimodal conditions, suggesting a PPS widening to include both hands. We propose that the detection of environmental regularities actively shapes PPS boundaries, thus optimizing the detection and reaction to incoming sensory stimuli.

Performance enhancement of wearable instrumentation for AR-based SSVEP BCI

This work addresses an innovative processing strategy to improve the classification of Steady-State Visually Evoked Potentials (SSVEPs). This strategy resorts to the combined use of fast Fourier transform and Canonical Correlation Analysis in time domain, and manages to outperform by over 5% previous results obtained for highly wearable, single-channel Brain–Computer Interfaces. In fact, a classification accuracy of 90% is reached with only 2-s time response. Then, the proposed algorithm is employed for an experimental characterization of three different Augmented Reality (AR) devices (namely, Microsoft Hololens I, Epson Moverio BT-350, and Oculus Rift S). These devices are used to generate the flickering stimuli necessary to the SSVEP induction. Also, in the three pieces of instrumentation under test, the number of simultaneous visual stimuli was increased with respect to the state-of-art solutions. The aim of the experimental characterization was to evaluate the influence of different AR technologies on the elicitation of user’s SSVEPs. Classification accuracy, time response, and information transfer rate were used as figures of merit on nine volunteers for each piece of instrumentation. Experimental results show that choosing an adequate AR headset is crucial for obtaining satisfying performance: in fact, it can be observed that the classification accuracy obtained with Microsoft Hololens is about 20% greater than Epson Moverio one.

What studies of audio-visual integration do not teach us about audio-visual integration

Auditory perception depends on more than just the processing of acoustic stimuli. Visual stimuli can also have a profound influence on listening. Salient examples of such effects include spatial ventriloquism—in which the location percept of an auditory stimulus is “captured” by that of a simultaneous visual stimulus—as well as drastically improved understanding of speech in noise when the talker's face is visible to the listener. These phenomena are typically described as “audio-visual integration,” and are often well modeled, as in the case of the ventriloquist effect, by ideal Bayesian causal inference. However, there is an over-reliance in these studies on single pairs of stimuli (i.e., one auditory and one visual stimulus) and the nature in which cross-modal discrepancies are resolved. This talk will first discuss two problems resulting from that: first, there is ambiguity about whether the integration occurs from weighing two independent sensory estimates or a single bound percept, and second, the design is less useful for studying integration when the stimuli are congruent. The talk will then describe recent work from our lab focused on new designs using multiple stimuli in an attempt to alleviate these issues and inform better models of integration.

Simultaneous and sequential subitizing are separate systems, and neither predicts math abilities

Small quantities of visual objects can be rapidly estimated without error, a phenomenon known as subitizing. Larger quantities can also be rapidly estimated, but with error, and the error rate predicts math abilities. This study addressed two issues: (a) whether subitizing generalizes over modalities and stimulus formats and (b) whether subitizing correlates with math abilities. We measured subitizing limits in primary school children and adults for visual and auditory stimuli presented either sequentially (sequences of flashes or sounds) or simultaneously (visual presentations, dot arrays). The results show that (a) subitizing limits for adults were one item larger than those for primary school children across all conditions; (b) subitizing for simultaneous visual stimuli (dots) was better than that for sequential stimuli; (c) subitizing limits for dots do not correlate with subitizing limits for either flashes or sounds; (d) subitizing of sequences of flashes and subitizing of sequences of sounds are strongly correlated with each other in children; and (e) regardless of stimuli sensory modality and format, subitizing limits do not correlate with mental calculation or digit magnitude knowledge proficiency. These results suggest that although children can subitize sequential numerosity, simultaneous and temporal subitizing may be subserved by separate systems. Furthermore, subitizing does not seem to be related to numerical abilities.

Covert Exogenous Cross-Modality Orienting between Audition and Vision

Control of visual attention by auditory stimuli is explored in seven previously unpublished experiments that were presented at conferences in the late 1980s. Reaction time (RT) to luminance targets was found to be affected by the spatial congruence between the target and a preceding or simultaneous, and non-informative, auditory event, suggesting that localizable auditory stimuli exogenously (rapidly and automatically) capture visual attention. These cuing effects were obtained in the absence of eye movements and do not appear to be mediated merely by criterion adjustments. When the information value of the auditory event was placed in conflict with its location (i.e., a tone on the right indicated that the visual target was likely to appear on the left), it was found that at short stimulus onset asynchronies (SOAs) reaction time (RT) was faster for targets at the stimulated location, an effect that disappeared within 500 ms and was reversed by 1000 ms. This demonstrates that it requires over 500 ms for endogenous orienting in response to probabilistic information about target location to overcome the powerful exogenous control of visual attention by localizable auditory stimulation. Simple RT to auditory stimuli was unaffected by the spatial congruence of a preceding or simultaneous visual stimulus. When uninformative, neither pitch contours (rising/falling tones) nor pitch (high/low tones) produced significant visual orienting along the vertical midline. When the direction of a pitch contour indicated the likely location of a visual target, participants were able to shift their attention if the relation between the natural meaning and the probabilistic information was compatible (e.g., rising contour signaled that a upper target was likely) but not when it was incompatible. The relation of these 30-year-old experiments to contemporary findings and ideas is discussed.

Neuro-oscillatory phase alignment drives speeded multisensory response times: an electro-corticographic investigation.

Even simple tasks rely on information exchange between functionally distinct and often relatively distant neuronal ensembles. Considerable work indicates oscillatory synchronization through phase alignment is a major agent of inter-regional communication. In the brain, different oscillatory phases correspond to low- and high-excitability states. Optimally aligned phases (or high-excitability states) promote inter-regional communication. Studies have also shown that sensory stimulation can modulate or reset the phase of ongoing cortical oscillations. For example, auditory stimuli can reset the phase of oscillations in visual cortex, influencing processing of a simultaneous visual stimulus. Such cross-regional phase reset represents a candidate mechanism for aligning oscillatory phase for inter-regional communication. Here, we explored the role of local and inter-regional phase alignment in driving a well established behavioral correlate of multisensory integration: the redundant target effect (RTE), which refers to the fact that responses to multisensory inputs are substantially faster than to unisensory stimuli. In a speeded detection task, human epileptic patients (N = 3) responded to unisensory (auditory or visual) and multisensory (audiovisual) stimuli with a button press, while electrocorticography was recorded over auditory and motor regions. Visual stimulation significantly modulated auditory activity via phase reset in the delta and theta bands. During the period between stimulation and subsequent motor response, transient synchronization between auditory and motor regions was observed. Phase synchrony to multisensory inputs was faster than to unisensory stimulation. This sensorimotor phase alignment correlated with behavior such that stronger synchrony was associated with faster responses, linking the commonly observed RTE with phase alignment across a sensorimotor network.

Presentation-order effects for aesthetic stimulus preference

For preference comparisons of paired successive musical excerpts, Koh (American Journal of Psychology, 80, 171-185, 1967) found time-order effects (TOEs) that correlated negatively with stimulus valence-the first (vs. the second) of two unpleasant (vs. two pleasant) excerpts tended to be preferred. We present three experiments designed to investigate whether valence-level-dependent order effects for aesthetic preference (a) can be accounted for using Hellström's (e.g., Journal of Experimental Psychology: Human Perception and Performance, 5, 460-477, 1979) sensation-weighting (SW) model, (b) can be generalized to successive and to simultaneous visual stimuli, and (c) vary, in accordance with the stimulus weighting, with interstimulus interval (ISI; for successive stimuli) or stimulus duration (for simultaneous stimuli). Participants compared paired successive jingles (Exp. 1), successive color patterns (Exp. 2), and simultaneous color patterns (Exp. 3), selecting the preferred stimulus. The results were well described by the SW model, which provided a better fit than did two extended versions of the Bradley-Terry-Luce model. Experiments 1 and 2 revealed higher weights for the second stimulus than for the first, and negatively valence-level-dependent TOEs. In Experiment 3, there was no laterality effect on the stimulus weighting and no valence-level-dependent space-order effects (SOEs). In terms of the SW model, the valence-level-dependent TOEs can be explained as a consequence of differential stimulus weighting in combination with stimulus valence varying from low to high, and the absence of valence-level-dependent SOEs as a consequence of the absence of differential weighting. For successive stimuli, there were no important effects of ISI on weightings and TOEs, and, for simultaneous stimuli, duration had only a small effect on the weighting.

Multisensory processing and neural oscillatory responses: separation of visuotactile congruency effect and corresponding electroencephalogram activities

By using electroencephalogram, we aimed at identifying the neural oscillations corresponding to two subcomponents of the visuotactile congruency process, facilitation and interference effects. The participants identified the location of the tactile stimulus presented at their index finger or thumb while ignoring simultaneous visual stimuli presented at the same (congruent) or different (incongruent) finger. Significant differences in reaction time were found between the congruent or incongruent condition and the unimodal baseline condition, indicating two different effects of visuotactile interaction. Electroencephalogram results showed significant increases in the gamma-band response in the congruent condition and in the theta-band response in the incongruent condition. These results suggest two systems of multisensory interaction.

Utilizing the ventriloquism-effect to investigate audio-visual binding

Audio-visual binding – as subset of crossmodal integration – describes the combination of information across both these senses to the subjective unified perception of a bound object. We investigated audio-visual binding by using the ventriloquism-effect (localization of a sound is biased towards and by a simultaneous visual stimulus) to act as an indicator for perceived binding. Simple visual and auditory stimuli were presented synchronously or asynchronously. fMRI was recorded during task performance ( n = 19 subjects) in order to reveal activation in areas discussed to be involved in multisensory processing in the literature. Contrasting trials with reported ventriloquism-effect versus the no-binding condition revealed activation in the insula, superior temporal sulcus and parieto-occipital sulcus. Implementing the ventriloquism-effect allows us to relate these activations to consciousness-related processes, which probably are different from stimulus-driven multisensory integration in subcortical areas.

Action and perception: Evidence against converging selection processes

In a series of experiments we investigated whether identification of a lateralized visual target would benefit from concurrent execution of a reaching movement on the same side of space. Participants were tested in a dual-task paradigm. In one task, they performed a speeded reach movement towards a lateralized target button. The reach was cued by an auditory stimulus, and performed out of the participant's sight. In the other task, participants identified one of two simultaneous visual stimuli presented to the left and right visual fields, close to movement target locations. If motor activity were effective in modulating perceptual processes via a visuo-attentional shift, identification performance should have improved when the visual stimulus appeared at the movement target location. In fact, identification was not affected by the side of reach. Such results suggest substantially independent selection processes in motor and visual domains.

Visual peripersonal space centred on the face in humans.

A convergent series of studies in monkeys and man suggests that the computation of visual space is performed in several brain regions for different behavioural purposes. Among these multiple spatial areas, the ventral intraparietal cortex, the putamen and the ventral aspect of the premotor cortex (area 6) contain a system for representing visual space near the face (peripersonal space). In these cerebral areas some neurons are bimodal: they have tactile receptive fields on the face, and they can also be driven by visual stimuli located near the tactile field. The spatial correspondence between the visual and tactile receptive fields provides a map of near visual space coded in body-part-centred co-ordinates. In the present study we demonstrate for the first time the existence of a visual peripersonal space centred on the face in humans. In patients with right hemispheric lesions, visual stimuli delivered in the space near the ipsilesional side of the face extinguished tactile stimuli on the contralesional side (cross-modal visuotactile extinction) to the same extent as did an ipsilesional tactile stimulation (unimodal tactile extinction). Furthermore, a visual stimulus presented in the proximity of the contralesional side of the face improved the detection of a left tactile stimulus: i.e. under bilateral tactile presentation patients were more accurate to report the presence of a left tactile stimulus when a simultaneous visual stimulus was presented near the left side of the face. However, when visual stimuli were delivered far from the face, visuotactile extinction and visuotactile facilitation effects were dramatically reduced. These findings are consistent with the hypothesis of a representation of visual peripersonal space coded in bodypart-centred co-ordinates, and they provide a striking demonstration of the modularity of human visual space.

Auditory-visual interaction in the generation of saccades in man

Four normal human subjects were investigated for evidence of auditory-visual interaction in the generation of horizontal saccades. In a first experiment it was shown that the mean amplitudes of initial saccades from primary position to auditory targets were significantly affected by the simultaneous occurrence of a distracting visual stimulus. If both auditory and visual stimuli were in the same hemifield, the mean amplitude of initial saccades to a fixed buzzer position was consistently increased or decreased depending on the position of the visual stimulus. The phenomenon is felt to be analogous to the "centre-of-gravity" effect previously described for two simultaneous visual stimuli. It did not occur if visual and auditory stimuli were in opposite hemifields when a simultaneous visual stimulus caused a slight reduction of mean initial saccadic amplitude compared to the mean amplitude to buzzer alone. In this case the reduction was independent of visual stimulus position. Similar effects were seen for mean final eye positions. In a second experiment, a similar procedure was carried out, but the eyes started by looking at a point at 15 degrees eccentricity. The same pattern of auditory-visual interaction was obtained in both experiments, consistent with the concept of eye-movement related movement of modality-specific sensory "maps" which has recently been shown to occur in the superior colliculus of primates.

Detection of Simultaneous Visual Stimuli by School-Children

Significant negative correlation between the recognition correctness of geometric patterns in a 7 ± 2 task and performance on Raven's Progressive Matrices was found in game-like experiments with school-children.

Latency differences in monoptic and dichoptic shape and color decision making.

Past research based on the McCollough effect and efficacy of chromatic cues in stereopsis suggests that color information is difficult to process binocularly as compared to form information. This study evaluates this proposition using a reaction time paradigm. Subjects decided whether two simultaneous visual stimuli were the same in shape or color. With presentation to the same eye, color reaction times led shape latencies. Presenting one target to one eye and one to the other slowed color times. Delaying one target by 500 msec yielded dichoptic and monoptic color decisions faster than shape. The results are related to the relationships of color and contrast pathways and possible differences in binocular rivalry.

Effects of Training Procedures on Recall of Auditory Sequences by Children

3 groups of preschool children were trained in short-term memory for word sequences by three different methods of stimulus presentation: (1) vocalized auditory sequences, (2) vocalized sequences with simultaneous visual stimuli, (3) vocal stimuli followed immediately by visual stimuli. Four different delay intervals between stimulus and response were utilized. The word sequences were 2, 3, and 4 words in length. The duration of the delay interval and the number of words in the sequence affected performance level. Method of training did not affect performance level but did produce significantly different gains in performance over a four-day period.

An electroencephalograph study of classical conditioning.

Unanesthetized monkeys and cats bearing indwelling electrodes show electrical activity evoked by clicks and light flashes in regions such as caudate nucleus, hypothalamus, and limbic system as well as in the classically defined sensory pathways. Amplitude and duration of the evoked responses, which display a similar triphasic form wherever recorded, vary spontaneously but can be experimentally enhanced by reinforcement or reduced by extinction in a simple Pavlovian conditioning procedure. When an animal displaying large responses to click stimuli appears to attend to a simultaneous visual stimulus (distraction) the click-evoked activity is remarkably reduced. Both wave shape and lability of the responses are preserved despite paralysis produced by a curare-like drug and (at auditory cortex) cutting the auditory pathway bilaterally at the midbrain level. Relevance of these results to some previous studies on learning is discussed, along with the possible relationship of these data to the neurophysiological findings reported on anesthetized preparations.