Stimuli Of Different Modalities Research Articles

How are overlapping time intervals perceived? Evidence for a weighted sum of segments model

This study investigated the way in which people time two overlapping intervals. Timing models already proposed in the literature predict different effects of the degree of overlap on each estimate, and empirical findings were compared to these predictions. Two unimodal experiments (in which each to-be-timed interval was a visual stimulus) and one bimodal experiment (in which one to-be-timed interval was auditory and the other visual) were conducted. The estimate of the first interval was either unaffected or decreased, and the estimate of the second interval consistently increased as the intervals were more temporally separated. The only model in the literature that could account for such result patterns is a single pacemaker single accumulator structure with an additional recency weighting (see the weighted sum of segments model). That is, participants appear to segment the two overlapping intervals into three non-overlapping and overlapping segments, time these segments separately, and then combine them to estimate each interval. Importantly, a recency weighting, determined by the time that has passed since the end of that segment, is also applied to each segment in the summation process. Further, in the bimodal experiment the order in which the stimuli of different modalities were presented affected the way in which they were timed, a finding that none of the current models can explain. This highlights that a comprehensive model of interval timing must consider not only the modalities of to-be-timed intervals but also the order in which different modalities must be timed.

Уровень развития интеллекта и особенности ассоциативной деятельности на стимулы разных модальностей у детей в норме и при шизофрении

The article contains results of comparison of level of intellectual development and features of verbal association measurements of young schoolchildren in normal and schizophrenia conditions in response to stimuli of different modalities. There have been examined two selections: schizophrenic children and mentally healthy children (60 schizophrenic children and 60 mentally healthy children at the age from 7 to 11 y.o.). There have been used experimental-psychological author’s complex of methods to research verbal associations in response to stimuli of different modalities (olfactory, audio-verbal, visual, tactile), as well as battery of K-ABC tests. The research results are discussed in the context of particularity of cognitive deficiency and manifestation of cognitive dysontogenesis of young school schizophrenic children. The article contains description of various combinations of connection/ absence of connection between intelligence level (according to Kauffman’s battery of K –ABC tests) and various measurements of association activity (productiveness, temporary indications of activity, commonality coefficient). The principal research result is relative independence of particularity of association activity of schizophrenic children (according to commonality coefficient) of their cognitive development level.

Auditory and Nociceptive Stimuli Responses in the Electroencephalogram. A Non-linear Measures and Time-frequency Representation Based Analysis.

This article is part of the Focus Theme of Methods of Information in Medicine on "Biosignal Interpretation: Advanced Methods for Neural Signals and Images". An efficient way to investigate the neural basis of nociceptive responses is the analysis of the event-related brain potentials (ERPs). The main objective of this work was to study how adaptation and fatigue affect the ERPs to stimuli of different modalities, by characterizing the responses to infrequent and frequent stimulation in different recording periods. In this work, series of averaged EEG epochs recorded after thermal, electrical and auditory stimulation were analyzed with time-frequency representation and non-linear measures as spectral entropy and auto-mutual information function. The study was performed by considering the traditional EEG frequency bands. The defined measures presented a statistical significance p-value < 0.01 and accuracy higher than 60% by differentiating windows of response to infrequent (I) and frequent (F) stimuli between the start and end of the EEG recording. These measures permitted to observe some aspects of the subject's adaptation and the nociceptive response.

Crossmodal attention switching: Auditory dominance in temporal discrimination tasks

Visual stimuli are often processed more efficiently than accompanying stimuli in another modality. In line with this “visual dominance”, earlier studies on attentional switching showed a clear benefit for visual stimuli in a bimodal visual–auditory modality-switch paradigm that required spatial stimulus localization in the relevant modality. The present study aimed to examine the generality of this visual dominance effect. The modality appropriateness hypothesis proposes that stimuli in different modalities are differentially effectively processed depending on the task dimension, so that processing of visual stimuli is favored in the dimension of space, whereas processing auditory stimuli is favored in the dimension of time. In the present study, we examined this proposition by using a temporal duration judgment in a bimodal visual–auditory switching paradigm. Two experiments demonstrated that crossmodal interference (i.e., temporal stimulus congruence) was larger for visual stimuli than for auditory stimuli, suggesting auditory dominance when performing temporal judgment tasks. However, attention switch costs were larger for the auditory modality than for visual modality, indicating a dissociation of the mechanisms underlying crossmodal competition in stimulus processing and modality-specific biasing of attentional set.

Multisensory Integration in Complete Unawareness: Evidence From Audiovisual Congruency Priming

Multisensory integration is thought to require conscious perception. Although previous studies have shown that an invisible stimulus could be integrated with an audible one, none have demonstrated integration of two subliminal stimuli of different modalities. Here, pairs of identical or different audiovisual target letters (the sound /b/ with the written letter “b” or “m,” respectively) were preceded by pairs of masked identical or different audiovisual prime digits (the sound /6/ with the written digit “6” or “8,” respectively). In three experiments, awareness of the audiovisual digit primes was manipulated, such that participants were either unaware of the visual digit, the auditory digit, or both. Priming of the semantic relations between the auditory and visual digits was found in all experiments. Moreover, a further experiment showed that unconscious multisensory integration was not obtained when participants did not undergo prior conscious training of the task. This suggests that following conscious learning, unconscious processing suffices for multisensory integration.

Hand proximity facilitates spatial discrimination of auditory tones.

The effect of hand proximity on vision and visual attention has been well documented. In this study we tested whether such effect(s) would also be present in the auditory modality. With hands placed either near or away from the audio sources, participants performed an auditory-spatial discrimination (Experiment 1: left or right side), pitch discrimination (Experiment 2: high, med, or low tone), and spatial-plus-pitch (Experiment 3: left or right; high, med, or low) discrimination task. In Experiment 1, when hands were away from the audio source, participants consistently responded faster with their right hand regardless of stimulus location. This right hand advantage, however, disappeared in the hands-near condition because of a significant improvement in left hand's reaction time (RT). No effect of hand proximity was found in Experiments 2 or 3, where a choice RT task requiring pitch discrimination was used. Together, these results that the perceptual and attentional effect of hand proximity is not limited to one specific modality, but applicable to the entire “space” near the hands, including stimuli of different modality (at least visual and auditory) within that space. While these findings provide evidence from auditory attention that supports the multimodal account originally raised by Reed et al. (2006), we also discuss the possibility of a dual mechanism hypothesis to reconcile findings from the multimodal and magno/parvocellular account.

A flexible user-interface for audiovisual presentation and interactive control in neurobehavioral experiments

A major problem facing behavioral neuroscientists is a lack of unified, vendor-distributed data acquisition systems that allow stimulus presentation and behavioral monitoring while recording neural activity. Numerous systems perform one of these tasks well independently, but to our knowledge, a useful package with a straightforward user interface does not exist. Here we describe the development of a flexible, script-based user interface that enables customization for real-time stimulus presentation, behavioral monitoring and data acquisition. The experimental design can also incorporate neural microstimulation paradigms. We used this interface to deliver multimodal, auditory and visual (images or video) stimuli to a nonhuman primate and acquire single-unit data. Our design is cost-effective and works well with commercially available hardware and software. Our design incorporates a script, providing high-level control of data acquisition via a sequencer running on a digital signal processor to enable behaviorally triggered control of the presentation of visual and auditory stimuli. Our experiments were conducted in combination with eye-tracking hardware. The script, however, is designed to be broadly useful to neuroscientists who may want to deliver stimuli of different modalities using any animal model.

From duration and distance comparisons to goal encoding in prefrontal cortex.

Timing is a very abstract representation that shares with other magnitudes, such as numerosity, the peculiarity of being independent from any particular sensory modality. Not only we can time stimuli in different modalities but we can also compare the durations of different visual, auditory and somatosensory stimuli. Furthermore, even though time is not directly associated with space, and we are inclined to consider space and time as two different perceptual dimensions of our existence, an increasing number of studies challenge this idea by showing that timing and spatial processing have some relationship that involves sharing computation resources and that time may have a spatial representation. A more general theory, called theory of magnitude (ATOM), considers both timing and spatial computations, together with other magnitudes, as originating from a general magnitude system [Walsh VA, Trends Cogn Sci 7(11):483-8, 2003]. The neural underpinnings of time and its relationship to the processing of spatial information have started to be investigated only recently, but the field is rapidly growing. It is addressing the representation of time in several cortical and subcortical brain areas. Information processing of time and space are not strictly specialized in neural and cognitive mechanisms and we believe that studying them only separately may restrict our understanding of these processes. In this chapter, we will firstly introduce the role of the prefrontal cortex (PF) in coding relative durations. We will point out that the comparison of durations makes use of intermediate computations based on the order of the events. Secondly, we will describe the comparison mechanisms that are implemented by PF to make perceptual decisions about durations in relation to those involved in making decisions about spatial locations and distances. We will distinguish the decision processes from the goal choices, and we will examine which computational resources are shared between different magnitudes and which are domain-specific. We will summarize our results within the context of a more general PF function in promoting the generation of goals from the current context, consisting of domain- and modality-specific coding of stimuli of different modalities or magnitudes.

Subcortical and Neocortical Mechanisms of Visual-Somatosensory Interplay

The concept of unisensory cortices has been challenged by growing evidence that neurons in primary sensory regions respond to stimuli of different modalities ([Driver and Noesselt, 2008][1]). The traditional view is that these crossmodal influences result from backward projections from higher-order

A flexible user-interface for audiovisual presentation and interactive control in neurobehavioral experiments

A major problem facing behavioral neuroscientists is a lack of unified, vendor-distributed data acquisition systems that allow stimulus presentation and behavioral monitoring while recording neural activity. Numerous systems perform one of these tasks well independently, but to our knowledge, a useful package with a straightforward user interface does not exist. Here we describe the development of a flexible, script-based user interface that enables customization for real-time stimulus presentation, behavioral monitoring and data acquisition. The experimental design can also incorporate neural microstimulation paradigms. We used this interface to deliver multimodal, auditory and visual (images or video) stimuli to a nonhuman primate and acquire single-unit data. Our design is cost-effective and works well with commercially available hardware and software. Our design incorporates a script, providing high-level control of data acquisition via a sequencer running on a digital signal processor to enable behaviorally triggered control of the presentation of visual and auditory stimuli. Our experiments were conducted in combination with eye-tracking hardware. The script, however, is designed to be broadly useful to neuroscientists who may want to deliver stimuli of different modalities using any animal model.

A flexible user-interface for audiovisual presentation and interactive control in neurobehavioral experiments

A major problem facing behavioral neuroscientists is a lack of unified, vendor-distributed data acquisition systems that allow stimulus presentation and behavioral monitoring while recording neural activity. Numerous systems perform one of these tasks well independently, but to our knowledge, a useful package with a straightforward user interface does not exist. Here we describe the development of a flexible, script-based user interface that enables customization for real-time stimulus presentation, behavioral monitoring and data acquisition. The experimental design can also incorporate neural microstimulation paradigms. We used this interface to deliver multimodal, auditory and visual (images or video) stimuli to a nonhuman primate and acquire single-unit data. Our design is cost-effective and works well with commercially available hardware and software. Our design incorporates a script, providing high-level control of data acquisition via a sequencer running on a digital signal processor to enable behaviorally triggered control of the presentation of visual and auditory stimuli. Our experiments were conducted in combination with eye-tracking hardware. The script, however, is designed to be broadly useful to neuroscientists who may want to deliver stimuli of different modalities using any animal model.

A flexible user-interface for audiovisual presentation and interactive control in neurobehavioral experiments

A major problem facing behavioral neuroscientists is a lack of unified, vendor-distributed data acquisition systems that allow stimulus presentation and behavioral monitoring while recording neural activity. Numerous systems perform one of these tasks well independently, but to our knowledge, a useful package with a straightforward user interface does not exist. Here we describe the development of a flexible, script-based user interface that enables customization for real-time stimulus presentation, behavioral monitoring and data acquisition. The experimental design can also incorporate neural microstimulation paradigms. We used this interface to deliver multimodal, auditory and visual (images or video) stimuli to a nonhuman primate and acquire single-unit data. Our design is cost-effective and works well with commercially available hardware and software. Our design incorporates a script, providing high-level control of data acquisition via a sequencer running on a digital signal processor to enable behaviorally triggered control of the presentation of visual and auditory stimuli. Our experiments were conducted in combination with eye-tracking hardware. The script, however, is designed to be broadly useful to neuroscientists who may want to deliver stimuli of different modalities using any animal model.

Facial reactions in response to dynamic emotional stimuli in different modalities in patients suffering from schizophrenia: a behavioral and EMG study

Emotional facial expression is an important low-level mechanism contributing to the experience of empathy, thereby lying at the core of social interaction. Schizophrenia is associated with pervasive social cognitive impairments, including emotional processing of facial expressions. In this study we test a novel paradigm in order to investigate the evaluation of the emotional content of perceived emotions presented through dynamic expressive stimuli, facial mimicry evoked by the same stimuli, and their functional relation. Fifteen healthy controls and 15 patients diagnosed with schizophrenia were presented with stimuli portraying positive (laugh), negative (cry) and neutral (control) emotional stimuli in visual, auditory modalities in isolation, and congruently or incongruently associated. Participants where requested to recognize and quantitatively rate the emotional value of the perceived stimuli, while electromyographic activity of Corrugator and Zygomaticus muscles was recorded. All participants correctly judged the perceived emotional stimuli and prioritized the visual over the auditory modality in identifying the emotion when they were incongruently associated (Audio-Visual Incongruent condition). The neutral emotional stimuli did not evoke any muscle responses and were judged by all participants as emotionally neutral. Control group responded with rapid and congruent mimicry to emotional stimuli, and in Incongruent condition muscle responses were driven by what participants saw rather than by what they heard. Patient group showed a similar pattern only with respect to negative stimuli, whereas showed a lack of or a non-specific Zygomaticus response when positive stimuli were presented. Finally, we found that only patients with reduced facial mimicry (Internalizers) judged both positive and negative emotions as significantly more neutral than controls. The relevance of these findings for studying emotional deficits in schizophrenia is discussed.

Feel, smell and see in an egg: emergence of perception and learning in an immature invertebrate, the cuttlefish embryo

It is now well established that prenatal sensory experience affects development itself and has long-term consequences in terms of postnatal behavior. This study focused on the functionality of the sensory system in cuttlefish in ovo. Embryos of stage 23, 25 and 30 received a tactile, chemical or visual stimulus. An increase of mantle contraction rhythm was taken to indicate a behavioral response to the stimulus. We clearly demonstrated that tactile and chemical systems are functional from stage 23, whereas the visual system is functional only from stage 25. At stage 25 and 30, embryos were also exposed to a repeated light stimulus. Stage 30 embryos were capable of habituation, showing a progressive decrease in contractions across stimulations. This process was not due to fatigue as we observed response recovery after a dishabituation tactile stimulus. This study is the first to show that cuttlefish embryos behaviorally respond to stimuli of different modalities and that the visual system is the last to become functional during embryonic development, as in vertebrate embryos. It also provides new evidence that the memory system develops in ovo in cuttlefish.

Congruence and Simultaneity Judgment with a Single Multisensory Target: Preliminary Results

The presence of congruent features may suggest to the brain that stimuli in different modalities are due to a common cause. Previous work has presented conflicting results regarding the impact of congruence on simultaneity judgments, widely thought to be related to judgments of common cause. Here, a novel experimental paradigm using a single changing multi-sensory object is described, with the goal of contrasting results obtained using a single multi-sensory target with those obtained from previous work using arrays of moving dots and staggered asynchronous pairs of targets. Some preliminary results from a pilot validating the experimental procedure are discussed.

Brief Exposure to Sensory Cues Elicits Stimulus-Nonspecific General Sensitization in an Insect

The effect of repeated exposure to sensory stimuli, with or without reward is well known to induce stimulus-specific modifications of behaviour, described as different forms of learning. In recent studies we showed that a brief single pre-exposure to the female-produced sex pheromone or even a predator sound can increase the behavioural and central nervous responses to this pheromone in males of the noctuid moth Spodoptera littoralis. To investigate if this increase in sensitivity might be restricted to the pheromone system or is a form of general sensitization, we studied here if a brief pre-exposure to stimuli of different modalities can reciprocally change behavioural and physiological responses to olfactory and gustatory stimuli. Olfactory and gustatory pre-exposure and subsequent behavioural tests were carried out to reveal possible intra- and cross-modal effects. Attraction to pheromone, monitored with a locomotion compensator, increased after exposure to olfactory and gustatory stimuli. Behavioural responses to sucrose, investigated using the proboscis extension reflex, increased equally after pre-exposure to olfactory and gustatory cues. Pheromone-specific neurons in the brain and antennal gustatory neurons did, however, not change their sensitivity after sucrose exposure. The observed intra- and reciprocal cross-modal effects of pre-exposure may represent a new form of stimulus-nonspecific general sensitization originating from modifications at higher sensory processing levels.

Adaptation to visual or auditory time intervals modulates the perception of visual apparent motion.

It is debated whether sub-second timing is subserved by a centralized mechanism or by the intrinsic properties of task-related neural activity in specific modalities (Ivry and Schlerf, 2008). By using a temporal adaptation task, we investigated whether adapting to different time intervals conveyed through stimuli in different modalities (i.e., frames of a visual Ternus display, visual blinking discs, or auditory beeps) would affect the subsequent implicit perception of visual timing, i.e., inter-stimulus interval (ISI) between two frames in a Ternus display. The Ternus display can induce two percepts of apparent motion (AM), depending on the ISI between the two frames: “element motion” for short ISIs, in which the endmost disc is seen as moving back and forth while the middle disc at the overlapping or central position remains stationary; “group motion” for longer ISIs, in which both discs appear to move in a manner of lateral displacement as a whole. In Experiment 1, participants adapted to either the typical “element motion” (ISI = 50 ms) or the typical “group motion” (ISI = 200 ms). In Experiments 2 and 3, participants adapted to a time interval of 50 or 200 ms through observing a series of two paired blinking discs at the center of the screen (Experiment 2) or hearing a sequence of two paired beeps (with pitch 1000 Hz). In Experiment 4, participants adapted to sequences of paired beeps with either low pitches (500 Hz) or high pitches (5000 Hz). After adaptation in each trial, participants were presented with a Ternus probe in which the ISI between the two frames was equal to the transitional threshold of the two types of motions, as determined by a pretest. Results showed that adapting to the short time interval in all the situations led to more reports of “group motion” in the subsequent Ternus probes; adapting to the long time interval, however, caused no aftereffect for visual adaptation but significantly more reports of group motion for auditory adaptation. These findings, suggesting amodal representation for sub-second timing across modalities, are interpreted in the framework of temporal pacemaker model.

Effects of Near-Threshold Stimulation of the Vestibular Apparatus on Postural Responses Evoked by Displacements of the Visualized Surrounding

In healthy subjects in the relaxed upward stance and perceiving a virtual visual environment (VVE), we recorded postural reactions to isolated visual and vestibular stimulations or their combinations. Lateral displacements of the visualized virtual scene were used as visual stimuli. The vestibular apparatus was stimulated by application of near-threshold galvanic current pulses to the proc. mastoidei of the temporal bones. Isolated VVE shifts evoked mild, nonetheless clear, body tilts readily distinguished in separate trials; at the same time, postural effects of isolated vestibular stimulation could be detected only after averaging of several trials synchronized with respect to the beginning of stimulation. Under conditions of simultaneous combined presentation of visual and vestibular stimuli, the direction of the resulting postural responses always corresponded to the direction of responses induced by VVE shifts. The contribution of an afferent volley from the vestibular organ depended on the coincidence/mismatch of the direction of motor response evoked by such a volley with the direction of response to visual stimulation. When both types of stimulations evoked unidirectional body tilts, postural responses were facilitated, and the resulting effect was greater than that of simple summation of the reactions to isolated actions of the above stimuli. In the case where isolated galvanic stimulation evoked a response opposite with respect to that induced by visual stimulation, the combined action of these stimuli of different modalities evoked postural responses identical in their magnitude, direction, and shape to those evoked by isolated visual stimulation. The above findings allow us to conclude that the effects of visual afferent input on the vertical posture under conditions of our experiments clearly dominate. In general, these results confirm the statement that neuronal structures involved in integrative processing of different afferent volleys preferably select certain type of afferentation carrying more significant or more detailed information on displacements (including oscillations) of the body in space.

Optimizing the early phase development of new analgesics by human pain biomarkers

Human pain biomarkers are based on standardized acute activation of pain pathways/mechanisms and quantitative assessment of the evoked responses. This approach can be applied to healthy volunteers, to pain patients, and before and after pharmacological interventions to help understanding and profile the mode of action (proof-of-concept) of new and existing analgesic compounds. Standardized stimuli of different modalities can be applied to different tissues (multimodal and multi-tissue) for profiling analgesic compounds with respect to modulation of pain transduction, transmission, specific mechanisms and processing. This approach substantiates which specific compounds may work in particular clinical pain conditions. Human pain biomarkers can be translational and may bridge animal findings in clinical pain conditions, which in turn can provide new possibilities for designing more successful clinical trials. Biomarker based proof-of-concept drug studies in either volunteers or selected patient populations provide inexpensive, fast and reliable mechanism-based information about dose–efficacy relationships. This is important information in the early drug development phase and for designing large expensive clinical trials.

Modality Effects in Memory for Basic Stimulus Attributes: A Temporal and Nontemporal Comparison

Previous research suggests that there are significant differences in the operation of reference memory for stimuli of different modalities, with visual temporal entries appearing to be more durable than auditory entries (Ogden, Wearden, & Jones, 2008 , 2010). Ogden et al. ( 2008 , 2010 ) demonstrated that when participants were required to store multiple auditory temporal standards over a period of delay there was significant systematic interference to the representation of the standard characterized by shifts in the location of peak responding. No such performance deterioration was observed when multiple visually presented durations were encoded and maintained. The current article explored whether this apparent modality-based difference in reference memory operation is unique to temporal stimuli or whether similar characteristics are also apparent when nontemporal stimuli are encoded and maintained. The modified temporal generalization method developed in Ogden et al. (2008) was employed; however, standards and comparisons varied by pitch (auditory) and physical line length (visual) rather than duration. Pitch and line length generalization results indicated that increasing memory load led to more variable responding and reduced recognition of the standard; however, there was no systematic shift in the location of peak responding. Comparison of the results of this study with those of Ogden et al. (2008, 2010) suggests that although performance deterioration as a consequence of increases in memory load is common to auditory temporal and nontemporal stimuli and visual nontemporal stimuli, systematic interference is unique to auditory temporal processing.