Abstract
Many salient visual events tend to coincide with auditory events, such as seeing and hearing a car pass by. Information from the visual and auditory senses can be used to create a stable percept of the stimulus. Having access to related coincident visual and auditory information can help for spatial tasks such as localization. However not all visual information has analogous auditory percepts, such as viewing a computer monitor. Here, we describe a system capable of detecting and augmenting visual salient events into localizable auditory events. The system uses a neuromorphic camera (DAVIS 240B) to detect logarithmic changes of brightness intensity in the scene, which can be interpreted as salient visual events. Participants were blindfolded and asked to use the device to detect new objects in the scene, as well as determine direction of motion for a moving visual object. Results suggest the system is robust enough to allow for the simple detection of new salient stimuli, as well accurately encoding direction of visual motion. Future successes are probable as neuromorphic devices are likely to become faster and smaller in the future, making this system much more feasible.
Highlights
Attentional orienting mechanisms allow us to notice important sensory events and reallocate perceptual resources to deal with them
Percent correct ranged from 55.247% (D100) to 70.699% (D400), with hit rate increasing with each increase in displacement (Fig 2a)
The present study investigated whether an auditory augmented-reality system could render visual events as salient auditory events and convey spatial information about the visual scene
Summary
Attentional orienting mechanisms allow us to notice important sensory events and reallocate perceptual resources to deal with them. A variety of visual events are known to trigger reorienting— motion stimuli and abrupt onsets of new objects [1, 2] and the ability to detect these attentional cues is critical to safely interacting with the world. Visual deficits or impairment of the attention orienting system due to stroke can be debilitating. Visual events often coincide with auditory events, providing a multimodal cue. For someone with a visual deficit, this coupling of auditory and visual information is critical because it affords the only indication of a potentially important change in the sensory world. Not all important sensory events are multimodal. Objects that start moving in a cluttered auditory scene might
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
