Characterizing real-world auditory scenes using 360° audio-visual capture

Abstract

Accurate sound identification is a critical aspect of situational awareness. Environmental sound identification is influenced by physical stimulus dimensions and subjective factors, such as pleasantness or familiarity. Ecological frequency, the prevalence of sounds’ occurrences in the environment, may also play an important role; however, accurately capturing this information for real-world sounds is difficult. Due to limited information about ecological frequency and the difficulty of accurately characterizing acoustic characteristics of real-world sounds studies do not typically account for this information during stimulus selection. The present study evaluates ecological frequency for environmental sounds collected using a RealSpace Camera, a spherical array of 64 microphones and 5 HD cameras. Environmental sound recordings were captured in various urban and rural settings. Samples were recorded for 30 seconds at five to ten minute intervals for one hour (resulting in up to six samples per session). These multichannel, short-interval recordings were augmented by hour-long single channel samples to enable comparison between long and short term spectro-temporal changes associated with the captured scenes. Human raters then evaluated recorded scenes with and without video confirmation to estimate ecological frequency. This effort is the first step in characterizing the real-world sensory environment and its influence on perceptual performance.