The influence of hearing and seeing on aesthetic judgments of performance spaces

Abstract

The perception of rooms involves different modalities, particularly hearing and seeing. Fundamental questions of audio-visual room perception have, however, not been answered yet. We investigated to what extent aesthetic judgments on music and speech performances in different spaces are based on hearing, seeing, and their interaction. Meeting methodological criteria such as optoacoustic commensurability and optoacoustic dissociation, and applying a data-based, three-dimensional, high-resolution optoacoustic simulation providing nearly all perceptually relevant physical cues allowed for a valid proportionate quantification of these shares for the first time. BRIRs and panoramic stereoscopic images were acquired in six rooms. Recordings of both a music and a speech performance were put in the rooms by applying dynamic binaural synthesis and chroma-key compositing. The scenes were played back by the use of a linearized extraaural headset and a semi-panoramic stereoscopic projection system. Test participants were asked to rate the pleasantness, powerfulness, and excitingness of the performance, their spatial presence in the virtual scene, and the matching of the acoustical and the optical room. Statistical analyses indicate among others, that the aesthetic judgments mainly relied on acoustic information, whereas the presence and the matching judgment was mainly influenced by the interaction of optical and acoustical room properties.