Basic research on auditory-visual interaction and listening in rooms

Abstract

Attending a live concert is a multisensory experience. In some cases it could be argued that hearing is the primary sense involved, but it is never the only one. Vision, smell, and even touch make important contributions to the overall experience. Moreover, the senses interact such that what one hears, for example, is influenced by what one sees, and vice-versa. This talk will address primarily the auditory aspects of the concert experience, focusing on the results of basic studies of human spatial hearing in reverberant environments, and how these results may help us understand the concert experience. The topics will include sound localization in anechoic and reverberant environments, the precedence effect, the cocktail party effect, the perception of distance, and the impact of room acoustics on loudness perception. Also discussed will be what has been learned from empirical research on auditory-visual interactions. In this area the focus will be on the visual capture effects, the best known of which is the ventriloquism effect. Finally, the limitations of modern psychoacoustics will be addressed in connection with the problem of fully revealing the complexities of the concert experience, especially individual differences in subjective impression.