Interactions among acoustics, digital signal processing, and movement

Abstract

A team of architects, electronic artists, and acousticians designed a system for manipulating acoustic, haptic, and spatial information based on the motion of dancers through space. The dance structure was designed to inspire physical exploration and interaction, to sense and process resultant sounds based on this dance, and to present an immersive sound field to the dancers and observers which complemented the visual impact of the movement. A microphone system captured the sounds of the dancers’ contact with the system. An electronic sensor system tracked the location and action of the dancers. A set of digital signal processing algorithms analyzed, modified, and spatialized in real time the information collected from the environment, integrating sensory cues arriving simultaneously and rapidly via multiple modalities. A multichannel speaker system re-presented the natural and modified sounds to the acoustic environment, which in turn affected the dancers’ movements. The nature of movement through physical space, the localization of movement in both vision and audition, and the abstraction, support, and deconstruction of both modalities were studied. The results were used to design a performance by a dance ensemble that explored the synergy between movement, multimodal sensory information, and sound.