Designing Generative Sound for Responsive 3D Digital Environment Interaction

Abstract

This paper examines three key areas of responsive sound interaction in 3D Digital Environments: designing generative sound that derives its composition and relevance from social and physical human interaction within a digital environment; the relation of sonic structure to the digital visual and spatial experience; and responsive, reactive real time sound generation activated by environmental conditions and human behaviours. The primary purposes for responsive sound design are: (1) to provide navigational cues supporting way-finding and spatial orientation; and (2) to provide realtime generative environmental sound that reflects social behaviour in a way that is meaningful and recognisable. The applied contexts for navigational cues and environmental generative sound include online (multi-user), synchronous Virtual Environments and Digital Installation Spaces (e.g. intelligent rooms, virtual reality and immersive environments). Outcomes of responsive sound design include: a trigger system of aural alerts, warnings and guidance; a computational system for generating sound in real time activated by spatial location and social interaction; and an audio (non-visual) tool aiding spatial orientation and way-finding interaction in 3D immersive Digital Environments.