Incorporating gestures into the musical control stream with the conductor’s jacket

Abstract

Much contemporary music is created in a studio, put to tape, and performed through speakers to a passive audience. In the process, some of the excitement and spontaneity of live performances is lost. There is arguably a growing problem that our contemporary musical forms have become so complicated that they cannot readily be performed in live concerts. One solution to this problem is to build real-time sensor-based systems for artists to interactively and spontaneously perform with. However, as has been shown by numerous others, these systems are either too simple in their functionality or require such constrained gestures that they force the performer to conform to unnatural or nonintuitive gestures. This paper presents an approach that adopts a unique synthesis-by-analysis method, whereby the signals created by numerous professional musicians are gathered under various rehearsal and performance conditions, studied, interpreted, and used to inform the choices in a real-time performance system. The final results, which will be presented in this paper, demonstrate a variety of attempts to generate more natural, expressive, and intuitive mappings between gesture and sound. We will discuss three different examples, including one taken from a conducting-style gestural metaphor, one from tai chi, and one fron dance.