Sketch-Based Musical Composition and Performance

Abstract

Sketching is a natural way for one person to convey their thoughts and intentions to another. With the recent rise of tablet-based computing, the use of sketching as a control and interaction paradigm is one that deserves exploration. In this paper we present an interactive sketch-based music composition and performance system called Drawchestra. The aim of the system is to give users an intuitive way to convey their musical ideas to a computer system with the minimum of technical training thus enabling them to focus on the creative tasks of composition and performance. The system provides the user with a canvas upon which they may create their own instruments by sketching shapes on a touch screen. The system recognises a certain set of shapes which it treats as virtual instruments or e↵ects. Once recognised, these instruments can then be played by the user in real time. The size of a sketched instrument shape is used to control certain parameters of the sound so the user can build complex orchestras containing many different shapes of di↵erent sizes. The sketched shapes may also be moved and resized as desired making it possible to customise and edit the virtual orchestra as the user goes along. The system has been implemented in Python and user tests conducted using an iPad as the control surface. We report the results of the user study at the end of the paper before briefly discussing the outcome and outlining the next steps for the system design.