Learning and extraction of violin instrumental controls from audio signal

Abstract

Acquisition of instrumental gestures in musical performances is an important task used in different fields ranging from acoustics and sound synthesis to motor learning or electroacoustic performances. The most common approach for acquiring gestures is by means of a sensing system. The direct measurement involves the use of usually expensive sensors with some degree of intrusivity and generally entails complex setups. Indirect acquisition is based on the processing of the audio signal and it is usually informed on acoustical or physical properties of the sound or sound production mechanism. In this paper we present an indirect acquisition method of violin controls from an audio signal based on learning of empirical data that is previously collected with a highly accurate sensing system. The learning consists of training of statistical models with a database of multimodal data from violin performances. The database includes audio spectral features and instrumental controls (bow tilt, bow force, bow velocity, bowing distance to the bridge and played string) and is designed to sample most part of the violin performance control space. We expect that once the indirect acquisition system is trained, no sensors should be required, so the indirect acquisition becomes a low-cost and non-intrusive acquisition method.