Indirect Acquisition of Violin Instrumental Controls from Audio Signal with Hidden Markov Models

Abstract

Acquisition of instrumental gestures in musical performances is a field of increasing interest with applications in different areas ranging from acoustics and sound synthesis to motor learning or artistic performances. Direct acquisition approaches are based on measurements with sensors attached on the instrument or the performer, a process that usually involves the use of expensive sensing systems and complex setups that are generally intrusive in practice. An alternative is the indirect acquisition without sensors based on analysis of the audio signal. This paper reports a novel indirect acquisition method for the estimation of continuous violin controls from audio-signal analysis based on the training of statistical models with a database of previously recorded violin performances. The database contains synchronized streams of audio features and instrumental controls. Audio signal was captured with a vibration transducer built into the violin bridge, and instrumental controls were measured with sensors. The controls include bowing parameters (played string, bowing velocity, bowing force, and bowing distance to the bridge) as well as fingering position. Once the model is trained for a specific violin, we can perform indirect acquisition from analysis of the signal captured with its embedded transducer without the need for the sensors any more. The statistical methods used are Hidden Markov Models (HMM) with observation distributions parameterized as Multivariate Gaussian Mixtures (GM). HMMs provide a means for note recognition and following and parameter prediction is based on GM regression. Results show that the presented method constitutes an accurate, non-intrusive and low-cost alternative for instrumental control acquisition of a previously calibrated violin and recording device.