Real-time timbre analysis as a technique for live synthesis control

Abstract

The possibility is explored of controlling synthesis using an instrumental or other sound source, not by trying to recreate the time-varying timbre of the musician’s sound exactly, but instead by making the output sound reflect changes in the input, in a way a musician can control in order to produce an interesting stream of synthetic sound. First the range of spectra available from the sound source, and also that available from the synthesis technique, are estimated. A very simple measure of timbre is used that maps any sound into a point in a ten-dimensional space. (As a byproduct, the ten-dimensional volume spanned by a sound source gives a rough measure of timbral variability.) Then synthesis parameters are chosen for the synthesis algorithm by searching a database of known output timbres.