Live-performance frequency and feature extraction for synthesis control

Abstract

Musical frequency and feature extraction techniques for guitars and other string instruments are described. These methods obtain rapid and accurate estimates of frequency, intensity, and related spectral features to be used for synthesis control in live musical performance. Live-performance control imposes severe constraints in that the string interface and the synthesis system to which it is attached must respond with the control intimacy characteristic of an acoustic instrument. This requires extremely rapid frequency determination, which is accomplished by using fingerboard information in combination with the acoustic signal from the string. For the guitar, a fret scanning system has been developed to provide an estimate prior to the sounding of the note. Specialized sensor technologies are used to isolate the individual strings. Waveform features are timed to obtain period estimates used to continuously update the initial information from the fret scan. This combination of fret-scan information and period estimation from the signal provides for very accurate tracking and synthesizer notes can be started within 2 ms of the string excitation. It will be demonstrated that the MIDI protocol imposes severe constraints on control intimacy. A more suitable communications protocol for synthesis control and a new generation of instruments that provide an intimate link between instrument interface and synthesis are presented.