Spectral modeling, analysis, and synthesis of musical sounds

Abstract

Spectral models provide general representations for sound well-suited for expressive musical transformations. These models allow us to extract and modify perceptually-relevant parameters such as amplitude, frequency, and spectrum. Thus, they are of great interest for the classification of musical sounds. A new analysis method was proposed to accurately extract the spectral parameters for the model from existing sounds. This method extends the classic short-time Fourier analysis by also considering the derivatives of the sound signal, and it can work with very short analysis windows. Although originally designed for stationary sounds with no noise, this method shows excellent results in the presence of noise and it is currently being extended in order to handle nonstationary sounds as well. A very efficient synthesis algorithm, based on a recursive description of the sine function, is able to reproduce sound in real time from the model parameters. This algorithm allows an extremely fine control of the partials of the sounds while avoiding signal discontinuities as well as numerical imprecision, and with a nearly optimal number of operations per partial. Psychoacoustic phenomena such as masking are considered in order to reduce on the fly the number of partials to be synthesized.