A review of time–frequency representations, with application to sound/music analysis–resynthesis

Abstract

Analysis–resynthesis (A–R) systems gain their flexibility for creative transformation of sound by representing sound as a set of musically useful features. The analysis process extracts these features from the time domain signal by means of a time–frequency representation (TFR). The TFR provides an intermediate representation of sound that must make the features accessible and measurable to the rest of the analysis. Until very recently, the short-time Fourier transform (STFT) has been the obvious choice for time–frequency representation, despite its limitations in terms of resolution. Recent and ongoing developments are providing several alternative schemes that allow for a more considered choice of TFR. This paper reviews these contemporary approaches in comparison with the more classical ones and with reference to their applicability, merits and shortcomings for application to sound analysis. (Where they have been successfully applied, details are provided.) The techniques reviewed include linear, bilinear and higher-order spectra, nonparametric and parametric methods and some sound-model-specific TFRs.