Effects created by time-variant spectral processing of sounds

Abstract

Various processes can be used to modify a sound to produce special effects. One approach is to first compute a time-frequency represention of a sound signal, follow that by altering the representation in various ways, and finally perform resynthesis to produce modified versions of the signal. Example modifications are (1) duration modification (time-scaling), (2) pitch change, (3) noise content manipulation, (4) spectral envelope modification, and (5) temporal envelope modification. The time-frequency representation can also be used to develop simplified synthesis models that depend on only a few time-varying parameters, such as overall amplitude, pitch, and spectral centroid and shape. In the special case of musical sounds with vibrato, spectral amplitudes and frequencies can be parameterized in terms of time-variant vibrato depth, rate, and mean envelope. These parameters can then be controlled independently of pitch and duration. Also, timbral parameters of instruments can be interchanged or blended to create hybrid instruments with novel timbral qualities. Several of these effects will be described and demonstrated.