When hybrid sound effects are better than real recordings

Abstract

Sound effects technicians (“Foley Artists”) have long exploited the fact that two physically different events can produce perceptually similar sounds, such as squeezing a box of cornstarch to imitate footsteps in the snow. Although some sound effects succeed because they produce acoustic waveforms nearly identical to the sounds they are imitating (their targets), in other cases there are obvious acoustic differences between sound effects and their targets. Those differences may provide information about which acoustic features are essential, and which are extraneous, for auditory recognition of an event. To address this question, nine pairs of sound effects and their associated target events were recorded. Listeners identified sounds presented over headphones. When identification was analyzed into the actions and the materials that caused each sound, some sound effects had advantages over their target recordings. Next, the listeners were informed of the target sound (e.g., footsteps in the snow) and were asked to rate the sound’s realism. Finally, hybrid sound effects were created by separating the temporal envelope and fine structure of each target and Foley sound effect before recombining their best-identified components. In two cases, hybrid sound effects were robustly judged as more realistic than recordings of their target events.