Abstract
Communicating an auditory experience with words is a difficult task and, in consequence, people often rely on imitative non-verbal vocalizations and gestures. This work explored the combination of such vocalizations and gestures to communicate auditory sensations and representations elicited by non-vocal everyday sounds. Whereas our previous studies have analyzed vocal imitations, the present research focused on gestural depictions of sounds. To this end, two studies investigated the combination of gestures and non-verbal vocalizations. A first, observational study examined a set of vocal and gestural imitations of recordings of sounds representative of a typical everyday environment (ecological sounds) with manual annotations. A second, experimental study used non-ecological sounds whose parameters had been specifically designed to elicit the behaviors highlighted in the observational study, and used quantitative measures and inferential statistics. The results showed that these depicting gestures are based on systematic analogies between a referent sound, as interpreted by a receiver, and the visual aspects of the gestures: auditory-visual metaphors. The results also suggested a different role for vocalizations and gestures. Whereas the vocalizations reproduce all features of the referent sounds as faithfully as vocally possible, the gestures focus on one salient feature with metaphors based on auditory-visual correspondences. Both studies highlighted two metaphors consistently shared across participants: the spatial metaphor of pitch (mapping different pitches to different positions on the vertical dimension), and the rustling metaphor of random fluctuations (rapidly shaking of hands and fingers). We interpret these metaphors as the result of two kinds of representations elicited by sounds: auditory sensations (pitch and loudness) mapped to spatial position, and causal representations of the sound sources (e.g. rain drops, rustling leaves) pantomimed and embodied by the participants’ gestures.
Highlights
Introduction- “It sounds like as if you would take a piece of corrugated cardboard [grabbing an imaginary piece of cardboard]
Vocal imitations reproduce rather directly some relevant features of the referent sounds: noisy textures are imitated with noisy vocalizations, tonal sounds with tonal vocalizations, etc
Most imitators have associated low frequencies with a low position of the hands along the vertical axis, and imitated pitch changes with consistent hand motions. This association is sometimes combined with a horizontal motion. With this idea, we observed that referent sounds with no pitch or no pitch change were imitated without any such gesture
Summary
- “It sounds like as if you would take a piece of corrugated cardboard [grabbing an imaginary piece of cardboard] First you scrape it [rubbing both hands], you tear it off [moving apart clenched fists, as tearing off an imaginary piece of cardboard in two pieces] and it sounds like Rrrrrrrr off the cardboard [right fist bouncing off his chest, as moved by an imaginary spring]. Ffffff [palms extended, both arms moving inward, as if rubbing a flat surface], and Rrrrrrrr [clenching fists, right fist bouncing off his chest to an extended arm position]. As illustrated by this example, speakers often use non-verbal imitative vocalizations and gestures to describe sounds when they run out of words, as sounds are notably difficult to describe for lay persons [2,3,4]. Imitative behaviors are widespread in humans [5,6,7,8], and we have recently shown that imitative vocalizations communicate sounds very effectively [9, 10]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
