Voice over: Audio-visual congruency and content recall in the gallery setting.

Merle T Fairhurst,Ophelia Deroy,Minnie Scott

doi:10.1371/journal.pone.0177622

Merle T Fairhurst, Ophelia Deroy + Show 1 more

Open Access

https://doi.org/10.1371/journal.pone.0177622

Copy DOI

Abstract

Experimental research has shown that pairs of stimuli which are congruent and assumed to ‘go together’ are recalled more effectively than an item presented in isolation. Will this multisensory memory benefit occur when stimuli are richer and longer, in an ecological setting? In the present study, we focused on an everyday situation of audio-visual learning and manipulated the relationship between audio guide tracks and viewed portraits in the galleries of the Tate Britain. By varying the gender and narrative style of the voice-over, we examined how the perceived congruency and assumed unity of the audio guide track with painted portraits affected subsequent recall. We show that tracks perceived as best matching the viewed portraits led to greater recall of both sensory and linguistic content. We provide the first evidence that manipulating crossmodal congruence and unity assumptions can effectively impact memory in a multisensory ecological setting, even in the absence of precise temporal alignment between sensory cues.

Highlights

The hypothesis of a general benefit of multisensory learning has been long defended by famous pedagogues [1,2] and has been thoroughly tested empirically since [3]
We capitalised on the widespread use of audio guides in galleries and museums to test whether the benefits of congruence extend to more complex stimuli presented over longer periods of time in an ecological setting
Participants rated gender matched audio guide tracks as matching more with the portrait than tracks read by an actor of the opposite gender to the sitter

Summary

Introduction

The hypothesis of a general benefit of multisensory learning has been long defended by famous pedagogues [1,2] and has been thoroughly tested empirically since [3]. Humans exhibit an early [7,8] and automatic tendency to match faces and voices [9], underpinned by the exchange of information between face and voice-sensitive areas in the brain [10,11]. This pairing facilitates the recognition of a speaker’s identity [e.g. 12,13], as well as the processing of speech and emotional cues [14], and occurs even in cases when the voice is not spatially congruent with the speaker’s face, as at the cinema.

Objectives

Methods

Results

Discussion

Conclusion