Person identification through faces and voices: An ERP study

Abstract

Different models have been proposed to explain how identity is extracted from faces and voices and how these two sensory systems interact. The neural loci of audio–visual interactions have been studied using neuroimaging techniques; however, the time course of these interactions is not well established. Here, we use event related potentials (ERPs) to study the temporal dynamics of the interaction of face and voice processing modules. We presented to the subjects either faces alone (F), voices alone (V) or faces and voices together (FV) in a familiarity detection task. Responses obtained for FV were compared with the sum of the responses obtained for F plus responses for V, for familiar and unfamiliar stimuli. This comparison shows differences in amplitude for different latencies, indicating cross-modal interactions. For unfamiliar stimuli, this interaction began very early (around 200 ms) and was restricted to the time window corresponding to the face N170 component. For familiar stimuli, the interaction was longer, began earlier and remained until after the N170 component. These results indicate that the interaction between faces and voices occurs from the initial stages of processing and continues as the person identification process goes on. This study is the first electrophysiological evidence of cross-modal interaction of faces and voices during the recognition of acquaintances' identities. It suggests that the assessment of person familiarity can result in direct information sharing between voice and face sensory modules from the early processing stages, before access to the person identity nodes.