Multisensory Perception of Affect

Abstract

Multisensory integration must stand out among the fields of research that have witnessed a most impressive explosion of interest this last decade. One of these new areas of multisensory research concerns emotion. Since our first exploration of this phenomenon (de Gelder et al., 1999) a number of studies have appeared and they have used a wide variety of behavioral, neuropsychological and neuroscientifc methods. The goal of this presentation is threefold. First, we review the research on audiovisual perception of emotional signals from the face and the voice followed by a report or more recent studies on integrating emotional information provided by the voice and whole body expressions. We will also include some recent work on multisensory music perception. In the next section we discuss some methodological and theoretical issues. Finally, we will discuss findings about abnormal affective audiovisual integration in schizophrenia and in autism.