Introduction to the Special Issue on MMAC: Multimodal Affective Computing of Large-Scale Multimedia Data

Abstract

The articles in this special section focus on multimodal affective computing of large scale multimedia data. Humans are emotional creatures. Emotion is present everywhere in our daily life and plays a vitally important role in our decisionmaking process. There are roughly two categories of modalities that are widely used by humans to express emotions: explicit affective cues and implicit affective stimuli. On the one hand, to distinguish whether an emotion is induced, one direct way is to check the physical changes of involved humans, such as facial expression, speech, action, gait, and physiological signals (e.g., electroencephalogram). These explicit affective cues can be directly observed and collected from an individual with specific sensors. On the other hand, the rapid development of digital photography and social networks has enabled humans to share their lives and expressing their opinions online using implicit affective stimuli, such as text, images, audios, and videos. The usergenerated content provides an implicit solution to analyze humans’ emotions. Affective computing of explicit and/or implicit large-scale multimedia data is rather challenging due to the following reasons. First, the development of affective analysis is constrained by the affective gap between low-level affective features and high-level emotions. Second, emotion is a subjective concept, and thus affective analysis involves multidisciplinary understanding of human perceptions and behaviors. Finally, emotions are often jointly expressed and perceived through multiple modalities. Multimodal data fusion and complementation need to be explored.