Toward Emotional Annotation of Multimedia Contents

Abstract

By annotating multimedia contents, users of a web resource can associate a word or a phrase (tag) with that resource such that other users can retrieve it by means of searching. Nowadays, tags play an important role in search and retrieval process in multimedia content sharing social networks. Explicit tagging refers to assigning tags directly in an explicit way such as typing. Implicit tagging, however, refers to assigning tags by observing users’ behaviors during exposure to multimedia contents. Among various kinds of information that can be obtained for the purpose of implicit tagging, emotional information about a given content is of great interest. In this chapter, we discuss various means of emotion recognition and emotional characterization, which can be used as tools for emotional tagging. A P300-based brain-computer interface system is proposed for the purpose of emotional tagging of multimedia content. We show that this system can successfully perform emotional tagging and naive users who have not participated in the training of the system can also use it efficiently. Furthermore, we present emotional annotating systems using multimedia content analysis and electroencephalogram signal processing and will compare them. Finally, a road map for developing a practical multimodal system for implicit emotional annotation of multimedia contents will be sketched out.KeywordsVideo ClipEmotion RecognitionMultimedia ContentGalvanic Skin ResponseEmotional CategoryThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.