A humem system for online multimedia feedback

Abstract

In this paper, a Humanized multimedia system with emotional cognition for home-care applications, naming HUMEM system, is proposed to accompany and interact with user for happiness improvement. Three main functionalities are built and coordinated in the proposed work. Firstly, HUMEM gives the capability to recognize user identity by facial recognition based on haar-like features and PCA method, to yield the eigenface for facial expert. Secondly, the emotional semantics recognition is to recognize four kinds of user's verbal emotion (i.e. happiness, anger, sadness, and neutral) through CKIP dictionary and Point-Wise Mutual Information (PMI) method. Particularly, above two audio-visual recognizers are useful for the third user-oriented multimedia feedback system, in which ten more online multimedia services that can meet user daily life and pastime request by embedded google ASR, for example, user can listen or watch the favorite music, video, on-line TV news, weather report, taking picture, general knowledge information retrieval etc. In the experiment, the objective analysis for user recognition and emotion recognition is applied, where the outside-test accuracies for user recognition and emotion recognition can reach 96% and 70%, respectively. In system usability, the mean opinion score (MOS) is applied for subjective evaluation, the result of MOS for the entire HUMEM system is able to acquire 4.4, which initially reveals that the proposed system is acceptable.