Multimodal emotion recognition (MER) system

Abstract

Today, a number of recognition systems have been proposed widely, from audio recognition to image recognition, and from two dimensional databases to three dimensional databases; the study and research on the emotion recognition system become more important than ever before. This paper shows the new research and development of the multimodal emotion recognition system (MER). There are two main categories in this MER System, a new database and the MER fusion recognition part. The MER database and recognition system. The use Microsoft XBOX KINECT sensor, the data include 2D facial images, 3D face feature points and audio wave in a concurrent time based. In the recognition system part, it use multimodal fusion level as final classifier, include decision level fusion, feature level fusion and a new fusion level combination. The MER achieves the best overall and individual emotion recognition that represent the true emotion of human bean.