Emotion recognition with consideration of facial expression and physiological signals

Abstract

An emotion recognition system with consideration of facial expression and physiological signals is proposed in this paper. A specific designed mood induction experiment is performed to collect facial expressing images and physiological signals of subjects. We detected 14 feature points and extracted 12 facial features from facial expression images. Meanwhile, we measure the skin conductivity, finger temperature and heart rate from the subject. Both facial and physiological features are adopted to train the classifiers. Two learning vector quantization (LVQ) neural networks were applied to classify four emotions: love, joy, surprise and fear. Experimental results show the proposed recognition system is able to identify four emotions by facial expressions, physiological signals, and both of them.