Automatic bi-modal emotion recognition system based on fusion of facial expressions and emotion extraction from speech

Abstract

Our software demo package consists of an implementation for an automatic human emotion recognition system. The system is bi-modal and is based on fusing of data regarding facial expressions and emotion that has been extracted from speech signal. We have integrated Viola & Jones face detector (OpenCV), active appearance model - AAM (AAM-API) for extracting the face shape and support vector machines (LibSVM) for the classification of emotion patterns. We have used optical flow algorithm for computing the features needed for the classification of facial expressions. Beside the integration of all processing components, the software system accommodates our implementation for the data fusion algorithm. Our C++ implementation has a working frame-rate of about 5 fps.