Real-Time Mobile Facial Expression Recognition System -- A Case Study

Abstract

This paper presents a mobile application for real time facial expression recognition running on a smart phone with a camera. The proposed system uses a set of Support Vector Machines (SVMs) for classifying 6 basic emotions and neutral expression along with checking mouth status. The facial expression features for emotion recognition are extracted by Active Shape Model (ASM) fitting landmarks on a face and then dynamic features are generated by the displacement between neutral and expression features. We show experimental results with 86% of accuracy with 10 folds cross validation in 309 video samples of the extended Cohn-Kanade (CK+) dataset. Using the same SVM models, the mobile app is running on Samsung Galaxy S3 with 2.4 fps. The accuracy of real-time mobile emotion recognition is about 72% for 6 posed basic emotions and neutral expression by 7 subjects who are not professional actors.