Spontaneous Facial Expression Recognition by Fusing Thermal Infrared and Visible Images

Abstract

In this paper, we propose a method for spontaneous facial expression recognition by fusing features extracted from visible and thermal infrared images. First, the active appearance model parameters and head motion features are extracted from the visible images, and several thermal statistical parameters are extracted from the infrared images. Second, a multiple genetic algorithms-based fusion method is proposed for fusing these two spectrums. We use this proposed fusion method to search for the optimal combination of a similarity measurement and a feature subset. Then, a k-nearest neighbors classifier with the optimal combination is used to classify spontaneous facial expressions. Comparative experiments implemented on the Natural Visible and Infrared Facial Expression database show the effectiveness of the proposed similarity measurement and the feature selection method, and demonstrate the fusion method’s advantage over only using visible features.Keywordsfacial expression recognitionthermal infraredvisiblefusiongenetic algorithm