ARL-IL CNN for Automatic Facial Expression Recognition of Infants under 24 Months of Age

Abstract

Automatic facial expression coding of infants plays an important role in infants-related applications, including computer-aided ASD diagnosis, automatic intervention for ASD children and diagnosis of ADHD, etc. However, most of existing facial expression researches focused on adult facial expression analysis, the infant facial expression recognition has been less investigated. Due to an age gap between the facial expression datasets of adults and infants, a facial expression recognition model trained on adult datasets usually shows poor generalization to infants datasets. A labeled infant facial expression dataset can mitigate this problem, and hence we first collect a facial expression dataset of 30 infants under 24 months of age by recording videos of infants’ facial expression during a face-to-face mother-infant interaction. Due to infants spontaneous facial behaviors, the dataset covers multiple challenges, such as large head-poses, occlusion, facial expression intensities, etc. To develop an automatic facial expression coding system, we propose a framework consisted of adaptive region learning and island loss, i.e., ARL-IL, to self-adaptively discover facial regions with higher discriminability between different emotion classes. The framework was verified on our collected dataset, and attained a classification accuracy of 86.86%, which has shown better performance than conventional method based on hand-crafted features and some basic CNN architectures. To interpret the effectiveness of ARL-IL, we also visualize the learned features and find that the proposed framework focuses on facial regions with more emotion information compared with other hand-crafted features or learned features from basic CNN architectures. The experimental results show that our proposed framework has robustness to the large head-poses and occlusion.