Facial expression intensity estimation using Siamese and triplet networks

Abstract

Abstract This paper investigates the Siamese and triplet networks abilities of emotional intensity estimation in facial image sequence. In our method, we extract the sequential relationship in the temporal domain that appears due to the natural onset apex offset variation in pattern of facial expression. Siamese and triplet networks are shown to perform better than the earlier convolutional neural networks in such task. The branches of the Siamese and triplet networks help in leading to an output that is more definite. Compared with Siamese network, the triplet network internal representation of learned features emerges clearer and more accurate localizations of those features appear with training. This property improves the network generalization when dealing with similar sequential images. We confirmed this by experiments on Cohn–Kanade, MUG and MMI datasets for intensity estimations and CASME, CASME II and CAS(ME) 2 datasets on micro-expressions detection.