Apex Frame Spotting Using Convolutional Neural Networks with Continuous Labeling

Abstract

Apex frame is the frame containing the highest intensity changes of facial movements in a sequence of video. It plays a crucial role in the analysis of micro-expressions, which generally have minute facial movements. This frame is hard to be identified that requires a laborious and time-consuming effort from highly skilled specialists. Therefore, a convolutional neural networks-based technique is proposed to automate apex frame detection using a novel continuous labeling scheme. The network is trained using ascending and descending labels according to the linear and exponential functions, pivoted on the ground truth apex frame. Two datasets, CASME II and SAMM databases are used to verify the proposed algorithm, where the apex frame is determined according to the maximum label intensity and a sliding window of the maximum label intensity. The results show that a linear continuous label with the sliding window approach produced the lowest average error of 14.37 frames.