Spotting micro‐movements in image sequence by introducing intelligent cubic‐LBP

Abstract

Invisible to naked human eyes, micro-movements are barely noticeable. There are wide ranges of micro-movement applications from spotting subtle changes in a volcano, vascular pulse, and blood vessel to micro-expression detection. In the latter, some evil thoughts can be unveiled, resulting in identifying crooks and lawbreakers and it arises from involuntary subtle and short-duration of facial muscles movements. Precise spotting of these tiny movements is possible only when multiple aspects of temporal images are scrutinized. Meanwhile, since motions often happen in one or two directions, it is rudimentary to extract complete feature sets in textural-based approaches such as cubic Local Binary Pattern (cubic-LBP). Approaches like cubic-LBP also have imposed an unnecessary computation burden. Hence, in this research, a novel method named intelligent cubic-LBP is proposed, which incorporates Convolutional Neural Network (CNN) model. This model learns to select the useful plane(s) automatically. Apex is then detected by applying Partial Autocorrelation Coefficient (PACF) on the selected plane(s). The experimental results show significant improvement in micro-movements identification. The accuracy of the apex frame detection has elevated to 10% and 17% in the Chinese Academy of Sciences Micro-Expressions (CASME) and the CASME II databases, respectively.