Robust Algorithm for Multimodal Deception Detection

Abstract

Automatic deception detection from the video has gained a paramount of interest because of their applicability in various real-life applications. The recorded videos contain various information such as temporal variations of the face, linguistics and acoustics, which can be used together, to detect deception automatically. In this work, we proposed a new approach based on multimodal information like audio, linguistic (or text) and non-verbal features. The proposed multimodal deception detection framework is based on combining the decision from the audio, text and non-verbal features using majority voting. The proposed multimodal deception system is banked on the audio system based on Cepstral Coefficients (CC) and Spectral Regression Kernel Discriminant Analysis (SRKDA) of fixed length audio sequences. The text system is based on bag-of-n-grams features and the linear Support Vector Machine (SVM) classifier while the non-verbal features are classified using the AdaBoost classifier. Extensive experiments are carried out on a publicly available real-life deception video dataset to evaluate the efficacy of the proposed scheme. The obtained results on a 25-cross-fold validation have indicated a deception detection accuracy of 97% out-performing both state-of-the-art techniques and human performance on the whole dataset.