Deception in the eyes of deceiver: A computer vision and machine learning based automated deception detection

Wasiq Khan,Keeley Crockett,James O'Shea,Abir Hussain,Bilal M Khan

doi:10.1016/j.eswa.2020.114341

Abstract

There is growing interest in the use of automated psychological profiling systems, specifically applying machine learning to the field of deception detection. Several psychological studies and machine-based models have been reporting the use of eye interaction, gaze and facial movements as important clues to deception detection. However, the identification of very specific and distinctive features is still required. For the first time, we investigate the fine-grained level eyes and facial micro-movements to identify the distinctive features that provide significant clues for the automated deception detection. A real-time deception detection approach was developed utilizing advanced computer vision and machine learning approaches to model the non-verbal deceptive behavior. Artificial neural networks, random forests and support vector machines were selected as base models for the data on the total of 262,000 discrete measurements with 1,26,291 and 128,735 of deceptive and truthful instances, respectively. The data set used in this study is part of an ongoing programme to collect a larger dataset on the effects of gender and ethnicity on deception detection. Some observations are made based on this data which should not be interpreted as scientific conclusions, but pointers for future work. Analysis of the above models revealed that eye movements carry relatively important clues to distinguish truthful and deceptive behaviours. The research outcomes align with the findings from forensic psychologists who also reported the eye movements as distinctive for the truthful and deceptive behavior. The research outcomes and proposed approach are beneficial for human experts and has many applications within interdisciplinary domains.

Highlights

There has been increasing interest in automatic detection of decep tive behavior, from law enforcement, national security, border controls, internet fraud detection and government agencies (Crockett et al, 2017)
The average combined accuracy results in Table 3 show that Random Forest (RF) has outperformed both Artificial Neural Networks (ANN) and Support Vector Machines (SVMs)
It should be noted that the dataset is mixed in terms of gender and ethnicity, and that the coverage across these two factors is quite balanced in the dataset

Summary

Introduction

There has been increasing interest in automatic detection of decep tive behavior, from law enforcement, national security, border controls, internet fraud detection and government agencies (Crockett et al, 2017). While most of the earlier research on deception detection is based on physiological sensors, such as the polygraph (Larson et al, 1932)or the subjective perception of trained experts undertaking a facial or frame-by-frame analysis (Ekman et al, 1991), each approach can potentially lead to biased human judgments, poor classification of deception and excessive analysis time throughput limits (Bond and DePaulo, 2006). Each method has overlapping as well as distinct indicators of deception. Information content from these indicators has been modelled through various approaches to identify the deceptive behaviour in different scenarios and application domains (O’Shea et al, 2018).

Objectives

Methods

Results

Conclusion