Faceture ID: face and hand gesture multi-factor authentication using deep learning

Abstract

Access control is the act of providing privacy to a resource, and authentication through a single factor is no longer reliable to provide robust protection against unauthorized access. Hence, there is a rapid growth of exploring novel multi-factor authentication (MFA) methods which combine two or more authentication factors– inherence, possession, and knowledge. Despite the increasing use of MFA, to the best of authors’ knowledge, none have so far explored the combination of face, one-time password (OTP) and hand gesture in MFA. Thus, this study produces a proof-of-concept of this combination to form a new authentication method (Faceture ID). Furthermore, this study highlights three contributions: i) face verification with single-sample gallery set using pre-trained Deep Convolutional Neural Network, ii) handwriting gesture recognition using Leap Motion controller for tracking motion and Convolutional Neural Network for classification, and lastly, iii) a new MFA method utilizing face, OTP and hand gesture. The experimental results on the face verification show an average false acceptance rate of 1.94% with average genuine acceptance rate of 67.7%, from the self-built database where the facial images are exposed to variations in pose, expression, and occlusion. In addition, the classifier for the handwriting gesture recognition can predict gestures at about 96% for both precision and recall. Furthermore, the proposed MFA provides a novel systematic approach, high accuracy and performance with the intent to contribute on strengthening the security on privacy of resources against identity theft and attacks.