WiCapose: Multi-modal fusion based transparent authentication in mobile environments

Abstract

The prosperity of mobile sensing technology makes smartphone-based authentication more prevalent in mobile environment. The present single-modality security authentication based on human biometrics is vulnerable to counterfeit, and the procedure of basic two-factor authentication (2FA) is cumbersome. In this work, we propose a 2FA method without additional devices and procedures, namely WiCapose. The essential technique is to use the unique correlation of the two side-channel to complete multi-modal feature extraction and fusion authentication. WiCapose can simultaneously extract the radio frequency (RF) gain feature that directly characterizes the finger movement and the inter-frame spatio-temporal difference from the rear camera that indirectly portrays the tapping behaviors, to blend the micro-scale behavior pattern feature of the user’s finger and implicit biological features. Then we design and train a deep neural network to fuse both factors for mitigating the limitation of the RF factor in environmental generalization and the uniqueness of the image factor on authentication performance. Experiments involving ten participants demonstrate that our method can achieve a 98% average accuracy on authentication and effectively resist shoulder-surfing attacks and mimic attacks.