CNN-based Multimodal Touchless Biometric Recognition System using Gait and Speech

Abstract

Person identification using biometric features is an effective method for recognizing and authenticating the identity of a person. Multimodal biometric systems combine different biometric modalities in order to make better predictions as well as for achieving increased robustness. This paper proposes a touchless multimodal person identification model using deep learning techniques by combining the gait and speech modalities. Separate pipelines for both the modalities were developed using Convolutional Neural Networks. The paper also explores various fusion strategies for combining the two pipelines and shows how various metrics get affected with different fusion strategies. Results show that weighted average and product fusion rules work best for the data used in the experiments.