A hybrid fusion framework to multi-modal bio metric identification

Abstract

In the recent decade, comprehensive research efforts have been carried out as the promising modality of bio metrics on humans’ physical features for person recognition. Despite this, the main issue encountered in identifying individuals is obtaining rich representation for multi-modal data that is invariant to diverse physical traits. The shortcomings of uni-modal bio metric systems can be tackled by combining derived knowledge from several modalities of bio metric systems embedded with several physical characteristics like Ear, Face, Iris, and Gait. This paper proposes a novel multi-modal bio metric identification framework based on a hybrid multi-phase feature fusion to render compact knowledge from multiple model traits. We employed transfer learning through several pertained networks such as Resnet101, Resnet-Inceptionv2, Densenet201, AlexNet, and Inceptionv2 to fuse with handcrafted feature vectors extracted via Hog feature descriptor. The fusion is performed using Discriminant Correlation Analysis (DCA) and Canonical Correlation Analysis (CCA) at each single and hybrid phase. Three state of the art bio metric databases, namely Face, Gait, and Ear, was utilized to evaluate the proposed framework. The proposed framework based on multi-phase hybrid fusion achieved up to 96.6% of identification accuracy using multi traits. Experimental results confirm the superior results over other recent multi-modal bio metric variants.