Sequence-wise multimodal biometric fingerprint and finger-vein recognition network (STMFPFV-Net)

Abstract

The existing multimodal biometric fingerprint and vein deep learning features were found effective for biometric recognition. However, the current performance of the deep learning features is limited due to missing temporal image dependence and the extracted features can obfuscate some vital information because of irrelevant image features. This work proposes a sequence of filtered spatial and temporal multimodal fingerprint and finger vein network (FS-STMFPFV-Net). The overall proposed FS-STMFPFV-Net is achieved from two-channel independent learning to improve image variabilities. In the first channel, the image sequence is generated by aligning the fingerprint and finger vein images together inside the image generator. The sequences are built into the five layers of a deep convolution neural network fusion model to extract spatial sequence-wise features. The second channel is where the extracted sequence-wise features are remembered in the long short-term memory for interactions between temporal and sequence dimensions which finally generate their long-term temporal dependencies as complementary information. These sequences are fused together, and the discriminative sequence features are selected using feature selection. We have presented the ReliefFS feature selection which serve as a complementary basis for selecting compact features from the extracted CNN-based features. To evaluate the performance of the proposed FS-STMFPFV-Net, the NUPT-FPV, FVC-2002-DBs, and the CASIA dataset were used, which provides fingerprint, finger-vein, and palmprint image databases, for experimental validation. The proposed FS-STMFPFV-Net when evaluated using standard protocols offers more than 97% accuracy across the different databases and is ten times more computationally friendly than the existing algorithms.