Multimodal Biometric Database Research Articles

A multimodal biometric database and case study for face recognition based deep learning

Recently, multimodal biometric systems have garnered a lot of interest for the identification of human identity. The accessibility of the database is one of the contributing elements that impact biometric recognition systems. In their studies, the majority of researchers concentrate on unimodal databases. There was a need to compile a fresh, realistic multimodal biometric database, nonetheless, because there were so few comparable multimodal biometric databases that were publically accessible. This study introduces the MULBv1 multimodal biometric database, which contains homologous biometric traits. The MULBv1 database includes 20 images of each person's face in various poses, facial emotions, and accessories, 20 images of their right hand from various angles, and 20 images of their right iris from various lighting positions. The database contains real multimodal data from 174 people, and all biometrics were accurately collected using the micro camera of the iPhone 14 Pro Max. A face recognition technique is also suggested as a case study using the gathered facial features. In the case study, the deep convolutional neural network (CNN) was used, and the findings were positive. Through several trials, the accuracy was (97.41%).

Multi-Modal Biometric Recognition for Face and Iris using Gradient Neural Network (Gen-NN)

In recent years, Biometric system are the one, which is widely used method for the recognition and identification of an individual that are highly demanded approach for its absolute security and accuracy which plays a vital roles in banking, commercials, business and other fields. Moreover this research is based on the multimodal biometrics which is recommended for its high recognition performances and it overcome the demerits of unimodal biometric approach. This research concentrate two multi-modal biometric traits such as face and iris, and propose Gradient Neural Network (Gen-NN) method to improve the biometric authentication by using the VISA face and iris multi-modal biometric database also used ResNet-101 and WaveNet for the feature extraction where the input of face and iris can be extracted.

Securing multimodal biometric template using local random projection and homomorphic encryption

Due to the rapid advancement of technology, biometrics is widely used in authenticating a person. The primary issue in biometric authentication is template protection, which protects the user’s privacy. To achieve this level of security, we transform the original template generated from the extracted features into a pseudo template. This paper considers two biometric traits, namely Iris and Fingerprint. First, the features are extracted from both the traits. After extracting the features, the feature vectors are compressed to reduce the size. Then the feature vectors of both the traits are fused. Later, local random projection (LRP) is applied to this template to build a revocable and un-linkable template. Finally, fully homomorphic encryption (FHE) is applied to this generated template to secure the user’s privacy, as all operations on the template are carried on encrypted data. Our proposed method improves the accuracy of the system as it generates rotational invariant codes. This proposed method is tested on the Children Multimodal Biometric Database (CMBD), which shows that our method can maintain user privacy with a good accuracy rate. Our method achieves an EER of 0.0214%.

Towards better and unlinkable protected biometric templates using label‐assisted discrete hashing

The growth of biometrics-based authentication in various services raises the need to protect biometric data at the storage level. Specifically, biometric templates need to be protected after features are extracted to avoid the leakage of biometric data and subsequent linkability issues. An approach based on discrete hashing is presented with the assistance of semantic labels to generate discriminative and privacy preserving protected templates in this work. The proposed approach can easily be adopted for a closed-enrolment set in which enrolment images are known a priori whereas the challenge of learning templates for a single subject remains open. To extend this approach for individual subject, the concept of auxiliary pseudouser enrolment data is introduced, through which a protected template can be generated at the user level. Through the use of a moderately sized multimodal biometric database of 94 subjects, the effectiveness of the proposed approach is illustrated to achieve a robust and secure template protection with irreversibility, unlinkability and renewability. With the set of experiments, the performance of the template protection approach is established and benchmarked against the popular bloom-filter technique. The proposed approach results in a high genuine match rate (≈100% at a false accept rate of 0.01%) and low equal error rate (EER ≈ 0%) and outperforms traditional approaches while satisfying other requirements of biometric template protection when the closed enrolment set is known. With auxiliary pseudousers, the performance of the proposed approach for user-level protected template creation results in an EER of 2.5%, indicating very low performance degradation compared with the known enrolment dataset. Along with the set of experimental validation of the proposed approach, a security analysis of the proposed approach is presented to demonstrate the unlinkability of the biometric templates using a state-of-art unlinkability metric.

VISA: a multimodal database of face and iris traits

In this paper, a new realistic and challenging Face-Iris multimodal biometric database called VISA database is described. One significant problem associated with the development and evaluation of multimodal biometric systems using face and iris biometric traits is the lack of publicly available multimodal databases that are acquired in an unconstrained environment. Currently, there exist no multimodal databases containing a sufficient number of common subjects involved in both face and iris data acquisition process under different conditions. The VISA database fulfills these requirements and it will be a useful tool for the design and development of new algorithms for developing multimodal biometric systems. The VISA iris images are acquired using the IriShield camera. Face images are captured using mobile device. The corpus of a new VISA database consists of face images that vary in expression, pose and illumination, and presence of occlusion whereas iris images vary in illumination, eye movement, and occlusion. A total of more than 5000 images of 100 subjects are collated and used to form the new database. The key features of the VISA dataset are the wide and diverse population of subjects (age and gender). The VISA database is able to support face and/or iris unimodal or multimodal biometric recognition. Hence, the VISA database is a useful addition for the purpose of research and development of biometric systems based on face and iris biometrics. This paper also describes the baseline results of state-of-the-art methods on the VISA dataset and other popular similar datasets. The VISA database will be made available to the public through https://vtu.ac.in/en/visa-multimodal-face-and-iris-biometrics-database/

Convolutional neural networks approach for multimodal biometric identification system using the fusion of fingerprint, finger-vein and face images.

In recent years, the need for security of personal data is becoming progressively important. In this regard, the identification system based on fusion of multibiometric is most recommended for significantly improving and achieving the high performance accuracy. The main purpose of this paper is to propose a hybrid system of combining the effect of tree efficient models: Convolutional neural network (CNN), Softmax and Random forest (RF) classifier based on multi-biometric fingerprint, finger-vein and face identification system. In conventional fingerprint system, image pre-processed is applied to separate the foreground and background region based on K-means and DBSCAN algorithm. Furthermore, the features are extracted using CNNs and dropout approach, after that, the Softmax performs as a recognizer. In conventional fingervein system, the region of interest image contrast enhancement using exposure fusion framework is input into the CNNs model. Moreover, the RF classifier is proposed for classification. In conventional face system, the CNNs architecture and Softmax are required to generate face feature vectors and classify personal recognition. The score provided by these systems is combined for improving Human identification. The proposed algorithm is evaluated on publicly available SDUMLA-HMT real multimodal biometric database using a GPU based implementation. Experimental results on the datasets has shown significant capability for identification biometric system. The proposed work can offer an accurate and efficient matching compared with other system based on unimodal, bimodal, multimodal characteristics.

Effect of face and ocular multimodal biometric systems on gender classification

This study is concerned with analysing face-ocular multimodal biometric systems for a person gender prediction. Particularly, this is the first study considering fusion of face and ocular biometrics to predict gender of a person via a hybrid multimodal scheme. The authors aim to investigate the effect of multimodal biometric systems at score and feature-level fusion on gender classification. The implementation of uniform local binary pattern (ULBP) feature extractor is taken into account to extract the face and ocular texture information. This paper proposes to select the efficient feature sets of both modalities using a novel evolutionary algorithm called backtracking search algorithm (BSA). On the other hand, support vector machine (SVM) is applied for classification purpose using the fused face and ocular features and scores. The proposed scheme is validated using CASIA-Iris-Distance and MBGC multimodal biometric databases with consideration of a subject-disjoint training and testing evaluation. The achieved gender recognition demonstrates the superiority of the hybrid multimodal face-ocular scheme over unimodal face and ocular schemes implemented in this study for a subject gender prediction.

MultiQ: single sensor-based multi-quality multi-modal large-scale biometric score database and its performance evaluation

Single sensor-based multi-modal biometrics is a promising approach that offers simple system construction, low cost, and wide applicability to real situations such as CCTV footage-based criminal investigations. In multi-modal biometrics, fusion at the score-level is a popular and promising approach, and data qualities that affect the matching score of each modality are often incorporated as a quality-dependent score-level fusion framework. This paper presents a very large-scale single sensor-based multi-quality multi-modal biometric score database called MultiQ Score Database version 2 to advance the research into evaluation, comparison, and benchmarking of score-level fusion approaches using both quality-independent and quality-dependent protocols. We extracted gait, head, and height modalities from the OU-ISIR Gait Database and introduce spatial resolution (SR), temporal resolution (TR) and view as quality measures that significantly affect biometric system performance. We considered seven and 10 scaling factors for SR and TR, respectively, with four view variations. We then constructed a database comprising approximately 4 million genuine and 7.5 billion imposter score databases. To evaluate this database, we set two different protocols, and provided a set recognition accuracy for state-of-the-art approaches using protocols for both quality-independent and quality-dependent schemes. This database and the evaluation results will be beneficial for score-level fusion research. Additionally, we provide detailed analysis of the recognition accuracies associated with gait, head, and height modalities in different spatial/temporal resolutions and views. These analyses may be useful in criminal investigation research.

ARTeM: a new system for human authentication using finger vein images

A new system (ARTeM) for human authentication using finger vein images is described here. The developed algorithm combines 1) a fuzzy contrast enhancement algorithm with 2) a mutual information and affine transformation based registration technique and 3) a correlation coefficient based template matching algorithm, to detect the identity of a person based on the match-scores with finger vein images stored in the database. For performance assessment of the ARTeM algorithm, the benchmark SDUMLA multimodal biometric database containing 3816 images of 106 persons is used. On the complete database, up to 95.28% classification accuracy is achieved with single finger images; while up to 98.11% accuracy is observed with a consensus of two fingers. On a reduced subset of 86 persons’ database, 98.84% accuracy is achieved with single finger classification and cent percent classification is obtained using a consensus of two fingers. Comparative analyses with other works also validate the effectiveness of the developed methodology.

Finger multibiometric cryptosystems: fusion strategy and template security

We address two critical issues in the design of a finger multibiometric system, i.e., fusion strategy and template security. First, three fusion strategies (feature-level, score-level, and decision-level fusions) with the corresponding template protection technique are proposed as the finger multibiometric cryptosystems to protect multiple finger biometric templates of fingerprint, finger vein, finger knuckle print, and finger shape modalities. Second, we theoretically analyze different fusion strategies for finger multibiometric cryptosystems with respect to their impact on security and recognition accuracy. Finally, the performance of finger multibiometric cryptosystems at different fusion levels is investigated on a merged finger multimodal biometric database. The comparative results suggest that the proposed finger multibiometric cryptosystem at feature-level fusion outperforms other approaches in terms of verification performance and template security.

MMU GASPFA: A COTS multimodal biometric database

This paper describes the baseline corpus of a new multimodal biometric database, the MMU GASPFA (Gait–Speech–Face) database. The corpus in GASPFA is acquired using commercial off the shelf (COTS) equipment including digital video cameras, digital voice recorder, digital camera, Kinect camera and accelerometer equipped smart phones. The corpus consists of frontal face images from the digital camera, speech utterances recorded using the digital voice recorder, gait videos with their associated data recorded using both the digital video cameras and Kinect camera simultaneously as well as accelerometer readings from the smart phones. A total of 82 participants had their biometric data recorded. MMU GASPFA is able to support both multimodal biometric authentication as well as gait action recognition. This paper describes the acquisition setup and protocols used in MMU GASPFA, as well as the content of the corpus. Baseline results from a subset of the participants are presented for validation purposes.

Multimodal biometric score fusion using Gaussian mixture model and Monte Carlo method

Multimodal biometric fusion is gaining more attention among researchers in recent days. As multimodal biometric system consolidates the information from multiple biometric sources, the effective fusion of information obtained at score level is a challenging task. In this paper, we propose a framework for optimal fusion of match scores based on Gaussian Mixture Model (GMM) and Monte Carlo sampling based hypothesis testing. The proposed fusion approach has the ability to handle: 1) small size of match scores as is more commonly encountered in biometric fusion, and 2) arbitrary distribution of match scores which is more pronounced when discrete scores and multimodal features are present. The proposed fusion scheme is compared with well established schemes such as Likelihood Ratio (LR) method and weighted SUM rule. Extensive experiments carried out on five different multimodal biometric databases indicate that the proposed fusion scheme achieves higher performance as compared with other contemporary state of art fusion techniques.

Multimodal Biometric Score Fusion Using Gaussian Mixture Model and Monte Carlo Method

Multimodal biometric fusion is gaining more attention among researchers in recent days. As multimodal biometric system consolidates the information from multiple biometric sources, the effective fusion of information obtained at score level is a challenging task. In this paper, we propose a framework for optimal fusion of match scores based on Gaussian Mixture Model (GMM) and Monte Carlo sampling based hypothesis testing. The proposed fusion approach has the ability to handle: 1) small size of match scores as is more commonly encountered in biometric fusion, and 2) arbitrary distribution of match scores which is more pronounced when discrete scores and multimodal features are present. The proposed fusion scheme is compared with well established schemes such as Likelihood Ratio (LR) method and weighted SUM rule. Extensive experiments carried out on five different multimodal biometric databases indicate that the proposed fusion scheme achieves higher performance as compared with other contemporary state of art fusion techniques.

The Multiscenario Multienvironment BioSecure Multimodal Database (BMDB)

A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1) over the Internet, 2) in an office environment with desktop PC, and 3) in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008.

A robust person authentication system based on score level fusion of left and right irises and retinal features

The recognition accuracy of a single biometric authentication system is often much reduced due to the environment, user mode and physiological defects. So a multimodal biometric approach for identity verification using two competent traits, iris and retina is proposed. This multimodal approach diminishes the drawback of single biometric system and improves the performance of an authentication system. Iris and Retina biometric recognition offers a highly reliable solution to person authentication. Iris recognition system is composed of segmentation, normalization, feature encoding and matching. Instead of using the entire iris code, only the bits that are consistent in the iris code called the Best bits are considered in the feature matching process. This reduces the computational time and storage requirements of iris code. To enhance the performance of recognition, the iris recognition process is applied to left and right irises separately and the corresponding distance scores are generated for each iris of a person. These scores are combined using the weighted sum fusion rule which further increases the recognition rate. In order to provide liveness verification for our authentication system, we also employed retinal blood vessel pattern recognition. This ensures the presence of only alive persons eliminating the possible spoofing attack. It is composed of segmentation, enhancement, feature encoding and matching. The scores from iris and retina recognition are then combined using weighted sum fusion rule, which further increases the recognition rate. To validate our approach, experiments were conducted on the iris and retina images obtained from CASIA and VARIA datasets respectively. A multimodal biometric database was constructed and then 1500 inter, intra image comparisons were made using the two datasets. The experimental results reveal that our multimodal biometric authentication system is much more reliable and precise than the single biometric approaches.

Multisensor biometric evidence fusion of face and palmprint for person authentication using Particle Swarm Optimisation (PSO)

This paper presents a novel biometric sensor fusion technique for face and palmprint images using Particle Swarm Optimisation (PSO). The proposed method can be visualised in the following steps: we first decompose the face and palmprint image obtained from different sensors using wavelet transformation and then, we employ PSO to select most informative wavelet coefficients from face and palmprint to produce a new fused image. We then employed Kernel Direct Discriminant Analysis (KDDA) for feature extraction and the decision about accept/reject is carried out using Nearest Neighbour Classifier (NNC). Extensive experiments carried out on a virtual multimodal biometric database of 250 users indicate the efficacy of the proposed method.

Biometric template protection in multimodal authentication systems based on error correcting codes*

The widespread deployment of biometric systems has raised public concern about security and privacy of personal data. In this paper, we present a novel framework for biometric template security in multimodal biometric authentication systems based on error correcting codes. Biometric recognition is formulated as a channel coding problem with noisy side information at the decoder based on distributed source coding principles. It is shown that the proposed method binds the biometric template in a cryptographic key which does not reveal any information about the original biometric data even if it is compromised by an attacker. Furthermore, the advantages of the proposed method in terms of security and impact on matching accuracy are discussed. We assess the performance of the proposed method in the context of HUMABIO, an EU Specific Targeted Research Project, where face and gait biometrics are employed in an unobtrusive application scenario for human authentication. Experimental evaluation on a multimodal biometric database demonstrates the validity of the proposed method.

BiosecurID: a multimodal biometric database

A new multimodal biometric database, acquired in the framework of the BiosecurID project, is presented together with the description of the acquisition setup and protocol. The database includes eight unimodal biometric traits, namely: speech, iris, face (still images, videos of talking faces), handwritten signature and handwritten text (on-line dynamic signals, off-line scanned images), fingerprints (acquired with two different sensors), hand (palmprint, contour-geometry) and keystroking. The database comprises 400 subjects and presents features such as: realistic acquisition scenario, balanced gender and population distributions, availability of information about particular demographic groups (age, gender, handedness), acquisition of replay attacks for speech and keystroking, skilled forgeries for signatures, and compatibility with other existing databases. All these characteristics make it very useful in research and development of unimodal and multimodal biometric systems.

An efficient technique for indexing multimodal biometric databases

This paper proposes an efficient indexing technique which can be used in an identification system with large multimodal biometric database. In this technique, multi-dimensional feature vectors of each trait (iris, signature, ear and face) are normalised and projected to a lower dimensional feature space. The reduced feature vectors are fused at feature level and used to index the database by forming Kd-tree. The performance of the proposed technique is also analysed with the feature vectors of all traits by first fusing them and projecting the fused feature vector to a lower dimensional space, and using it for indexing. Performance is also compared with the indexing based on score-level fusion. The experiment is performed on a multimodal database consisting of 5400 images of 150 subjects (i.e. nine images per subject, per trait). Out of the nine, eight images are used for training and one is used for testing. Our experiment shows that the proposed technique significantly reduces the data retrieval time along with possible error rates.

Spoken signature for user authentication

We propose a new user authentication system based on spoken signatures, where online signature and speech signals are acquired simultaneously. The main benefit of this multimodal approach is better accuracy at no extra cost for the user in terms of access time or inconvenience. Another benefit lies in a better robustness against intentional forgeries due to the extra difficulty for the forger to produce both signals. We set up an experimental framework to measure these benefits on MyIDea, a realistic multimodal biometric database publicly available. More specifically, we evaluate the performance of state of the art modeling systems based on Gaussian mixture models (GMM) and hidden Markov models (HMM) applied independently to the pen and voice signal, where a simple rule-based score fusion procedure is used. We conclude that the best performance is achieved by the HMMs, provided that their topology is optimized on a per user basis. Furthermore, we show that more precise models can be obtained through the use of maximum a posteriori probability (MAP) training instead of the classically used expectation maximization (EM). We also measure the impact of multisession scenarios versus monosession scenarios, and the impact of skilled versus unskilled signature forgeries attacks.