With the development of intelligent application, biometrics recognition technology has been widely concerned and applied in many fields of the real world, such as access control and payment. The traditional biometrics are usually based on single modality data of the subjects, but they are limited by the feature information capacity and the bottleneck in recognition accuracy. In this paper, a multi-modal biometric recognition framework is presented, which utilizes a multi-kernel learning algorithm to fuse heterogeneous information of different modal data. In order to extract complementary information from them, we combine the kernel matrix to form the mixed kernel matrix, and then give the final classification results. The experimental results on multiple biometric datasets show that our method can obtain higher recognition accuracy compared with the existing single mode and multi-mode fusion methods.

In the recent years, we have witnessed the rapid development of face recognition, though it is still plagued by variations such as facial expressions, pose, and occlusion. In contrast to the face, the ear has a stable 3D structure and is nearly unaffected by aging and expression changes. Both the face and ear can be captured from a distance and in a nonintrusive manner, which makes them applicable to a wider range of application domains. Together with their physiological structure and location, the ear can readily serve as supplement to the face for biometric recognition. It has been a trend to combine the face and ear to develop nonintrusive multimodal recognition for improved accuracy, robustness, and security. However, when either the face or the ear suffers from data degeneration, if the fusion rule is fixed or with inferior flexibility, a multimodal system may perform worse than the unimodal system using only the modality with better quality sample. The biometric quality-based adaptive fusion is an avenue to address this issue. In this paper, we present an overview of the literature about multimodal biometrics using the face and ear. All the approaches are classified into categories according to their fusion levels. In the end, we pay particular attention to an adaptive multimodal identification system, which adopts a general biometric quality assessment (BQA) method and dynamically integrates the face and ear via sparse representation. Apart from a refinement of the BQA and fusion weights selection, we extend the experiments for a more thorough evaluation by using more datasets and more types of image degeneration.

Metric learning has significantly improved machine learning applications such as face re-identification and image classification using K-Nearest Neighbor (KNN) and Support Vector Machine (SVM) classifiers. However, to the best of our knowledge, it has not been investigated yet, especially for the multimodal biometric recognition problem in immigration, forensic and surveillance applications with uncontrolled ear datasets. Therefore, it is interesting and very attractive to propose a novel framework for multimodal biometric recognition based on Learning Distance Metric (LDM) via kernel SVM. This paper considers metric learning for SVM by investigating a hybrid Learning Distance Metric and Directed Acyclic Graph SVM (LDM-DAGSVM) model for multimodal biometric recognition, where LDM and DAGSVM are two emerging techniques in dealing with classification problems. Different from existing multimodal biometric recognition methods, the proposed approach aims to learn Mahalanobis distance metric via kernel SVM to maximize the inter-class variations and minimize the intra-class variations, simultaneously. Experimental results on the uncontrolled datasets such as AR face and AWE ear datasets show that the proposed approach achieves competitive performance compared with models working on individual modalities and overperforms the state-of-the-art multimodal methods. The proposed model achieves five-fold classification accuracy around 99.85 % for the face and ear images.

In this paper, we propose an optical multimodal biometric encryption technique which uses digital holography (DH) for multiple biometric recording and successive optical encryption methods, by means of keys generated from biometric data. Physiological biometrics, such as fingerprints or iris scans, along with behavioral biometrics such as voice, are recorded simultaneously as multimodal biometrics, using multi-parameter off-axis DH. Moreover, multiple biometrics embedded in the same holograms are encrypted using the Fresnel domain double random phase encoding method, in which keys are generated from biometric data via a phase retrieval algorithm. By employing the decryption procedure for this encoding method, along with the correct keys and reconstruction parameters of DH, the original multimodal biometrics can be successfully retrieved . The proposed method provides multimodal biometrics with a higher level of security by availing itself of the advantages of optical techniques. Our proposed multimodal biometric security scheme is demonstrated and validated by the results of the optical recording and encryption of biometric data presented below.

When compared to unimodal systems, multimodal biometric systems have several advantages, including lower error rate, higher accuracy, and larger population coverage. However, multimodal systems have an increased demand for integrity and privacy because they must store multiple biometric traits associated with each user. In this paper, we present a deep learning framework for feature-level fusion that generates a secure multimodal template from each user's face and iris biometrics. We integrate a deep hashing (binarization) technique into the fusion architecture to generate a robust binary multimodal shared latent representation. Further, we employ a hybrid secure architecture by combining cancelable biometrics with secure sketch techniques and integrate it with a deep hashing framework, which makes it computationally prohibitive to forge a combination of multiple biometrics that passes the authentication. The efficacy of the proposed approach is shown using a multimodal database of face and iris and it is observed that the matching performance is improved due to the fusion of multiple biometrics. Furthermore, the proposed approach also provides cancelability and unlinkability of the templates along with improved privacy of the biometric data. Additionally, we also test the proposed hashing function for an image retrieval application using a benchmark dataset. The main goal of this paper is to develop a method for integrating multimodal fusion, deep hashing, and biometric security, with an emphasis on structural data from modalities like face and iris. The proposed approach is in no way a general biometrics security framework that can be applied to all biometrics modalities, as further research is needed to extend the proposed framework to other unconstrained biometric modalities.

Blockchain technology has been commonly used in the last years in numerous fields, such as transactions documenting and monitoring real assets (house, cash) or intangible assets (copyright, intellectual property). The internet of things (IoT) technology, on the other hand, has become the main driver of the fourth industrial revolution, and is currently utilized in diverse fields of industry. New approaches have been established through improving the authentication methods in the blockchain to address the constraints of scalability and protection in IoT operating environments of distributed blockchain technology by control of a private key. However, these authentication mechanisms do not consider security when applying IoT to the network, as the nature of IoT communication with numerous entities all the time in various locations increases security risks resulting in extreme asset damage. This posed many difficulties in finding harmony between security and scalability. To address this gap, the work suggested in this paper adapts multimodal biometrics to strengthen network security by extracting a private key with high entropy. Additionally, via a whitelist, the suggested scheme evaluates the security score for the IoT system with a blockchain smart contract to guarantee that highly secured applications authenticate easily and restrict compromised devices. Experimental results indicate that our system is existentially unforgeable to an efficient message attack, and therefore, decreases the expansion of infected devices to the network by up to 49 percent relative to traditional schemes.

Joint discriminative feature learning for multimodal finger recognition

Credit Card Reader with Face Recognition Based on Webcam and Multimodal Biometrics

In this paper, we propose a new method for fuzzy adaptation of the Gap Generation and mutation parameters in Genetic algorithms to optimize Fuzzy Systems used as integration methods in modular neural networks for multimodal biometrics. The Genetic Algorithm is an optimization method inspired on the evolutionary ideas of natural selection and genetics; therefore, we propose an improvement to the convergence of the genetic algorithms using fuzzy logic. Simulation results show that the proposed approach improves the performance of Genetic Algorithms. A comparison of the proposed method using type-1 fuzzy logic for dynamic parameter adaptation with respect to the original Genetic Algorithms approach is presented. Additionally, a statistical test is presented to prove the performance enhancement in the application provided by fuzzy parameter adaptation in the genetic algorithm. The main contribution in this work is the fuzzy adaptation of parameters in the genetic algorithm using type-1 fuzzy logic and with this finding the optimal values of the parameters of the fuzzy integrators, to improve the recognition percentage of the modular neural network for multimodal biometrics.

With the increasing demand for information security and security regulations all over the world, biometric recognition technology has been widely used in our everyday life. In this regard, multimodal biometrics technology has gained interest and became popular due to its ability to overcome a number of significant limitations of unimodal biometric systems. In this paper, a new multimodal biometric human identification system is proposed, which is based on a deep learning algorithm for recognizing humans using biometric modalities of iris, face, and finger vein. The structure of the system is based on convolutional neural networks (CNNs) which extract features and classify images by softmax classifier. To develop the system, three CNN models were combined; one for iris, one for face, and one for finger vein. In order to build the CNN model, the famous pertained model VGG-16 was used, the Adam optimization method was applied and categorical cross-entropy was used as a loss function. Some techniques to avoid overfitting were applied, such as image augmentation and dropout techniques. For fusing the CNN models, different fusion approaches were employed to explore the influence of fusion approaches on recognition performance, therefore, feature and score level fusion approaches were applied. The performance of the proposed system was empirically evaluated by conducting several experiments on the SDUMLA-HMT dataset, which is a multimodal biometrics dataset. The obtained results demonstrated that using three biometric traits in biometric identification systems obtained better results than using two or one biometric traits. The results also showed that our approach comfortably outperformed other state-of-the-art methods by achieving an accuracy of 99.39%, with a feature level fusion approach and an accuracy of 100% with different methods of score level fusion.

A fast feature selection technique in multi modal biometrics using cloud framework

Voice assistant devices function as interaction gateways in the Internet-of-Things (IoT) smart home. By using voice assistants, users are able to control smart homes via speech commands. However, voice assistants introduce potential security risks and privacy disclosures. For example, malicious actors could impersonate genuine users to send smart home speech commands. Speaker/user verification thus becomes a critical issue for smart home security. This article proposes a secure method for speaker verification in IoT smart homes using millimeter-wave (mmWave) radar. Specifically, we utilize the radar to capture both vocal cord vibration (VCV) and lip motion (LM) as multimodal biometrics for identifying speakers. Traditional voice-based speaker verification methods are vulnerable to impostor attacks, such as replay attacks and voice synthesis attacks, that use recorded or imitated voice audio to spoof the system. Our approach is able to protect IoT smart homes from these attacks by continuously detecting the liveness of the user using mmWave sensing and deep learning techniques. Extensive experiments show that the proposed approach can achieve high verification accuracy and be more robust against imposter attacks.

Deployment of biometric systems in the applications of real world includes the most of unimodal biometric systems. Unimodal biometric system based on the information collected from single source. Sometimes single source of information may not identify the individual correctly because of some limitations such as Non-universality, Noisy data, Intra-class variation, Spoof attacks and Intra-class similarities. Various limitations of unimodal biometric systems are overridden by the multimodal biometric systems which involves multiple sources of information. Multimodal systems can be constructed by fusing of information of multiple modalities. This fusion can take place at various steps of processing such as at image acquisition, extraction of features of the traits, matching of test vectors with trained vectors and during decision taking based on classification. This paper presents a system of multimodal biometrics using face and voice biometric traits by including four fusion methods. Fusion takes place at i) feature level using concatenation of face and voice features, ii) score level using method involving the maximum of mode of scores obtained from two matchers, iii) rank level using borda count & iv) decision level fusion using logical conjunction (AND). Fusing of Log Gabor & Local Binary Pattern (LBP) takes place at the facial feature extraction. The voice features are also fused using Mel Frequency Ceptral coefficients (MFCCs) and Linear Predictive Coefficient features (LPC). Computation of similarity between test feature vectors and training vectors is carried out using Euclidian distance during matching process. KNN Classifier is used during decision making. Performance evaluation of these techniques are also carried out using performance measures such as Accuracy, False Acceptance Rate (FAR), False Rejection Rate (FRR) and ROC curves.

The awareness of the Biometric mechanism to the customer is an essential one during online transactions. Till now, the single biometric mechanism is used in various fields like banking, forensic, etc. However, the security issues are still lagging under many aspects, which show the need for stronger security levels; thus moving to multimodal biometrics. This paper aims to investigate customer awareness of the biometric mechanism (both unimodal and multimodal) during online transactions. Initially, questionnaire preparation is done based on certain core perspectives and is distributed among customers. Once the data collection is over (responses), this work makes an analytical study based on all the four core perspectives called customer awareness, perception/feedback, unimodal vs. multimodal, and data privacy. The initial one defines how much awareness the customers have about the biometrics usage in the online transaction. The second phase defines the thought on the usage of biometrics by the customer and third is the awareness about the unimodal or multimodal biometric facility and the final one is about the privacy preservation of data using multimodal biometric. The analysis is made concerning the overall model assessment, measurement model, and structural model. The experiment is conducted on 108 samples with measurement variables 45 and latent constructs 4. The analysis shows that 93.4% of the customers have ensured the use of multimodal biometrics in the future, as well as 68.87% of customers, assured the interested in buying the online product if the multimodal biometrics facility is available. From this analysis, the statistics show that it is worth investigation on the interestingness and awareness of the consumers in implementing multimodal biometrics.

Multimodal biometrics is an emerging technology for distributed data security. Single and multi-user data authentication plays a vital role in commercial or e-governance applications. Many approaches have been implemented in literature to secure the single user data using biometric security systems. Most of these systems are based on static initialization parameters and fixed multi-modal biometric features for data authentication. Also, traditional multi-modal biometric based data authentication schemes are independent of dynamic variation in integrity verification. In order to overcome these problems, a new multi-user based multi-modal authentication framework is designed and implemented on large image data types. In this framework, different biometric features such as IRIS, facial and fingerprint features are used to find the unique integrity of user for data authentication and security process. A new integrity computational algorithm and encryption technique are implemented to provide the strong data integrity verification and data security in distributed applications. Experimental results show that the proposed multi-modal integrity-based encryption model has nearly 7% of computational integrity bit change and 5% of runtime on large dataset.

In multimodal biometric systems, the fusion of information is an important step. Information can be fused at the different levels, i.e., at the feature, matching or decision level, of the recognition system. In this research feature fusion of three biometric features viz. face, profile face, and ear has fused using discriminant correlation analysis. Discriminant correlation analysis performs an effective functional fusion through maximizing the pair correlations of the two feature sets and simultaneously eliminating correlations between classes and limiting correlations to the classes. In pattern recognition applications, the proposed method can be used to fuse features extracted from multiple modalities or combine different vectors extracted from a single modality. Many sets of experiments carried out on different biometric databases and using various techniques for extraction of the features demonstrate the effectiveness of the proposed method, which results in more advanced approaches.

Security and authentication which are a major concern in this internet era, need to be addressed and one way to achieve this is to use multi-model biometrics. Multi-model biometrics offer more flexibility than unimodal biometrics. In this proposed system, face and retina biometrics are used. We created database comprising of face and retina images belonging to 7 different individuals, extracted their features, fused them together and latter encryption is performed on it. During authentication if the decrypted result of retina and face belong to same individual access is granted or else denied. In this project security is enhanced by adopting encryption of user data.

Now-a-days, in the field of machine learning the data augmentation techniques are common in use, especially with deep neural networks, where a large amount of data is required to train the network. The effectiveness of the data augmentation technique has been analyzed for many applications; however, it has not been analyzed separately for the multimodal biometrics. This research analyzes the effects of data augmentation on single biometric data and multimodal biometric data. In this research, the features from two biometric modalities: fingerprint and signature, have been fused together at the feature level. The primary motivation for fusing biometric data at feature level is to secure the privacy of the user’s biometric data. The results that have been achieved by using data augmentation are presented in this research. The experimental results for the fingerprint recognition, signature recognition and the feature-level fusion of fingerprint with signature have been presented separately. The results show that the accuracy of the training classifier can be enhanced with data augmentation techniques when the size of real data samples is insufficient. This research study explores that how the effectiveness of data augmentation gradually increases with the number of templates for the fused biometric data by making the number of templates double each time until the classifier achieved the accuracy of 99%.

The authentication of the Wireless Body Area Networks (WBANs) nodes is a vital factor in its medical applications. This paper, investigates methods of authentication over these networks. Also, an effective unimodal and multimodal biometrics identification approaches based on individual face and voice recognition or combined using different fusion types are presented. The cryptography and non-cryptography-based authentication are discussed in this research work and its suitability with the medical applications. Cryptographic based authentication is not suitable for WBANs. The biometrics authentication is discussed and its challenges. In this work, different fusion types in multimodal biometric are presented. There are two unimodal schemes have been presented based on using the voice and face image individually, these two biometrics have been used in the multimodal biometric scheme. The presneted multimodal scheme is evaluated and applied using the feature and score fusion. The mechanism operation of presented algorithm starts with capturing the biometics signals ‘Face/Voice’, the second step is the feature extracting from each biometric individually. The Artificial Neural Network (ANN), The Support Vector Machine (SVM) and the Gaussian Mixture Model (GMM) classifiers have been employed to perform the classification process individually. The computer simulation experiments reveal that the cepstral coefficients and statistical coefficients for voice recognition performed better for the voice scenario. Also, the Eigenface and support vector machine tools in the face recognition scheme performed better than other schemes. The multimodal results better than the unimodal schemes. Also, the results of the scores fusion-based multimodal biometric scheme is better than the feature fusion-based scheme. Hence, the biometric-based authentication is effective and applicable for the WBANs authentication and personality continuous authentication on these medical applications wireless networks.

In the last decade, the biometrics refers to automatic recognition of persons using their physiological or behavioral characteristics. The combination of multiple biometrics or, multimodal biometrics have higher accuracy to verify the person and ensure that its information or data is safer compared to system based on single biometrics modality. In this regard, this paper introduces a scheme for multimodal biometric recognition system based on the fusion of finger-vein and face images using Convolutional Neural Network (CNN) and different classifiers. The pre-processed finger-vein image using Adaptive Histogram Equalization (AHE) is input into a CNN model. Then, Random Forest (RF) classifier performs as a recognizer. In addition, a hybrid CNN-Linear Support Vector Machine (SVM) model is used for recognizing face images. After this process, the score level fusion of bimodal biometric based on the weighted concatenation is applied to identify the identity of the individual. Experimental results on publicly available VERA Fingervein database, Color Feret and Ar face database have shown significant capability of identification biometric system. The proposed system provides high recognition accuracy rate by 99,98% compared with other classical methods and traditional techniques based on normal recognition or CNN architectures.