Multimodal Authentication Research Articles

Serial Multimodal Biometrics Authentication and Liveness Detection Using Speech Recognition with Normalized Longest Word Subsequence Method

Biometric authentication aims to verify whether an entity matches the claimed identity based on biometric data. Despite its advantages, vulnerabilities, particularly those related to spoofing, still exist. Efforts to mitigate these vulnerabilities include multimodal approaches and liveness detection. However, these strategies may potentially increase resource requirements in the authentication process. This paper proposes a multimodal authentication process incorporating voice and facial recognition, with liveness detection applied to voice data using speech recognition. This paper introduces Normalized Longest Word Subsequence (NLWS), a combination of Intersection Over Union (IOU) and the longest common subsequence, to compare the prompted system sentence with the user's spoken sentence at speech recognition. Unlike the Word Error Rate (WER), NLWS has a measurable range between 1 and 0. Furthermore, the paper introduces decision-level fusion in the multimodal approach, employing two threshold levels in voice authentication. This approach aims to reduce resource requirements while enhancing the overall security of the authentication process. This paper uses cosine similarity, Euclidean distance, random forest, and extreme gradient boosting (XGBoost) to measure distance or similarity. The results show that the proposed method has better accuracy compared to unimodal approaches, achieving accuracies of 98.44%, 98.83%, 97.46%, and 99.22% using cosine similarity, Euclidean distance, random forest, and XGBoost calculations. The proposed method also demonstrates resource savings, reducing from 5.19 MB to 0.792 MB, from 7.3294 MB to 1.9437 MB, from 6.6512 MB to 1.3284 MB, and from 7.8632 MB to 2.1517 MB in different distance or similarity measurements

AI Powered Contactless Fingerprint Recognition

This comprehensive review delves into the realm of AI-powered contactless fingerprint recognition. We trace its evolution from traditional methods to modern contactless approaches, highlighting the pivotal role of AI and machine learning. Key topics include the principles of contactless fingerprint data acquisition, challenges like environmental variations, and privacy concerns. We examine the state-of-the-art AI techniques such as deep learning and convolutional neural networks, showcasing how they enhance accuracy and robustness in contactless fingerprint recognition. The paper also explores diverse applications, spanning security, access control, and healthcare, while addressing societal impacts and ethical considerations. Furthermore, we provide insights into ongoing research and future prospects, including potential improvements, emerging trends, and multi-modal authentication possibilities. This concise review serves as an essential resource for researchers, practitioners, and policymakers interested in the intersection of AI and biometric authentication technologies. Keywords: - AI-powered, Contactless fingerprint recognition, Biometrics, Artificial intelligence, Machine learning, Data acquisition, Deep learning, Convolutional neural networks

Multimodal Authentication System with Gender Classification

Multimodal authentication system which takes input as face, fingerprint and iris images and fusion of these images is done after the feature extraction. The fused images is used to authenticate the user. If the user is authenticated user then it also classifies the gender of the user

Multimodal Biometric Authentication System Using of Autoencoders and Siamese Networks for Enhanced Security

Ensuring secure and reliable identity verification is crucial, and biometric authentication plays a significant role in achieving this. However, relying on a single biometric trait, unimodal authentication, may have accuracy and attack vulnerability limitations. On the other hand, multimodal authentication, which combines multiple biometric traits, can enhance accuracy and security by leveraging their complementary strengths. In the literature, different biometric modalities, such as face, voice, fingerprint, and iris, have been studied and used extensively for user authentication. Our research introduces a highly effective multimodal biometric authentication system with a deep learning approach. Our study focuses on two of the most user-friendly safety mechanisms: face and voice recognition. We employ a convolutional autoencoder for face images and an LSTM autoencoder for voice data to extract features. These features are then combined through concatenation to form a joint feature representation. A Siamese network carries out the final step of user identification. We evaluated our model’s efficiency using the OMG-Emotion and RAVDESS datasets. We achieved an accuracy of 89.79% and 95% on RAVDESS and OMG-Emotion datasets, respectively. These results are obtained using a combination of face and voice modality.

Multi-modal Authentication Model for Occluded Faces in a Challenging Environment

Multi-modal Authentication Model for Occluded Faces in a Challenging Environment

A Literature Survey on Personalized AI-Based Voice Assistant for Authentication

This literature review critically examines the evolving landscape of personalized AI-based voice assistants in the realm of authentication services, The review begins with an exploration of the development and nuanced functionalities of voice recognition technologies, emphasizing their integration in multi-factor authentication systems. It delves into the synergistic relationship between voice and facial recognition technologies, highlighting advancements and challenges in biometric authentication, including accuracy, user experience, and security vulnerabilities.A significant focus is placed on the role of AI in personalizing user interactions with authentication services, where adaptive learning and contextual awareness are pivotal.The review also addresses the enduring relevance of traditional security measures, such as secure PIN’s, and their interplay with biometric methods to fortify authentication processes.The economic and commercial implications of these technologies are explored, considering their impact on business models and market trends.This comprehensive review identifies gaps in the current research landscape and suggests potential directions for future inquiry, particularly in enhancing security measures and improving user-centric designs. It aims to provide a holistic understanding of the current state and future potential of personalized AI-based voice assistants in authentication services, offering valuable insights for researchers and practitioners in the field. Key Words: Voice Recognition Technologies, Facial Recognition, Multimodal Authentication, Secure PINs, Biometric Authentication.

PUFGuard: Vehicle-to-Everything Authentication Protocol for Secure Multihop Mobile Communication

Vehicle area networks (VANs) encompass a spectrum of communication modes, including point-to-point visible light communication, 5G/6G cellular wireless communication, and Wi-Fi ad hoc multihop communication. The main focus of this paper is the introduction and application of physically unclonable functions (PUFs) as a pivotal element in secure key generation, authentication processes, and trust metric definition for neighboring vehicles. The multifaceted protocols proposed herein encompass comprehensive security considerations, ranging from authentication and anonymity to the imperative aspects of the proof of presence, freshness, and ephemeral session key exchanges. This paper provides a systematic and comprehensive framework for enhancing security in VANs, which is of paramount importance in the context of modern smart transportation systems. The contributions of this work are multifarious and can be summarized as follows: (1) Presenting an innovative and robust approach to secure key generation based on PUFs, ensuring the dynamic nature of the authentication. (2) Defining trust metrics reliant on PUFs to ascertain the authenticity and integrity of proximate vehicles. (3) Using the proposed framework to enable seamless transitions between different communication protocols, such as the migration from 5G/6G to Wi-Fi, by introducing the concept of multimodal authentication, which accommodates a wide spectrum of vehicle capabilities. Furthermore, upholding privacy through the encryption and concealment of PUF responses safeguards the identity of vehicles during communication.

RHEMAT: Robust human ear based multimodal authentication technique

Ear biometrics is a fast-developing field of computer vision that offers several advantages over existing biometric authentication methods. The performance of the ear is superior to or comparable to that of other biometric traits. However, the recognition performance of 2D ear images degrades due to the impacts of posture, illumination, and scaling. To overcome these challenges, we propose combining 3D and 2D ear images and constructing a keypoint detector and descriptor using a covariance matrix derived from texture and geometry information. The feature vector of the descriptor is then used to describe keypoints and utilized to identify keypoints with the smallest distance between a probe and gallery for the subsequent registration process. The registration error that results serves as the matching score. The proposed method is evaluated on a large database of ear images and compared to state-of-the-art techniques, demonstrating superior performance in occlusion, noise, and pose variations.

Multimodal biometric authentication method by federated learning

The Internet of Health Things requires rigid security policies to control access to sensitive data. However, nowadays, classic methods for user authentication may not meet the requirements for protection against unauthorized users during the collection, storage, and transmission of data. Therefore, there is a need for the evolution of technologies that allows the authentication of users based on unique personal identifiers (biometric characteristics). This work presents a security management approach for authentication that stands out for using two combined convolutional neural networks (CNN) for the biometric identification of users. The new approach relies on Federated Learning (FL) which is a Machine Learning paradigm that can support data management and privacy by training decentralized models collaboratively without effective data exchange. The new approach also combines Photoplethysmography and Electrocardiogram signals which improves identification accuracy and establishes a multimodal authentication. In sum, the new security management approach achieves high Accuracy, and a low false acceptance rate, guaranteeing protection against unauthorized access attempts.

English

A multi modality biometric authentication system can combine information from various modalities and provides accurate results compared to biometric systems used individually. A novel ensemble classifier-based multimodal biometric authentication system has been proposed in this work. The performance of the proposed multimodal authentication system is measured using parameters such as Accuracy, Sensitivity and Specificity and compared with the SVM classifier, Decision tree classifier when fingerprint, Iris, and Face features are used. The results of the multimodal biometric system are also compared with the biometric authentication system when fingerprint features are used and combined with Fingerprint & Iris features. The proposed ensemble classifier-based multimodal biometric authentication system provides an accuracy of 96.75%, Sensitivity of 94.74%, Specificity of 98.95%, FAR of 1.04 and FRR of 5.26. The proposed ensemble classifier outperforms SVM and decision tree classifiers regarding performance measures.

Optimal Score Level Fusion for Multi-Modal Biometric System with Optimised Deep Ensemble Technique

ABSTRACT Multi-modal biometric refers to the use of various biometric indicators for individual identification by personal recognition systems. When compared to unimodal biometrics, which uses only one biometric data, such as a fingerprint, face, palm print or iris, multi-modal authentication offers a higher degree of authentication. A new optimal score value that fuses deep learning and multi-modal biometrics would be produced in the project study. A proposed approach was split into three main groups: feature extraction, pre-processing and ensemble recognition. First, median filtering and ROI procedures were utilised for pre-processing-captured original biometric information for the wrist, dorsal, palm vein and palm print. Pertinent features are then retrieved from the corresponding preprocessed images for every modality. These extracted features are subjected to an imposter or genuine determination. Neural Network 1, Neural Network 2 and Deep Convolution Neural Network create a new deep ensemble model in the event of a forgery or accurate estimation (DCNN). The outputs of NN1 and NN2 are the inputs to DCNN, which provides information on whether the biometric data are authentic or not. Finally, the results are fine-tuned by the weight of DCNN utilising new hybrid optimisation scheme referred as Butterfly combined Tunicate Swarm Optimisation (BTSA).

Research of autoencoder-based user biometric verification with motion patterns

In the current research, we continue our previous study regarding motion-based user biometric verification, which consumes sensory data. Sensory-based verification systems empower the continuous authentication narrative – as physiological biometric methods mainly based on photo or video input meet a lot of difficulties in implementation. The research aims to analyze how various components of sensor data from an accelerometer affect and contribute to defining the process of unique person motion patterns and understanding how it may express the human behavioral patterns with different activity types. The study used the recurrent long-short-term-memory autoencoder as a baseline model. The choice of model was based on our previous research. The research results have shown that various data components contribute differently to the verification process depending on the type of activity. However, we conclude that a single sensor data source may not be enough for a robust authentication system. The multimodal authentication system should be proposed to utilize and aggregate the input streams from multiple sensors as further research.

User-centred multimodal authentication: securing handheld mobile devices using gaze and touch input

ABSTRACT Handheld mobile devices store a plethora of sensitive data, such as private emails, personal messages, photos, and location data. Authentication is essential to protect access to sensitive data. However, the majority of mobile devices are currently secured by singlemodal authentication schemes which are vulnerable to shoulder surfing, smudge attacks, and thermal attacks. While some authentication schemes protect against one of these attacks, only few schemes address all three of them. We propose multimodal authentication where touch and gaze input are combined to resist shoulder surfing, as well as smudge and thermal attacks. Based on a series of previously published works where we studied the usability of several user-centred multimodal authentication designs and their security against multiple threat models, we provide a comprehensive overview of multimodal authentication on handheld mobile devices. We further present guidelines on how to leverage multiple input modalities for enhancing the usability and security of user authentication on mobile devices.

Multimodal Personal Ear Authentication Using Acoustic Ear Feature for Smartphone Security

In recent years, biometric authentication technology for smartphones has become widespread, with the mainstream methods being fingerprint authentication and face recognition. However, fingerprint authentication cannot be used when hands are wet, and face recognition cannot be used when a person is wearing a mask. Like the face and fingers, the ear as a biometric contains features that enable human identification and has been the subject of research on personal authentication. Authentication systems based on the acoustic transfer function of the pinna (PRTF: Pinna Related Transfer Function) have been investigated. However, the authentication accuracy decreases due to the positional fluctuation across each measurement. In this paper, we propose multimodal personal authentication on smartphones using PRTF. The pinna image and positional sensor information are used with the PRTF, and the effectiveness of the authentication method is examined. Half total error rate (HTER) of 9.3% for single-modal authentication using only PRTF was improved to 1.6% for multimodal authentication using images and sensor data. We demonstrate that the proposed authentication system can compensate for the positional changes in each measurement and improve the robustness.

Secure Multimodal Authentication Scheme for Wireless Sensor Networks

In the current era, it is necessary to device authorization and authentication techniques to secure resources in information technology. There are several methods to substantiate authorization and authentication. User authentication is essential for authenticating user access control in WSNs. Biometric recognition error, lack of anonymity and vulnerability to attacks, user verification problem, revocation problem and disclosure of session key by the gateway node are some of the security flaws encountered. In this study, a Multimodal Authentication Scheme for Wireless Sensor Networks (WSN-MAS) is proposed to authenticate legitimate users. The main objective is the fusion of fingerprint and iris biometric features at feature level to enable additional accuracy to verify and match user identity with stored templates. In this paper, multimodal biometric features are used for authentication to improve performance, reduce system error rates to achieve better security in WSN.

Voice and Face Recognition for Web Browser Security

Abstract: This paper analysis about browser privacy multimodal authentication mechanisms. Face and voice recognition will be used as authentication methods in this process. The OpenCV library is used in the framework's face recognition section. It detects and recognizes faces from a database using basic eigen face recognition approaches. The MFCC (Mel Frequency Cepstrum Coefficients) and Gaussian Mixture Model are used to recognize voices. Following successful authentication, the cookies on the local hard disc are decrypted, allowing us access to the browser cookies. Initially, after a user registers, we will encrypt the browser cookies with AES, one of the most secure encryption methods available. keywords: MFCC, Gaussian Mixture Model, Browser cookies, authentication, AES, encryption, decryption, Open CV, Eigen.

A Deep Learning Framework for Intrusion Detection and Multimodal Biometric Image Authentication

Nowadays, a demand is increased all over the world in the field of information security and security regulations. Intrusion detection (ID) plays a significant role in providing security to the information, and it is an important technology to identify various threats in network during transmission of information. The proposed system is to develop a two-layer security model: (1) Intrusion Detection, (2) Biometric Multimodal Authentication. In this research, an Improved Recurrent Neural Network with Bi directional Long Short-Term Memory (I-RNN-BiLSTM) is proposed, where the performance of the network is improved by introducing hybrid sigmoid-tanh activation function. The intrusion detection is performed using I-RNN-BiLSTM to classify the NSL-KDD dataset. To develop the biometric multimodal authentication system, three biometric images of face, iris, and fingerprint are considered and combined using Shuffling algorithm. The features are extracted by Gabor, Canny Edge, and Minutiae for face, iris, and fingerprint, respectively. The biometric multimodal authentication is performed by the proposed I-RNN-BiLSTM. The performance of the proposed I-RNN-BiLSTM has been analysed through different metrics like accuracy, f-score, and confusion matrix. The simulation results showed that the proposed system gives better results for intrusion detection. Proposed model attains an accuracy of 98% for the authentication process and accuracy of 98.94% for the intrusion detection process.

Multi-modal biometric fusion based continuous user authentication for E-proctoring using hybrid LCNN-Salp swarm optimization.

In Covid 19, pandemic remote proctoring of the employee or human being is evolved as a big challenge for the information retrieval process. On the other side, memory-based system access authentication is becoming outdated and less preferred for live applications, especially where data security and customer privacy are crucial. Multi-modal authentication has outperformed the unimodal process with high accuracy and improved security in the user authentication field. Multi-modal biometric verification includes user attributes such as keystrokes, iris, speech, face, etc. For real-time execution of multi-modal biometric fusion-based live tracking for compatible applications. The study proposes an efficient continuous biometric user authentication system for a new challenge of pandemic time, a live online authentication of the evaluation process (CBUA-OE). The proposed CBUA-OE system can address the challenges associated with live proctoring and is also compatible with real-time implementation, deployment of authentication systems. The modified wolf optimization algorithm and CUBA-OE's optimal feature fusion algorithm give an edge over the other contemporary methods and make it more robust. In modern forms of authentication, the classification stage affects the overall outcome of the system, and the model's performance is also a factor of varying quality of datasets. In contrast, a hybrid LCNN-Salp swarm optimization-based classifier is more efficient and consistent in continuous user authentication. Here the performance of the proposed hybrid LCNN-Salp swarm optimization classifier is analyzed with different standard datasets. The results are compared with the existing state-of-art classifiers regarding the accuracy, precision, recall, and F-measure. This projected work is novel in terms of usability factors and scalability to live tracking systems.

Design and Implementation of Continuous Authentication Mechanism Based on Multimodal Fusion Mechanism

Most of the current authentication mechanisms adopt the “one-time authentication,” which authenticate users for initial access. Once users have been authenticated, they can access network services without further verifications. In this case, after an illegal user completes authentication through identity forgery or a malicious user completes authentication by hijacking a legitimate user, his or her behaviour will become uncontrollable and may result in unknown risks to the network. These kinds of insider attacks have been increasingly threatening lots of organizations, and have boosted the emergence of zero trust architecture. In this paper, we propose a Multimodal Fusion-based Continuous Authentication (MFCA) scheme, which collects multidimensional behaviour characteristics during the online process, verifies their identities continuously, and locks out the users once abnormal behaviours are detected to protect data privacy and prevent the risk of potential attack. More specifically, MFCA integrates the behaviours of keystroke, mouse movement, and application usage and presents a multimodal fusion mechanism and trust model to effectively figure out user behaviours. To evaluate the performance of the MFCA, we designed and implemented the MFCA system and the experimental results show that the MFCA can detect illegal users in quick time with high accuracy.

Robust multimodal biometric authentication on IoT device through ear shape and arm gesture

Nowadays, authentication is required for both physical access to buildings and internal access to computers and systems. Biometrics are one of the emerging technologies used to protect these highly sensitive structures. However, biometric systems based on a single trait enclose several problems such as noise sensitivity and vulnerability to spoof attacks. In this regard, we present in this paper a fully unobtrusive and robust multimodal authentication system that automatically authenticates a user by the way he/she answers his/her phone, after extracting ear and arm gesture biometric modalities from this single action. To deal the challenges facing ear and arm gesture authentication systems in real-world applications, we propose a new method based on image fragmentation that makes the ear recognition more robust in relation to occlusion. The ear feature extraction process has been made locally using Local Phase Quantization (LPQ) in order to get robustness with respect to pose and illumination variation. We also propose a set of four statistical metrics to extract features from arm gesture signals. The two modalities are combined on score-level using a weighted sum. In order to evaluate our contribution, we conducted a set of experiments to demonstrate the contribution of each of the two biometrics and the advantage of their fusion on the overall performance of the system. The multimodal biometric system achieves an equal error rate (EER) of 5.15%.