Biometric Research Articles

LSTM-Based Multimodal Model for Self-Harm Prediction Combining Mental Health Case Reports and Biometric Signals

LSTM-Based Multimodal Model for Self-Harm Prediction Combining Mental Health Case Reports and Biometric Signals

Banning biometric mind reading: the case for criminalising mind probing

ABSTRACT Several emerging technologies afford a specific type of interference with privacy which has not yet been conceptualised: inferring information about mental properties of persons on the basis of physiological or biometric signals, in short: mind probing. This paper conceptualises it, analyses several forms and proposes concrete norms that ban and criminalise it. Facial emotion recognition, neuroimaging, or wearable consumer devices such as smartwatches process biometric data to draw inferences about minds; the prevalence and sophistication of such mind probes will soon increase through AI data analysis. This paper explores how human rights law, the EU General Data Protection Regulation and the EU AI Act address mind probes and shows why the current level of protection appears insufficient. This motivates novel norms banning and criminalising non-consensual biometric-based forms of mind reading, ensuring that at least some parts of the mind remain in principle free from non-consensual mind probing.

Increasing adherence and collecting symptom-specific biometric signals in remote monitoring of heart failure patients: a randomized controlled trial.

Mobile health (mHealth) regimens can improve health through the continuous monitoring of biometric parameters paired with appropriate interventions. However, adherence to monitoring tends to decay over time. Our randomized controlled trial sought to determine: (1) if a mobile app with gamification and financial incentives significantly increases adherence to mHealth monitoring in a population of heart failure patients; and (2) if activity data correlate with disease-specific symptoms. We recruited individuals with heart failure into a prospective 180-day monitoring study with 3 arms. All 3 arms included monitoring with a connected weight scale and an activity tracker. The second arm included an additional mobile app with gamification, and the third arm included the mobile app and a financial incentive awarded based on adherence to mobile monitoring. We recruited 111 heart failure patients into the study. We found that the arm including the financial incentive led to significantly higher adherence to activity tracker (95% vs 72.2%, P = .01) and weight (87.5% vs 69.4%, P = .002) monitoring compared to the arm that included the monitoring devices alone. Furthermore, we found a significant correlation between daily steps and daily symptom severity. Our findings indicate that mobile apps with added engagement features can be useful tools for improving adherence over time and may thus increase the impact of mHealth-driven interventions. Additionally, activity tracker data can provide passive monitoring of disease burden that may be used to predict future events.

Fuzzy logic and biometric-based lightweight cryptographic authentication for metaverse security

As Metaverse advances, robust authentication systems become more critical. Traditional security measures are not efficient and adaptable for metaverse scenarios. In this context, this study proposed a two-phase security framework that combines fuzzy logic and convolutional neural network (CNN) techniques for biometric authentication and a lightweight Cryptographic protocol for securing the Metaverse. In the first stage, the CNN model is used to extract accurate biometric signatures from hand tremor data. This innovative method takes into account the inherent unpredictability of biometrics. During the second phase, a cryptographic scheme is used to provide safe mutual authentication between the user and the Metaverse infrastructure. Our findings, which have been verified by BAN Logic analysis, show that the proposed framework achieves a high level of accuracy in user authentication and has robust defensive capabilities against typical cyber-attacks.

Simple Siamese Model with Long Short-Term Memory for User Authentication with Field-Programmable Gate Arrays

Recent studies have focused on user authentication methods that use biometric signals such as electrocardiogram (ECG) and photo-plethysmography (PPG). These authentication technologies have advantages such as ease of acquisition, strong security, and the capability for non-aware authentication. This study addresses user authentication using electromyogram (EMG) signals, which are particularly easy to acquire, can be fabricated in a wearable form such as a wristwatch, and are readily expandable with technologies such as human–machine interface. However, despite their potential, they often exhibit lower accuracy (approximately 90%) than traditional methods such as fingerprint recognition. Accuracy can be improved using complex algorithms and multiple biometric authentication technologies; however, complex algorithms use substantial hardware resources, making their application to wearable devices difficult. In this study, a simple Siamese model with long short-term memory (LSTM) (SSiamese-LSTM) is proposed to achieve a high accuracy of over 99% with limited resources suitable for wearable devices. The hardware implementation was accomplished using field-programmable gate arrays (FPGAs). In terms of accuracy, EMG measurement results from Chosun University were used, and data from 100 individuals were employed for verification. The proposed digital logic will be integrated with an EMG sensor in the form of a watch that can be used for user authentication.

Continuous Authentication in the Digital Age: An Analysis of Reinforcement Learning and Behavioral Biometrics

This research article delves into the development of a reinforcement learning (RL)-based continuous authentication system utilizing behavioral biometrics for user identification on computing devices. Keystroke dynamics are employed to capture unique behavioral biometric signatures, while a reward-driven RL model is deployed to authenticate users throughout their sessions. The proposed system augments conventional authentication mechanisms, fortifying them with an additional layer of security to create a robust continuous authentication framework compatible with static authentication systems. The methodology entails training an RL model to discern atypical user typing patterns and identify potentially suspicious activities. Each user’s historical data are utilized to train an agent, which undergoes preprocessing to generate episodes for learning purposes. The environment involves the retrieval of observations, which are intentionally perturbed to facilitate learning of nonlinear behaviors. The observation vector encompasses both ongoing and summarized features. A binary and minimalist reward function is employed, with principal component analysis (PCA) utilized for encoding ongoing features, and the double deep Q-network (DDQN) algorithm implemented through a fully connected neural network serving as the policy net. Evaluation results showcase training accuracy and equal error rate (EER) ranging from 94.7% to 100% and 0 to 0.0126, respectively, while test accuracy and EER fall within the range of approximately 81.06% to 93.5% and 0.0323 to 0.11, respectively, for all users as encoder features increase in number. These outcomes are achieved through RL’s iterative refinement of rewards via trial and error, leading to enhanced accuracy over time as more data are processed and incorporated into the system.

Multipoint ECG Signal Extraction on Palms and Figures for Human Identification System

The ECG signal contain vital information for cardiac disease which is one of main cause of catastrophic mortality. In conventional technique, the signals are extracted through different points located on the limbs and surface of the chest. The biological impedance of human body eliminates any practical means of making fake copies of the relevant physiological traits. Moreover, this method needs the complex data acquisition system and apparatus as well as inconvenience way of acquisition of signal at the chest. Hence, there is a demand to find out the promising alternative technique to extract the ECG signals in more convenient way with high accuracy. In this paper, the real-time ECG monitor has been developed to extract the signals from the multipoint on the hands and fingers. Placement of the electrodes are done based on the acupressure point on palms and fingers. A total of 120 volunteers were investigated to develop the ECG database as the development dataset. This work uses the template matching algorithm and distance classification method to analyze and to find the best similarities between the developed dataset of ECG biometric signals to improve the identification rate. In order to lessen the noise that was recorded with the ECG signals, signal averaging was used to create ECG databases and templates. The recognition rate rose to 98% accuracy on the development dataset when the prescreening procedure was introduced to create a combined system model. ECG biometric model was created by combining the two models and using the development dataset's results. The algorithm was applied on the entire developed ECG dataset and 96% of accuracy rate was achieved in identification.

Detection of distracted driving through the analysis of real-time driver, vehicle, and roadway volatilities

Distracted driving adversely impacts drivers’ decision-making and leads to safety-critical events (SCEs). Early detection of driver distraction is critical to prevent traffic crashes by providing warning messages to drivers and the surrounding vehicles. This study harnesses real-time multidimensional data collected through sensors that examine the variations in driver biometrics, vehicle kinematics, and roadway surroundings in different driving scenarios conducted on a Multimodal Virtual Reality Simulator. The driving behaviors of the study participants were examined under various visual detection response tasks of increasing complexity. The study classifies driving behaviors on a 5-level ordinal scale by estimating a Panel Ordered Logit Model, Random Forest, and Artificial Neural Network, using real-time volatilities in driver biometric signals, vehicle speed and acceleration, and roadway surroundings. The study results reveal that the driver gaze and the coefficients of variation in vehicle speed, driver eye movements, vehicular distances from the lane centerline, and the following vehicle significantly impact distracted driving. The study’s findings align with the principles of the safe systems approach by emphasizing the development of proactive safety measures in the form of feedback and warning the driver and surrounding vehicles of a potential distracted driving event, helping to foster safer user behavior and vehicles.

Exploring the Possibility of Photoplethysmography-Based Human Activity Recognition Using Convolutional Neural Networks.

Various sensing modalities, including external and internal sensors, have been employed in research on human activity recognition (HAR). Among these, internal sensors, particularly wearable technologies, hold significant promise due to their lightweight nature and simplicity. Recently, HAR techniques leveraging wearable biometric signals, such as electrocardiography (ECG) and photoplethysmography (PPG), have been proposed using publicly available datasets. However, to facilitate broader practical applications, a more extensive analysis based on larger databases with cross-subject validation is required. In pursuit of this objective, we initially gathered PPG signals from 40 participants engaged in five common daily activities. Subsequently, we evaluated the feasibility of classifying these activities using deep learning architecture. The model's performance was assessed in terms of accuracy, precision, recall, and F-1 measure via cross-subject cross-validation (CV). The proposed method successfully distinguished the five activities considered, with an average test accuracy of 95.14%. Furthermore, we recommend an optimal window size based on a comprehensive evaluation of performance relative to the input signal length. These findings confirm the potential for practical HAR applications based on PPG and indicate its prospective extension to various domains, such as healthcare or fitness applications, by concurrently analyzing behavioral and health data through a single biometric signal.

Respiratory Analysis with Electrocardiogram Data: Evaluation of Pan-Tompkins Algorithm and Cubic Curve Interpolation Method

Advancements in bioinstrumentation have facilitated the easier monitoring of biometric signals such as electrocardiogram (ECG) and respiration. This development is particularly crucial for the diagnosis and management of various conditions like stress and sleep disorders. Two commonly used features in heart rate variability (HRV) analysis derived from ECG data are standard deviation and serial correlation coefficients of R-R intervals (the time durations between heartbeats). The former utilizes the fundamental components of QRS complexes, while the latter is designed to extract relationships between respiration and heart rate. In the proposed methodology, R-R wave detection is performed on processed ECG data using the Pan-Tompkins algorithm, and the respiration duration for each R-R interval from respiration data is selected. Additionally, missing respiration data for selected R-R intervals is interpolated based on the interpolation method. The results of this study are compared with the standard interpolation and cubic spline interpolation models to assess the effectiveness of the proposed method and its ability to capture temporal fluctuations. Since standard interpolation fails to accurately detect respiration data from R-R intervals and cannot precisely handle missing R-R intervals in short samples, cubic spline interpolation is recommended as a replacement and its results are presented. The obtained results provide insights into the effectiveness and application of the Pan-Tompkins algorithm, FFT (fast fourier transform) implementation, and cubic spline interpolation in the selection of respiration and R-wave features. According to the findings of the study, in the analysis conducted on 2-second samples with a 1000 Hz sampling frequency created from each participant's respiratory data set, missing respiratory data were successfully reconstructed from the R-R intervals of the EKG data using standard and cubic curve interpolation methods. Upon examination of RMSE (Root mean square error) values, it was observed that for 30% of the participants, as RMSE values increased, completion counts for standard interpolation increased, while completion counts for cubic curve interpolation decreased. Conversely, when RMSE values decreased, 60% of the participants showed a decrease in completion counts for standard interpolation and an increase in completion counts for cubic curve interpolation. A 10% participant group was identified where there was no apparent relationship between RMSE values and interpolation method. This indicates that in 90% of the participants, there is a linear relationship between the study's interpolation method, RMSE values, and completion counts for missing R-R intervals.

Validating EmotiBit, an open-source multi-modal sensor for capturing research-grade physiological signals from anywhere on the body

Peripheral physiological signals provide a powerful window into understanding the body and mind. Peripheral physiology devices that are currently available on the market, however, face a number of challenges. Consumer-grade devices (e.g. FitBit, Apple Watch) are easy to wear, but provide limited access to unprocessed data. Combined with black-box signal processing algorithms, this makes it difficult to interpret the data for scientific research purposes. Research-grade devices (e.g. Empatica, Shimmer, BIOPAC) provide greater access to high-quality data but remain in closed ecosystems and at price points that are out of reach for many. To bridge the gaps in available biometric solutions, our labs have created an open-source physiological sensing platform called EmotiBit (http://www.emotibit.com/), measuring EDA, multi-wavelength PPG, temperature, and 9-axis IMU. This study compares EmotiBit biometric signals to gold-standard devices by Brain Products and finds that the physiological signals exhibit a high degree of similarity, validating their use in research.

Multimodal Biometric Login System Using Face and Signature

Biometrics has developed to be one of the most relevant technologies used in Information Technology (IT) security. Uni-modal biometric systems have a variety of problems which decreases the performance and accuracy of these systems. One way to overcome the limitations of the Unimodal biometric systems is through fusion to form a multimodal biometric system. Hence this process is developed based on Multimodal Biometric system based on Face, Finger print, Signature etc. Multimodal biometric system employing Convolutional Neural Networks (CNNs) for effective feature extraction and classification. The system combines facial and signature biometrics, harnessing the unique advantages of each modality to create a more resilient and accurate authentication framework. Multimodal biometric system with CNN integration holds promise for applications in secure access control, financial transactions, and other domains where reliable authentication is crucial. Its adaptability and scalability make it a viable solution for addressing the evolving challenges in biometric security systems. Key Word: Unimodal biometric system , Multimodal Biometric system , Convolutional Neural Networks (CNNs),deep learning.

Biometric contrastive learning for data-efficient deep learning from electrocardiographic images.

Artificial intelligence (AI) detects heart disease from images of electrocardiograms (ECGs). However, traditional supervised learning is limited by the need for large amounts of labeled data. We report the development of Biometric Contrastive Learning (BCL), a self-supervised pretraining approach for label-efficient deep learning on ECG images. Using pairs of ECGs from 78 288 individuals from Yale (2000-2015), we trained a convolutional neural network to identify temporally separated ECG pairs that varied in layouts from the same patient. We fine-tuned BCL-pretrained models to detect atrial fibrillation (AF), gender, and LVEF < 40%, using ECGs from 2015 to 2021. We externally tested the models in cohorts from Germany and the United States. We compared BCL with ImageNet initialization and general-purpose self-supervised contrastive learning for images (simCLR). While with 100% labeled training data, BCL performed similarly to other approaches for detecting AF/Gender/LVEF < 40% with an AUROC of 0.98/0.90/0.90 in the held-out test sets, it consistently outperformed other methods with smaller proportions of labeled data, reaching equivalent performance at 50% of data. With 0.1% data, BCL achieved AUROC of 0.88/0.79/0.75, compared with 0.51/0.52/0.60 (ImageNet) and 0.61/0.53/0.49 (simCLR). In external validation, BCL outperformed other methods even at 100% labeled training data, with an AUROC of 0.88/0.88 for Gender and LVEF < 40% compared with 0.83/0.83 (ImageNet) and 0.84/0.83 (simCLR). A pretraining strategy that leverages biometric signatures of different ECGs from the same patient enhances the efficiency of developing AI models for ECG images. This represents a major advance in detecting disorders from ECG images with limited labeled data.

Improved Transfer Learning for Detecting Upper-Limb Movement Intention Using Mechanical Sensors in an Exoskeletal Rehabilitation System.

The objective of this study was to propose a novel strategy for detecting upper-limb motion intentions from mechanical sensor signals using deep and heterogeneous transfer learning techniques. Three sensor types, surface electromyography (sEMG), force-sensitive resistors (FSRs), and inertial measurement units (IMUs), were combined to capture biometric signals during arm-up, hold, and arm-down movements. To distinguish motion intentions, deep learning models were constructed using the CIFAR-ResNet18 and CIFAR-MobileNetV2 architectures. The input features of the source models were sEMG, FSR, and IMU signals. The target model was trained using only FSR and IMU sensor signals. Optimization techniques determined appropriate layer structures and learning rates of each layer for effective transfer learning. The source model on CIFAR-ResNet18 exhibited the highest performance, achieving an accuracy of 95% and an F-1 score of 0.95. The target model with optimization strategies performed comparably to the source model, achieving an accuracy of 93% and an F-1 score of 0.93. The results show that mechanical sensors alone can achieve performance comparable to models including sEMG. The proposed approach can serve as a convenient and precise algorithm for human-robot collaboration in rehabilitation assistant robots.

Enhancing Handwritten Signature Identification and Palm Biometric Objectives

Soft biometrics are already widely used as a support tool for user identification. However, it is not the only use for biometric information that is conceivable because such information can be sufficient to obtain minimal details from the user that are unrelated to his identity. Examples of what might be referred to as soft biometrics include gender, hand orientation, and emotional state. Utilizing physiologic modalities for soft-biometric work is extremely prevalent, prediction, but behavioral data is frequently disregarded. Keystroke dynamics and handwriting signature are two potential behavioral modalities that could be used to predict a user's gender, but they are rarely discussed in the literature together. This study seeks to fill this gap by examining the influence of combining these two distinct biometric modalities on the accuracy of gender prediction and the best way. Key Words: Item key-strokes, Bio-metric signatures, digital signs, dynamic temporal wrapping (DTW)

Taming Irregular Cardiac Signals for Biometric Identification

Cardiac patterns are being used to provide hard-to-forge biometric signatures in identification applications. However, this performance is obtained under controlled scenarios where cardiac signals maintain a relatively uniform pattern, facilitating the identification process. In this work, we analyze cardiac signals collected in more realistic (uncontrolled) scenarios and show that their high signal variability makes them harder to obtain stable and distinct features. When faced with these irregular signals, the state-of-the-art (SOTA) reduces its performance significantly. To solve these problems, we propose the CardioID framework 1 with two novel properties. First, we design an adaptive method that achieves stable and distinct features by tailoring the filtering process according to each user’s heart rate. Second, we show that users can have multiple cardiac morphologies, offering us a bigger pool of cardiac signals compared to the SOTA. Considering three uncontrolled datasets, our evaluation shows two main insights. First, while using a PPG sensor with healthy individuals, the SOTA’s balanced accuracy (BAC) reduces from 90–95% to 75–80%, while our method maintains a BAC above 90%. Second, under more challenging conditions (using smartphone cameras or monitoring unhealthy individuals), the SOTA’s BAC reduces to values between 65–75%, and our method increases the BAC to values between 75–85%.

A multimodal study of the interplay between stress, executive function, and biometrics in game-based assessment

Managing stress is a crucial soft skill that affects cognitive performance and health. Stress detection through biometrics can be used to improve and evaluate stress management. However, measuring the effects of stress on biometrics and executive functions is difficult and dependent on the individual. Despite these challenges, this paper presents a case study that collects a comprehensive multimodal dataset with two stress metrics, four biometric signals, and twenty-two executive function metrics from Game-based Assessment (GBA) trace data specifically designed for this purpose. The experiments suggest that biometrics, especially the heart rate and skin temperature, are effective predictors of stress. Additionally, noteworthy correlations were observed between heart rate and certain executive function variables. The levels of GBA that measured shifting and processing speed showed a higher heart rate than the response inhibition levels. This case study, together with the developed stress detectors, enables the detection of persons who struggle to manage stress and measure their executive function performance under stressful situations.

Multi-cut based architectural obfuscation and handprint biometric signature for securing transient fault detectable IP cores during HLS

This paper presents a novel dual defense security methodology for fault detectable reusable hardware intellectual property (IP) core against reverse engineering and piracy using multi-cut based architectural obfuscation and handprint biometric signature, useful for consumer electronics (CE) systems. The proposed approach, firstly generates a fault detectable design corresponding to target application framework. Subsequently, multiple cut insertion is executed resulting in multi-checkpointing which in turn camouflages the hardware design architecture during high level synthesis (HLS), thereby making it uninterpretable for an adversary. Next, handprint biometric signature of IP vendor is covertly embedded into the fault detectable IP design, which provides robust and seamless detective countermeasure against piracy. The proposed methodology achieves the following: a) strength of obfuscation ∼14 % b) robust detective control against piracy using handprint biometric signature at 0.0 % design cost overhead c) robust security in terms of digital proof against piracy as evident from lesser probability of coincidence and higher tamper tolerance.

P171: Identifying pre-agitation biometric signature in dementia patients: A feasibility study

Objectives:Agitation and aggression (AA) occur frequently in patients with dementia (PwD), are challenging to manage, and are distressing for PwD, families, caregivers, and healthcare systems. Physiological parameters, such as Actigraphy, Heart Rate Variability, and Electrodermal Activity, measured via wearable sensors are correlated with AA in PwD. It is unclear whether these parameters could be compiled into an operational algorithm to create a pre-agitation biometric marker (i.e. parameters of Autonomous Nervous System’s arousal: elevated EDA, more frequent HR, lower heart rate variability (HRV), as well as higher motor activity) capable of predicting episodes of AA. This study will assess the feasibility and clinical utility of collecting physiological parameters via wearable multi-sensor Empatica E4 device in relation to clinically recorded episodes of AA in PwD.Methods:This study is leveraging a clinical trial (ClinicalTrials.gov/NCT04516057) taking place at Ontario Shores Centre for Mental Health Sciences. Participants are inpatients, males and females, 55-years old or older, with clinically significant AA, and a diagnosis of a Major Neurocognitive Disorder due to Alzheimer’s disease or multiple aetiologies. Participants wear the E4 device for 48 to 72 hours on three occasions during the 8-week study period. Participant demographics, and clinical measures used to assess behavior are collected at specific time intervals during the study period.Results:The study is ongoing and currently to-date we have been able to acquire approximately 240 hours of recordings from patients. We will be presenting feasibility data (proportion of participants successfully completing a minimum 48-hours of recordings), correlation analysis between physiological measures and clinical measures to identify pre-agitation triggers. Further, we will use generalized linear models to test whether physiological measures can predict pre-agitation triggers. This study will allow estimation of sample size needed to detect a meaningful effect size, which will be determined from the prediction model. Deep learning using Python will be used to create a predictive algorithm using the physiological data to profile participants’ behaviors and detect pre-agitation triggers.Conclusion:Early detection of AA in PwD will allow caregivers to offer timely, ndividualized, non-medical or medical interventions which will help avoid crises and critical incidents and improve quality of life of the PwD and their caregivers.

Identifying pre-agitation biometric signature in patients with dementia: A feasibility study

Background:Agitation and aggression (AA) occur frequently in patients with dementia (PwD), and cause distress to PwD and caregivers. This study will investigate whether physiological parameters, such as actigraphy, heart rate variability, temperature, and electrodermal activity, measured via wearable sensors, correlate with AA in PwD. It will also explore whether these parameters could be compiled to create a pre-agitation biometric marker capable of predicting episodes of AA in PwD.Methods:This study will take place at Ontario Shores Centre for Mental Health Sciences. Thirty inpatient participants who are inpatients, males, and females, aged 60 or older, with clinically significant AA, and diagnosis of Major Neurocognitive Disorder will be recruited. Participants will wear the device for 48 to 72 hours on three occasions during an 8-week study period. Participant demographics and clinical measures used to assess behavior will be collected at specific time intervals during the study period.Ceiling mounted cameras and clinical data are collected to annotate episodes of AA, which will allow identification of peripheral physiological markers “signature” unique to the patientResults:the algorithm connecting wearable devices, cloud and cameras was tested on healthy volunteers and demonstrated feasibility and reliability. The feasibility of implementation in PwD has been demonstrated in our sample of PwD previously in a sample of 6 participants. Feasibility in this larger sample will be assessed. Correlation analysis between physiological measures, camera capture of agitation onset and clinical measures will be calculated to identify agitation events and pre-agitation triggers. Various machine learning and features extraction/exploration techniques will be used to test whether physiological measures can detect exact time of agitation and predict pre-agitation triggers. This study will provide a reasonable estimation of sample size needed to detect a meaningful effect size, which will be determined from the prediction model.Conclusion:Early detection of AA in PwD will allow caregivers to offer timely and personalized interventions which will help avoid crises and critical incidents and improve quality of life in PwD and their caregivers.