Face Tracking Research Articles

Assessment of autostereoscopic perception using artificial intelligence-enhanced face tracking technology.

Stereopsis, the ability of humans to perceive depth through distinct visual stimuli in each eye, is foundational to autostereoscopic technology in computing. However, ensuring stable head position during assessments has been challenging. This study evaluated the utility of artificial intelligence (AI)-enhanced face tracking technology in overcoming this challenge by ensuring that each eye consistently receives its intended image. The Lume Pad 2, an autostereoscopic tablet with AI-enhanced face tracking, was utilized to simulate quantitative parts of the Stereo Fly test and TNO Stereotests for contour and random dot stereopsis. The study recruited 30 children (14 males and 16 females, mean age of 9.2 ± 0.3 years, age range of 6-12 years) and 30 adults (10 males and 20 females, mean age of 29.4 ± 1.0 years, age range of 21-42 years) to assess the tablet's inter-session reliability. Agreement between conventional and the autostereoscopic tablet-simulated stereotests was tested in a larger group of 181 children (91 males and 90 females, mean age of 9.1 ± 0.4 years, age range of 6-12 years) and 160 adults (69 males and 91 females, mean age of 38.6 ± 2.1 years, age range of 21-65 years). Inter-session reliability and agreement were analyzed using weighted Kappa coefficient and non-parametric Bland-Altman analysis. The autostereoscopic tablet demonstrated high inter-session reliability (κ all > 0.80), except for the simulated TNO Stereotest in adults, which demonstrated moderate inter-session reliability (κ = 0.571). Non-parametric Bland-Altman analysis revealed zero median differences, confirming consistent inter-session reliability. Similar patterns were observed in comparing AI-based and conventional methods, with both the weighted Kappa coefficient (κ all > 0.80) and non-parametric Bland-Altman analysis indicating significant agreement. The agreement between methodologies was confirmed by permissible differences, which were smaller than the minimum step range. The integration of AI-based autostereoscopic technology with sub-pixel precision demonstrates significant potential for clinical stereopsis measurements.

버추얼 프로덕션을 위한 IMU 모션캡쳐 정확도와 활용에 대한 고찰

IMU-based motion capture systems are gaining attention as a low-cost, easy-to-install alternative to Optical Motion Capture (OMC) due to their wide capture volume. This study analyzes the technical features of IMU systems and compares their accuracy with OMC, reviewing correction techniques to address magnetic interference and sensor drift. Six prior studies showed that post-correction accuracy had an RMSE of 0.3-7.2, with sports movements achieving 79.32-100% accuracy. IMU systems are advantageous for real-time processing and live performance, making them valuable in virtual production. However, they have limitations in precise motion capture, such as facial capture, requiring hybrid system support and further technological advancements.

Sign Language Interpreting System Using Recursive Neural Networks

According to the World Health Organization (WHO), 5% of people around the world have hearing disabilities, which limits their capacity to communicate with others. Recently, scientists have proposed systems based on deep learning techniques to create a sign language-to-text translator, expecting this to help deaf people communicate; however, the performance of such systems is still low for practical scenarios. Furthermore, the proposed systems are language-oriented, which leads to particular problems related to the signs for each language. For this reason, to address this problem, in this paper, we propose a system based on a Recursive Neural Network (RNN) focused on Mexican Sign Language (MSL) that uses the spatial tracking of hands and facial expressions to predict the word that a person intends to communicate. To achieve this, we trained four RNN-based models using a dataset of 600 clips that were 30 s long; each word included 30 clips. We conducted two experiments; we tailored the first experiment to determine the most well-suited model for the target application and measure the accuracy of the resulting system in offline mode; in the second experiment, we measured the accuracy of the system in online mode. We assessed the system’s performance using the following metrics: the precision, recall, F1-score, and the number of errors during online scenarios, and the results computed indicate an accuracy of 0.93 in the offline mode and a higher performance for the online operating mode compared to previously proposed approaches. These results underscore the potential of the proposed scheme in scenarios such as teaching, learning, commercial transactions, and daily communications among deaf and non-deaf people.

Using an Artificial Intelligence (AI) Agent to Support Teacher Instruction and Student Learning

The options for Artificial intelligence (AI) tools used in teacher education are increasing daily, but more is only sometimes better for teachers working in already complex classroom settings. This team discusses the increase of AI in schools and provides an example from administrators, teacher educators, and computer scientists of an AI virtual agent and the research to support student learning and teachers in classroom settings. The authors discuss the creation and potential of virtual characters in elementary classrooms, combined with biometrics and facial emotional recognition, which in this study has impacted student learning and offered support to the teacher. The researchers share the development of the AI agent, the lessons learned, the integration of biometrics and facial tracking, and how teachers use this emerging form of AI both in classroom-based center activities and to support students’ emotional regulation. The authors conclude by describing the application of this type of support in teacher preparation programs and a vision of the future of using AI agents in instruction.

Dynamic 4D facial capture pipeline with appearance driven progressive retopology based on optical flow

This paper presents a production-oriented 4D facial reconstruction pipeline designed to produce high-fidelity facial mesh sequences with a consistently structured topology, while preserving the wireframe structure specified by artists. We have designed and developed a compact, efficient, and fast optical capture system based on synchronized camera arrays for high-precision dynamic 3D facial imaging. Unlike prevailing methods that primarily concentrate on single-frame reconstruction, often reliant on labor-intensive manual annotation, our framework exploits the constraint of appearance consistency to autonomously establish feature correspondence and uphold temporal coherence within the mesh. Consequently, our approach eliminates mesh drifting and jitter, enabling full parallelization for dynamic facial expression capture. The proposed pipeline decouples the non-linear deformation of facial expressions from the rigid movements of the skull through a stable external device. Leveraging progressive retopology, our methodology employs artist-guided templates as priors, ensuring the preservation of wireframe structures across the result sequence. Progressive retopology is achieved by constraining different fine-grained features of 3D landmarks, scan surface shapes, and appearance textures. The results of our study showcase facial mesh sequences with production-quality topology, adept at faithfully reproducing character expressions from photographs while achieving artist-friendly stable facial movements.

Micro and macro facial expressions by driven animations in realistic Virtual Humans

Computer Graphics (CG) advancements have allowed the creation of more realistic Virtual Humans (VH) through modern techniques for animating the VH body and face, thereby affecting perception. From traditional methods, including blend shapes, to driven animations using facial and body tracking, these advancements can potentially enhance the perception of comfort and realism in relation to VHs. Previously, Psychology studied facial movements in humans, with some works separating expressions into macro and micro expressions. Also, some previous CG studies have analyzed how macro and micro expressions are perceived, replicating psychology studies in VHs, encompassing studies with realistic and cartoon VHs, and exploring different VH technologies. However, instead of using facial tracking animation methods, these previous studies animated the VHs using blendshapes interpolation. To understand how the facial tracking technique alters the perception of VHs, this paper extends the study to macro and micro expressions, employing two datasets to transfer real facial expressions to VHs and analyze how their expressions are perceived. Our findings suggest that transferring facial expressions from real actors to VHs significantly diminishes the accuracy of emotion perception compared to VH facial animations created by artists.

Confusion and Diffusion Techniques for Image Encryption Process Based on Chaos System

Face recognition uses biometric technologies to identify humans based on their facial characteristics. This method is commonly used to restrict information access control. The benefits of face recognition systems encompass their ease of use and security. The human face recognition process consists of face detection, face tracking, and face recognition. The process uses some algorithms: the Viola-Jones for face detection, the Kanade-Lucas-Tomasi (KLT) for face tracking, and the principal component analysis (PCA) for face recognition. Furthermore, this research proposed face recognition with an encryption process to protect the data stored in a database. The encryption process consists of two main processes: confusion and diffusion. The confusion process is randomizing the position of the original image pixels. This research utilized the Arnold’s cat map (ACM) for the confusion process, and the diffusion process was performed using the XOR operation with the key generated from the 1D chaos system. Three different 1D chaos systems, namely logistic map, Bernoulli map, and tent map, were compared to see which chaos system had the best encryption results. Tests were conducted by comparing various parameters on the three proposed 1D chaos systems, including correlation coefficient, histogram analysis, entropy value, number of pixel rate changes (NPCR), and unified average change intensity (UACI). Based on testing the image encryption results, the diffusion process utilizing the tent map produced the best image encryption compared to other chaotic systems.

THE REAL TIME HIGH PRECISION CRIMINAL FACE RECOGNITION FOR ATM SECURITY IN DEEP LEARNING

The proposed ATM security system, utilizing composable deep face recognition, emphasizes user protection during transactions with a specific focus on eye retina accuracy. In the event of a potential threat identified through real-time face detection and tracking, an automatic alert is initiated. Prioritizing precision, the system employs advanced face detection algorithms for accurate location and tracking, even in dynamic scenarios. A down sampling technique ensures efficient processing of facial data, crucial for meticulous identification in crowded ATM locations. The integration of a face tracking ID unit enhances accuracy, providing continuous monitoring and persistence in identification, particularly focusing on the distinctive patterns in the eye retina. The scoring method, rooted in face tracking and embedding distance, significantly elevates user identification reliability, minimizing the risk of false positives, especially in criminal identification based on unique eye retina features. In response to potential threats, the system promptly generates automatic alerts and dispatches secure emails to relevant entities, including the affected card user, the respective bank, and cybersecurity authorities. These emails include comprehensive information, such as a high-precision image of the detected criminal's face, detailed facial recognition data, and a thorough summary of their known criminal history. This accuracy-driven and retina focused approach enhances safety, emphasizing continuous refinement and adherence to privacy and ethical considerations during implementation. KEYWORDS: Crime prevention, down-sampling, face recognition.

EYE-BLINK DETECTION ASSISTIVE SYSTEM FOR PARALYZED PATIENT

Paralysis is defined as the complete loss of muscle function in any part of the body. It occurs when there is a problem with the passage of messages between the muscles and the brain. Some paralyzed people cannot move even a single part of the body other than their eyes. Hence, the main aim of this project is to design a real time interactive system that can assist the paralyzed to control appliances such as lights, fans or by playing pre-recorded audio messages, through a predefined number of eye blinks. Image processing techniques have been implemented in order to detect the eye blinks. In our system, the face tracking is accomplished by using a set of trained Haar cascade classifier, and a template matching technique is employed to track the eye. Initially, the involuntary blinks of the paralyzed person are used to locate the patients eyes by finding the number of connected components in a frame.

Action unit intensity regression for facial MoCap aimed towards digital humans

AbstractDue to the increasing demand for virtual avatars, there has been a recent growth in the research and development of frameworks for realistic digital humans, which create a demand for realistic and adaptable facial motion capture systems. Most frameworks belong to private companies or represent high investments, which is why the creation of democratized solutions is relevant for the growth of digital human content creation. This research work proposes a facial motion capture framework for digital humans with the use of machine learning for facial codification intensity regression. The main focus is to use coded face movement intensities to generate realistic expressions on a digital human. The ablation studies performed on the regression models show that Neural Networks, using Histogram of Oriented Gradients as features, and with person-specific normalization, present overall better performance against other methods in the literature. With an RMSE of 0.052, the proposed framework offers reliable results that can be rendered in the face of a MetaHuman.

P‐2.3: Development Prospects and Current Status of Deep Learning Neural Network‐based Facial Capture in the Metaverse Field

P‐2.3: Development Prospects and Current Status of Deep Learning Neural Network‐based Facial Capture in the Metaverse Field

A survey on drowsiness detection system with advanced face tracking

To address the increasing dangers associated with driver and worker fatigue, this project introduces an advanced Drowsiness Detection System featuring state-of-the-art face-tracking capabilities. The pressing need for fatigue detection is evident in the alarming figures of 800 annual fatalities and 50,000 injuries resulting from drowsy driving incidents. This research expands the application of the technology to industrial workplaces, where the consequences of drowsiness are equally severe. Our comprehensive approach involves real-time monitoring of facial features, with a focus on eye movements and eyelid patterns. This system goes beyond traditional boundaries, covering drivers and industrial workers operating heavy machinery. By incorporating facial landmarks and introducing the innovative Eyes Aspect Ratio parameter, our technology offers a precise assessment of weariness in individuals within the current frame. This approach enhances safety measures in smart transportation systems and industrial settings. Integrating facial landmarks allows for a nuanced understanding of fatigue, recognizing subtle changes in facial expressions and movements indicative of drowsiness. The Eyes Aspect Ratio parameter, a novel addition, improves weariness assessment precision by considering factors such as eye closure duration, blink frequency, and gaze direction. These outcomes signify a significant contribution to road safety and broader workplace security. By mitigating inherent risks associated with drowsiness among industrial workers, our technology aims to reduce accidents, prevent injuries, and save lives across various occupational settings. The potential impact extends beyond individuals, influencing organizational safety protocols and contributing to the overall well-being of workers in high-risk environments.

Real-Time Scalable System For Face Tracking In Multi-Camera

Face detection and tracking have become increasingly popular in recent years. It has critical importance in security, defense, and robotics applications uses encountered in everyday life. For this purpose, many decision support or expert systems have been developed using artificial intelligence and machine learning. Thanks to the developments in the field of deep learning and hardware many effective and reliable face tracking systems have been realized. However there are still very few real-time scalable end-to-end systems. Also, the realization of this system on multiple cameras is a real challenge. In this study, a real-time, multi-camera, deep learning-based face tracking system has been developed. In the realized system, SCRFD model is used for face detection, ArcFace model is used for face recognition, and an updated DeepSORT algorithm is used for more stable face tracking. In addition, Apache Kafka stream processing system and Socket.IO bidirectional communication library were used to process multi-camera data in real-time and scalable. In the proposed system, when an image is input into the system, it can be displayed on the web page after approximately 127 ms

Human Skin Detection and Segmentation Based on Convolutional Neural Networks

Human skin detection is the process of classifying the image pixels or regions as skin or non-skin. Skin detection has many applications, such as face tracking, skin diseases, nudity recognition, hand gestures, video surveillance, web content filtering, and pornographic content filtering. Skin detection is a challenging problem due to skin color variation, the human race, aging, gender, makeup, complex backgrounds, etc. This paper suggests detecting the skin region in the image and finding the location of the skin based on a convolutional neural network. In this proposal, the proposed CNN will be modified by adding two layers before the first layer of CNN and after the last layer of CNN. The main purpose of these layers is to prepare the input image by using a sliding window, which inputs an indexed small part of the image into the CNN. The network classifies each part as skin or non-skin and then sends the result into the second suggested layer. After processing all the image pixels, the non-skin blocks are mapped to the original image as black regions. The final image contains the skin regions with black in the background. The contribution of the proposed method is the ability to detect the skin from any part of the human body, unlike previous works, which focused on one part of the body. The input image is processed as blocks instead of the entire image as in the previous works, and then in the output, the original image is reconstructed. This method works well with most of the challenges that face the detection of skin, and finally, the designed network facilitates the localization and segmentation of the skin region almost accurately, while the previous networks focused on the classification of the image as skin or non-skin. The accuracy of the detection of skin when testing with images different from the training images was 95.4%.

Effectiveness of IMPUTE ADT-1 mobile application in children with autism spectrum disorder: An interim analysis of an ongoing randomized controlled trial.

IMPUTE Inc., a software firm dedicated to healthcare technology, has developed a mobile medical application known as IMPUTE ADT-1 for children with autism spectrum disorder (ASD) based on the principle of applied behavior analysis. The primary objective of this trial was to compare the efficacy of add-on treatment with IMPUTE ADT-1 in children with ASD aged two to six years as compared to standard care alone for 12 weeks (in terms of change in Autism Diagnostic Observation Schedule [ADOS-2] scores). The secondary objective of the study was to assess the compliance with IMPUTE ADT-1 among participants and also to evaluate the feedback of parents regarding IMPUTE ADT-1 at the end of 12 weeks. The application provides personalized programs tailored to each user's needs, and the program evolves based on the user's progress. It also utilizes face tracking, eye tracking, and body tracking to gather behavior-related information for each child and apply it in reinforcement learning employing artificial intelligence-based algorithms. Till the time of interim analysis, 37 and 33 children had completed 12-week follow-up in IMPUTE ADT-1 and control arm. At 12 weeks, as compared to baseline, change in social affect domain, repetitive ritualistic behavior domain, total ADOS-2 score, and ADOS-2 comparison score was better in the intervention group as compared to the control group (P < 0.001 for all). A total of 30 (81%), 28 (75%), and 29 (78%) caregivers in the IMPUTE ADT-1 group believed that the ADT-1 app improved their child's verbal skills, social skills, and reduced repetitive behavior, respectively. IMPUTE ADT-1 mobile application has the efficacy to improve the severity of autism symptoms in children. Parents of these children also feel that the application is beneficial for improving the socialization and verbal communication of their children.

Robust, Resilient Enhanced CAMSHIFT Model: Advancing Face Detection and Tracking Stability in Challenging Environments

This paper presents the development of a robust CAMSHIFT model for theoretical face detection and tracking. The proposed model integrates innovative techniques such as Perceptual Grouping, three Connected Component Operators, Weighted Adaptive Colour Histogram, and Selective Adaptation. Experimental results highlight its superior performance across scenarios like occlusions, varying illumination, near/far face tracking, skin-like background tracking, and disturbance from multiple faces. The normalized log-likelihood index serves as a robust indicator for face tracking analysis. Connected Component operations provide strong markers for error detection in video sequences. The enhanced CAMSHIFT algorithm exhibits resilience and stability, even in the presence of occlusions. Comparisons with the original CAMSHIFT reveal the enhanced model's superiority, extending tracking range to 500 cm, a calculated enhancement of 42.9 percent improvement. The study consistently favours the robust and resilient CAMSHIFT model in tracking against skin-like backgrounds and disturbances. Despite webcam convenience in used for algorithm development, the benefits of high-performance camera systems are envisioned for future research. This model is a significant advancement in face detection methods, promising improved adaptability and tracking capabilities.

Audio-Visual Bimodal Combination-Based Speaker Tracking Method for Mobile Robot

Initiative service is a key research direction for the new generation of service robots. It is important to automatically track humans for initiative service in human-robot interaction. To solve the problems of low precision and poor anti-interference capability of only using single-modal (audio or visual) information, a speaker positioning and tracking method based on an audio-visual bimodal combination is proposed. First, the azimuth of the speaker is obtained based on the time difference of arrival using a microphone array, and face detection based on AdaBoost is carried out using the camera. A distance and azimuth calculation model is established to obtain the position of the speaker. Second, a speaker positioning strategy based on an audio-visual bimodal combination is designed to handle different situations. Third, the path is planned by which the azimuth and distance between the robot and the speaker are maintained in a limited range. Different azimuths and distances for speaker tracking are set to perform various simulations. Finally, the mobile robot is driven to follow the path using the STM32 real-time control system. Information from the microphone array and the camera is collected and processed by Raspberry Pi. The tracking accuracy was tested under a single-face situation by setting 20 different target points, and 10 tests were carried out under each point. Under multi-face situations, the audio-visual bimodal information is combined to identify the speaker, and then the Kalman filter is used in face tracking. The experimental results demonstrate that the running trajectory of the mobile robot is close to the ideal trajectory, which ensures effective speaker tracking.

Correlation of 3D Morphometric Changes, Kinematics, and Muscle Activity During Smile.

Knowing the morphological, kinematic, and electrophysiological parameters of the smile in healthy individuals may contribute to evaluating, planning, and monitoring the smile reanimation. This study aimed to determine the correlation between 3D morphometric changes, movement kinematics, and muscle activity in the facial soft tissue of healthy individuals. In this cohort study, 20 volunteers were selected from healthy individuals with no facial disorders. During smiling, three-dimensional face scanning, facial motion capture, and surface electromyography (sEMG) were performed. The average displacement, velocity, and acceleration during facial movements were measured. The mean change in 3D surface morphometry and activation of the zygomaticus major were determined. The volunteers, comprising 10 males and 10 females, had a mean age of 24 ± 10 years; for female, mean age was 23 ± 5 years and for men 26 ± 13 years. Significant correlations were found between kinematic and morphometric data (r = 0.51, p < 0.001), sEMG and morphometric (r = 0.50, p < 0.001) data, and sEMG and kinematic data (r = 0.49, p < 0.002). The maximum acceleration occurred during approximately 65% of the muscle activation time and 64% of the peak muscle activation value. Additionally, the maximum velocity was reached at around 73% of the muscle activation time and 67% of the peak muscle activation value. Furthermore, the maximum displacement values were observed at approximately 88% of the muscle activation time and 76% of the peak muscle activation value. The findings may provide insights into the smile's functional parameters, contribute to understanding facial muscle-related disorders, and aid in improving the diagnosis and treatment of the smile. NA Laryngoscope, 134:3112-3119, 2024.

Efficient Pooling Operator for 3D Morphable Models.

Learning the latent representation of three-dimensional (3D) morphable geometry is useful for several tasks, such as 3D face tracking, human motion analysis, and character generation and animation. For unstructured surface meshes, previous state-of-the-art methods focus on designing convolution operators and share the same pooling and unpooling operations to encode neighborhood information. Previous models use a mesh pooling operation based on edge contraction, which is based on the euclidean distance of vertices rather than the actual topology. In this study, we investigated whether such a pooling operation can be improved, introducing an improved pooling layer that combines the vertex normals and adjacent faces area. Furthermore, to prevent template overfitting, we increased the receptive field and improved low-resolution projection in the unpooling stage. This increase did not affect processing efficiency because the operation was implemented once on the mesh. We performed experiments to evaluate the proposed method, whose results indicated that the proposed operations outperformed Neural3DMM with 14% lower reconstruction errors and outperformed CoMA by 15% by modifying the pooling and unpooling matrices.

Using IoT to Smart Security Door System with Face Tracking Camera

Using IoT to Smart Security Door System with Face Tracking Camera