Facial Feature Tracking Research Articles

What not to do in facial infrared thermographic measurements: A post data enhancement

The accuracy of infrared thermographic measurements depends on several factors, including movement of target. In this study, accuracy of nose tip temperatures obtained in a mental workload assessment using a thermal imaging camera were impacted by participants’ movement and camera zooming/panning. To correct these measurement errors, we compared manual facial landmark identification techniques using data labelling software with an automated deep learning-based approach utilised for facial landmark tracking and evaluated both against the built-in tracking features of the thermal camera, Thermal Spot Tracking. Using the Manual Thermal Landmark Annotation measurements as the ground truth, our results show that the Automated Facial Feature Tracking approach, which is the AI based approach performed better than the Thermal Spot Tracking as it matched comparatively more spatial coordinates and temperature datapoints as well as showed comparatively lower mean relative error. The study highlights the potential of AI in enhancing the accuracy of thermographic measurements, particularly in applications involving facial temperature analysis.

Identification and Tracking of Facial Features for Attendance System

Identification and Tracking of Facial Features for Attendance System

Correction: A Lightweight Neural Learning Algorithm for Real-Time Facial Feature Tracking System via Split-Attention and Heterogeneous Convolution

Correction: A Lightweight Neural Learning Algorithm for Real-Time Facial Feature Tracking System via Split-Attention and Heterogeneous Convolution

A Lightweight Neural Learning Algorithm for Real-Time Facial Feature Tracking System via Split-Attention and Heterogeneous Convolution

A Lightweight Neural Learning Algorithm for Real-Time Facial Feature Tracking System via Split-Attention and Heterogeneous Convolution

Face feature tracking algorithm of aerobics athletes based on Kalman filter and mean shift

In order to solve the problems of low accuracy and long time-consuming in face image tracking of aerobics athletes in traditional methods, a face feature tracking algorithm based on Kalman filter and mean shift is proposed. Three-frame difference method is used to extract the colour features of aerobics athletes' face images, measure the geometric feature similarity of aerobics athletes' face images, calculate the grey value of local images of aerobics athletes' face features, and match corner features by NCC matching algorithm. The Kalman filter method is introduced to denoise the different pixels of the feature image, and the mean shift of the aerobics athletes' face features is obtained by means of the mean shift algorithm to realise the tracking of the aerobics athletes' face features. The experimental results show that the tracking accuracy of the proposed method is up to 97%, and the shortest tracking time is about 1.5 s.

Study of DRMF and ASM facial landmark point for micro expression recognition using KLT tracking point feature

Micro-expression recognition is one of the popular researches in analysing expressions on the face. Micro-expression is a facial movement that occurs in a short time and is difficult to identify manually by human eyes. In general research, facial landmarks are used to form a large size ROI for each facial feature for the feature extraction process. In this study, we track the subtle motions of micro expressions by using point features. This approach aims to get feature extraction from tracking results and then analyse micro-expression. We compared the Active Shape Model and Response Map Fitting methods to produce accurate points and fast time on facial features. To measure the subtle motion tracking of facial features in each frame tracking is done using the Kanade-Lucas-Tomasi method. To estimate the rationality of our method, we conducted an experiment on CASME II and SAMM dataset for micro-expressions. The results show that the points on DRMF are more accurate with point-to-point error is 7.9 and the time taken is faster which requires time is 0.02 second. We evaluated the method proposed for evaluation showed that using CASME II - Naive Bayes (79.3%) and SAMM - Naive Bayes (74.6%).

Object acquisition and selection using automatic scanning and eye blinks in an HCI system

This paper presents an object acquisition and selection approach in human computer interaction systems. In this approach, objects placed over computer screen are automatically scanned and the user performs voluntary eye blinks for object selection when the focus comes over the object of interest. Here, scanning means moving the focus over objects placed on the computer screen one by one and the scanning time is the time taken to move focus from one object to the next object. The user is not required to perform any physical movement, the moving part is only the eye lids. A low cost webcam and MATLAB software with computer vision toolbox are required to implement the proposed approach. The performance of the proposed approach has been compared with the Camera Mouse for selection of text and graphic objects. The Camera Mouse utilizes facial feature tracking for mouse cursor control and dwell time for object selection. Three experiments were performed for evaluation of the proposed method in which ten healthy users voluntarily participated. The proposed method has given significantly better performance than the Camera Mouse when selection of text objects was performed in an html file. For selection of graphic objects placed on computer screen, where page scrolling is not required, no significant difference has been found in the performance of both the systems. The proposed method has also been evaluated for performing mouse analogous operations using eye blinks and a performance comparison has been made with state-of-the-art methods.

PERFORMANCE COMPARISON OF OBJECT ACQUISITION USING AUTOMATIC SCANNING AND FACIAL FEATURE TRACKING

This paper presents a comparison of object acquisition in an HCI system using two different techniques: automatic scanning and facial feature tracking. In automatic scanning the focus moves from one object to next object automatically after a predefined time period called scanning time. Automatic scanning has been implemented by using MATLAB algorithm which virtually activates the tab key after each scanning time and the focus moves from object to object. The user activates a selection trigger for selection of the object when the focus comes over the object of interest. Whereas, in facial feature tracking approach the mouse cursor is moved in proportion to the movement of user’s face. To implement this technique Camera Mouse has been used which requires a simple webcam. It continuously takes facial images of the user and finds the mouse cursor position from the face coordinates. The two techniques are compared based upon accuracy and acquisition time for acquisition of text and graphic objects.

Dynamic facial landmarking selection for emotion recognition using Gaussian processes

Facial features are the basis for the emotion recognition process and are widely used in affective computing systems. This emotional process is produced by a dynamic change in the physiological signals and the visual answers related to the facial expressions. An important factor in this process, relies on the shape information of a facial expression, represented as dynamically changing facial landmarks. In this paper we present a framework for dynamic facial landmarking selection based on facial expression analysis using Gaussian Processes. We perform facial features tracking, based on Active Appearance Models for facial landmarking detection, and then use Gaussian process ranking over the dynamic emotional sequences with the aim to establish which landmarks are more relevant for emotional multivariate time-series recognition. The experimental results show that Gaussian Processes can effectively fit to an emotional time-series and the ranking process with log-likelihoods finds the best landmarks (mouth and eyebrows regions) that represent a given facial expression sequence. Finally, we use the best ranked landmarks in emotion recognition tasks obtaining accurate performances for acted and spontaneous scenarios of emotional datasets.

Facial feature tracking: a psychophysiological measure to assess exercise intensity?

ABSTRACTThe primary aim of this study was to determine whether facial feature tracking reliably measures changes in facial movement across varying exercise intensities. Fifteen cyclists completed three, incremental intensity, cycling trials to exhaustion while their faces were recorded with video cameras. Facial feature tracking was found to be a moderately reliable measure of facial movement during incremental intensity cycling (intra-class correlation coefficient = 0.65–0.68). Facial movement (whole face (WF), upper face (UF), lower face (LF) and head movement (HM)) increased with exercise intensity, from lactate threshold one (LT1) until attainment of maximal aerobic power (MAP) (WF 3464 ± 3364mm, P < 0.005; UF 1961 ± 1779mm, P = 0.002; LF 1608 ± 1404mm, P = 0.002; HM 849 ± 642mm, P < 0.001). UF movement was greater than LF movement at all exercise intensities (UF minus LF at: LT1, 1048 ± 383mm; LT2, 1208 ± 611mm; MAP, 1401 ± 712mm; P < 0.001). Significant medium to large non-linear relationships were found between facial movement and power output (r2 = 0.24–0.31), HR (r2 = 0.26–0.33), [La−] (r2 = 0.33–0.44) and RPE (r2 = 0.38–0.45). The findings demonstrate the potential utility of facial feature tracking as a non-invasive, psychophysiological measure to potentially assess exercise intensity.

A robust incremental clustering-based facial feature tracking

Emerging significance of person-independent, emotion specific facial feature tracking has been actively tracked in the machine vision society for decades. Among distinct methods, the Constrained Local Model (CLM) has shown significant results in person-independent feature tracking. In this paper, we propose an automatic, efficient, and robust method for emotion specific facial feature detection and tracking from image sequences. A novel tracking system along with 17-point feature model on the frontal face region has also been proposed to facilitate the tracking of human basic facial expressions. The proposed feature tracking system keeps patch images and face shapes till certain number of key frames incorporating CLM-based tracker. After that, incremental patch and shape clustering algorithms is applied to build appearance model and structure model of similar patches and similar shapes respectively. The clusters in each model are built and updated incrementally and online, controlled by amount of facial muscle movement. The overall performance of the proposed Robust Incremental Clustering-based Facial Feature Tracking (RICFFT) is evaluated on the FGnet database and the Extended Cohn-Kanade (CK+) database. RICFFT demonstrates mean tracking accuracy of 97.45% and 96.64% for FGnet and CK+ database respectively. Also, RICFFT is more robust by minimizing average shape distortion error of 0.20% and 1.86% for FGnet and CK+ (apex frame) database, as compared with classic method CLM.

Incremental Clustering-Based Facial Feature Tracking Using Bayesian ART

Person-independent, emotion specific facial feature tracking have been of interest in the machine vision society for decades. Among various methods, the constrained local model (CLM) has shown significant results in person-independent feature tracking. In 63this paper, we propose an automatic, efficient, and robust method for emotion specific facial feature detection and tracking from image sequences. Considering a 17-point feature model on the frontal face region, the proposed tracking framework incorporates CLM with two incremental clustering algorithms to increase robustness and minimize tracking error during feature tracking. The Patch Clustering algorithm is applied to build an appearance model of face frames by organizing previously encountered similar patches into clusters while the shape Clustering algorithm is applied to build a structure model of face shapes by organizing previously encountered similar shapes into clusters followed by Bayesian adaptive resonance theory (ART). Both models are used to explore the similar features/shapes in the successive images. The clusters in each model are built and updated incrementally and online, controlled by amount of facial muscle movement. The overall performance of the proposed incremental clustering-based facial feature tracking (ICFFT) is evaluated using the FGnet database and the extended Cohn-Kanade (CK+) database. ICFFT demonstrates better results than baseline-method CLM and provides robust tracking as well as improved localization accuracy of emotion specific facial features tracking.

Human Facial Features Detection and Tracking in Images and Video

Human Facial Features Detection and Tracking in Images and Video

AAM Based Facial Feature Tracking with Kinect

Abstract Facial features tracking is widely used in face recognition, gesture, expression analysis, etc. AAM (Active Appearance Model) is one of the powerful methods for objects feature localization. Nevertheless, AAM still suffers from a few drawbacks, such as the view angle change problem. We present a method to solve it by using the depth data acquired from Kinect. We use the depth data to get the head pose information and RGB data to match the AAM result. We establish an approximate facial 3D gird model and then initialize the subsequent frames with this model and head pose information. To avoid the local extremum, we divide the model into several parts by the poses and match the facial features with the closest model. The experimental results show improvement of AAM performance when rotating the head.

New interface for musical instruments using lip reading

As smart audio-visual multimedia devices are developed for various applications, there has been a growing interest in effective human–computer interaction (HCI) interfaces for specific environments. There have also been great efforts to implement HCI interfaces into musical instruments, in which it would be possible to take intuitions, comfort and expressiveness into the musical instruments. However, most of the traditional HCI interfaces are not applicable because both hands are likely to be occupied while playing a musical instrument. In this environment, a lip reading method can be used. A lip reading method is a HCI method that analyses lip motion to recognise spoken words. In this study, a lip reading method is proposed with its application. As a specific example of the interface for musical instruments, the guitar effector application is presented. The proposed lip reading method uses a constrained local model instead of the conventional active appearance model for effective facial feature tracking. The proposed method also uses a dynamic time warping-based classifier for word recognition which is effective for simple real-time implementation of lip reading. The proposed lip reading method shows 85.0% word recognition accuracy on OuluVS database and is effectively applied to the proposed guitar effector application.

Driver Gaze Tracking and Eyes Off the Road Detection System

Distracted driving is one of the main causes of vehicle collisions in the United States. Passively monitoring a driver's activities constitutes the basis of an automobile safety system that can potentially reduce the number of accidents by estimating the driver's focus of attention. This paper proposes an inexpensive vision-based system to accurately detect Eyes Off the Road (EOR). The system has three main components: 1) robust facial feature tracking; 2) head pose and gaze estimation; and 3) 3-D geometric reasoning to detect EOR. From the video stream of a camera installed on the steering wheel column, our system tracks facial features from the driver's face. Using the tracked landmarks and a 3-D face model, the system computes head pose and gaze direction. The head pose estimation algorithm is robust to nonrigid face deformations due to changes in expressions. Finally, using a 3-D geometric analysis, the system reliably detects EOR.

IntraFace.

Within the last 20 years, there has been an increasing interest in the computer vision community in automated facial image analysis algorithms. This has been driven by applications in animation, market research, autonomous-driving, surveillance, and facial editing among others. To date, there exist several commercial packages for specific facial image analysis tasks such as facial expression recognition, facial attribute analysis or face tracking. However, free and easy-to-use software that incorporates all these functionalities is unavailable. This paper presents IntraFace (IF), a publicly-available software package for automated facial feature tracking, head pose estimation, facial attribute recognition, and facial expression analysis from video. In addition, IFincludes a newly develop technique for unsupervised synchrony detection to discover correlated facial behavior between two or more persons, a relatively unexplored problem in facial image analysis. In tests, IF achieved state-of-the-art results for emotion expression and action unit detection in three databases, FERA, CK+ and RU-FACS; measured audience reaction to a talk given by one of the authors; and discovered synchrony for smiling in videos of parent-infant interaction. IF is free of charge for academic use at http://www.humansensing.cs.cmu.edu/intraface/.

Facial Expression Recognition Using SVM Classifier

Facial feature tracking and facial actions recognition from image sequence attracted great attention in computer vision field. Computational facial expression analysis is a challenging research topic in computer vision. It is required by many applications such as human-computer interaction, computer graphic animation and automatic facial expression recognition. In recent years, plenty of computer vision techniques have been developed to track or recognize the facial activities in three levels. First, in the bottom level, facial feature tracking, which usually detects and tracks prominent landmarks surrounding facial components (i.e., mouth, eyebrow, etc), captures the detailed face shape information; Second, facial actions recognition, i.e., recognize facial action units (AUs) defined in FACS, try to recognize some meaningful facial activities (i.e., lid tightener, eyebrow raiser, etc); In the top level, facial expression analysis attempts to recognize some meaningful facial activities (i.e., lid tightener, eyebrow raiser, etc); In the top level, facial expression analysis attempts to recognize facial expressions that represent the human emotion states. In this proposed algorithm initially detecting eye and mouth, features of eye and mouth are extracted using Gabor filter, (Local Binary Pattern) LBP and PCA is used to reduce the dimensions of the features. Finally SVM is used to classification of expression and facial action units.

Identification and Tracking of Facial Features

Abstract Facial feature points such as eyebrows, eyes, nose and mouth are prominent landmarks surrounding facial components. They depicts critical information about facial expression and head movement .Hence accurate and perfect detection of facial features is important in several applications like face recognition,face tracking,criminal identification,security and surveillance systems,Human computer interaction(HCI), model based video coding. This paper represents traditional methodologies and taxonomies along with brief introduction to recent approaches and advancements in the field proposed by various researchers that acts as a bottleneck under various situations like occlusion, pose variations, different lighting conditions.

Hybrid Approach for Facial Feature Detection and Tracking under Occlusion

Hybrid Approach for Facial Feature Detection and Tracking under Occlusion