Faces In Video Sequences Research Articles

Framework for multiple person identification using YOLOv8 detector: a transfer learning approach

Video surveillance extensively uses person detection and tracking technology based on video. The majority of person detection and classification techniques currently in use encounter challenges in video sequences brought on by occlusion, ambient lighting, and variations in human facial position. This paper proposed an effective person identification and classification system based on deep learning, which comprises a you only look once at version 8 (YOLOv8) detection and classification model, to classify human faces in video sequences accurately. This work proposes a new staff-detection and classification (S-DEC) dataset for comprehensive performance evaluation. visual tracker benchmark (VTB) standard database is used for performance comparison with the proposed S-DEC dataset. The proposed technique achieved 98.67% precision accuracy. For the S-DEC dataset, the system gave 94.67% accuracy in identifying facial images from a video sequence of 38 people addressing the pose variation occlusion challenge. Earlier methods used to provide approximately 85% to 90% results taking more execution time. Many existing techniques were successful in detecting people only-identification of the detected person has been done in limited papers. The proposed method uses the cross-stage partial connections (CSPDarknet53) model, integrated with YOLOv8, to achieve faster results. The proposed framework took 35 minutes to train a deep learning model. A testing time of 2 minutes ensured that the proposed framework outplayed other existing methodologies and successfully identified extra information about the detected person.

Wavelet Transform-based Methods for Forensic Analysis of Digital Images

In recent years, Generative Adversarial Networks (GANs) have been utilized in many applications of our daily lives to create digital media that was previously impossible. This paper utilizes GANs in multimedia forensics, where the precision and accuracy of image classification are crucial. The objective of the study is to detect sophisticated manipulated images generated by advanced deep learning methods using classical transformation methods. The method combines the classical wavelet transform for feature extraction with a classifier model to distinguish between real and fake images. The proposed method extracts unique features that differentiate between fake and real images, which are then fed to several gradient-boosting-based classifiers. The proposed method was tested using the FaceForensics++ dataset, which contains low-resolution video sequences of faces. The proposed model achieves an accuracy of 95% in detecting manipulated videos compared with 87% accuracy of other classical techniques.

Non-contact Heart Rate Measurement from Face Video Sequences using SIFT and FastICA.

Non-contact Heart Rate Measurement from Face Video Sequences using SIFT and FastICA.

Non-contact heart rate measurement from face video sequences using SIFT and FastICA

Non-contact heart rate measurement from face video sequences using SIFT and FastICA

A Real-Time Eye Tracking System for the Detection of Eye Blinks.

This paper proposes an eye blink detection system that automatically detects eye blinks, which can be an indicator of fatigue or cognitive load, among others. As a key feature, the real-time capability of the system is being required to use it, for example, as a monitoring system for people in potentially critical situations (e.g., drivers or operators of heavy machinery). The system uses the Viola-Jones algorithm for face detection and the median flow tracker to track the face in video sequences. Eye detection is implemented using face proportions, and template matching is used for blink detection. The resulting system processes 40-47 frames per second on default consumer hardware and achieves an accuracy of 80.33% and a precision of 85.22% in the evaluation. The proposed system shows promising results under ideal viewing conditions but has difficulty maintaining high precision during head movements. The proposed system could be integrated with various health-related assistance systems to monitor the individual's well-being in real time, as long as their head is observed from the front if possible.

The effect of masks on the recognition of facial expressions: A true-to-life study on the perception of basic emotions.

Mouth-to-nose face masks became ubiquitous due to the COVID-19 pandemic. This ignited studies on the perception of emotions in masked faces. Most of these studies presented still images of an emotional face with a face mask digitally superimposed upon the nose-mouth region. A common finding of these studies is that smiles become less perceivable. The present study investigated the recognition of basic emotions in video sequences of faces. We replicated much of the evidence gathered from presenting still images with digitally superimposed masks. We also unearthed fundamental differences in comparison to existing studies with regard to the perception of smile which is less impeded than previous studies implied.

Automatic Detection of Face in Video Sequences by using Extended Semi Local Binary Patterns

Machine analysis of detection of the face is an active research topic in Human-Computer Interaction today. Most of the existing studies show that discovering the portion and scale of the face region is difficult due to significant illumination variation, noise and appearance variation in unconstrained scenarios. To overcome these problems, we present a method based on Extended Semi-Local Binary Patterns. For each frame, an aggregation of the pixel values over a neighborhood is considered and a local binary pattern is obtained. From these a binary code is obtained for each pixel and then histogram features is computed. Adaboost algorithm is used to learn and classify these discriminative features with the help of exemplar face and non-face signature of the images for detecting the location of face region in the frame. This Extended Semi Local Binary Pattern is sturdy to variations in illumination and noisy images. The developed methods are deployed on the real time YouTube video face databases and found to exhibit significant performance improvement owing to the novel features when compared to the existing techniques.

It is in your face-Alexithymia impairs facial mimicry.

Alexithymia is characterized by a reduced ability to identify and differentiate emotional aspects of social interaction. In this study we investigated, for the first time, whether alexithymia impairs facial mimicry in response to dynamic naturalistic facial affect expressions. Potential volunteers were recruited by means of an online survey (N = 3503). Based on their Toronto Alexithymia Scale-20 sum-score (TAS-20) probands were assigned to an alexithymic group (AG; M = 58.11, SD = 4.58) or a nonalexithymic healthy control group (HC; M = 32.05, SD = 5.56). Both groups were matched by age, gender, and education. All probands were shown digitally generated naturalistic video sequences of faces displaying the basic affects of fear, sadness, disgust, anger, and joy. During the presentation, the participants' facial mimicry responses were recorded by registering the electromyographic (EMG) activity of the corrugator supercilii and zygomaticus major muscles. Overall, the alexithymic probands showed a significantly lower facial EMG activity in response to the affective faces compared to HC. The results thus suggest that alexithymia is associated with a reduced facial mimicry. We discuss the implications of these findings from the perspective of alexithymic impairments within social interaction and the consideration for psychotherapeutic treatment. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Dynamic texture analysis for detecting fake faces in video sequences

The creation of manipulated multimedia content involving human characters has reached in the last years unprecedented realism, calling for automated techniques to expose synthetically generated faces in images and videos.This work explores the analysis of spatio-temporal texture dynamics of the video signal, with the goal of characterizing and distinguishing real and fake sequences. We propose to build a binary decision on the joint analysis of multiple temporal segments and, in contrast to previous approaches, to exploit the textural dynamics of both the spatial and temporal dimensions. This is achieved through the use of Local Derivative Patterns on Three Orthogonal Planes (LDP-TOP), a compact feature representation known to be an important asset for the detection of face spoofing attacks.Experimental analyses on state-of-the-art datasets of manipulated videos show the discriminative power of such descriptors in separating real and fake sequences, and also identifying the creation method used. Linear Support Vector Machines (SVMs) are used which, despite the lower complexity, yield comparable performance to previously proposed deep models for fake content detection.

EYE RECOGNITION SYSTEM TO PREVENT ACCIDENTS ON THE ROAD

Abstract. Today, possibilities of artificial intelligence allow us to see the emergence of autonomous cars. However, there are still many problems in this area at present. Often, such vehicles are “too slow to think”, are not able to reliably process data from video cameras in the event of reflections, glare, and there are also questions about the safety of such driving in difficult weather conditions or in heavy traffic. At the same time, the human factor plays a major role in accidents of driven vehicles. Many accidents involve driver fatigue, distraction, or even falling asleep. At the same time, it is potentially possible to monitor the state of a person behind the wheel by a video sequence received from a camera installed in the car's interior and registering the driver's face in video sequence. In this paper, the existing databases of images of faces and eyes are considered, and an algorithm is presented that detects the state of closed eyes based on Haar detectors and convolutional neural networks.

Efficient face detection and tracking in video sequences based on deep learning

Video-based face detection and tracking technology has been widely used in video surveillance, safe driving, and medical diagnosis. In video sequences, most existing face detection and tracking methods face interference caused by occlusion, ambient illumination, and changes in human posture. To accurately track human faces in video sequences, we propose an efficient face detection and tracking framework based on deep learning, which includes a SENResNet face detection model and a Regression Network-based Face Tracking (RNFT) model. Firstly, the SENResNet model integrates the Squeeze and Excitation Network (SEN) with the Residual Neural Network (ResNet). To solve the problem that deep neural networks are difficult to train, we use ResNet to overcome the problem of gradient disappearance in deep network training. To fuse the features of each channel during the convolution operation, we further integrate the SEN module into the SENResNet model. SENResNet accurately detects facial information in each frame and extracts the position of the target face, thereby providing an initialization window for face tracking. Then, the RNFT model extracts facial features from adjacent frames and predict the position of the target face in the next frame. To address the problem of feature scaling, we add a correction network to the RNFT model. The improved RNFT model extracts the rectangular frame of the target face in the previous frame and strengthens the perception of feature scaling, thereby improving its accuracy. Extensive experimental results on public facial and video datasets show that the proposed SENResNet and RNFT models are superior to the state-of-the-art comparison methods in terms of accuracy and performance.

Learning How to Smile: Expression Video Generation With Conditional Adversarial Recurrent Nets

While several research studies have focused on analyzing human behavior and, in particular, emotional signals from visual data, the problem of synthesizing face video sequences with specific attributes (e.g. age, facial expressions) received much less attention. This paper proposes a novel deep generative model able to produce face videos from a given image of a neutral face and a label indicating a specific facial expression, e.g. spontaneous smile. Our framework consists of two main building blocks: an image generator and a frame sequence generator. The image generator is implemented as a deep neural model which combines generative adversarial networks and variational auto-encoders, while the sequence generator is a label-conditioned recurrent neural network. In the proposed framework, given as input a neural face and a label, the sequence generator outputs a set of hidden representations with smooth transitions corresponding to video frames. Then, the image generator is used to decode the hidden representations into the actual face images. To impose that the net generates videos consistent with the given label, a novel identity adversarial loss is proposed. Our experimental results demonstrate the effectiveness of the framework and the advantage of introducing an adversarial component into recurrent models for face video generation.

Pain Facial Expression Recognition from Video Sequences Using Spatio-temporal Local Binary Patterns and Tracking Fiducial Points

Monitoring the facial expressions of patients in clinical environments is a necessity in addition to vital sign monitoring. Pain monitoring of patients by facial expressions from video sequences eliminates the need for another person to accompany patients. In this paper, a novel approach is presented to monitor the expression of face and notify in case of pain using tracking fiducial points of face in video sequences and spatio-temporal Local Binary Patterns (LBPs) for eyes and eyebrows. The motion of eight fiducial points on facial features such as mouth, eyes, eyebrows are tracked by Lucas-Kanade algorithm and the movement angles are recorded in a feature vector which along with the spatio-temporal histogram of LBPs creates a concatenated feature vector. Spatio-temporal LBPs boost the proposed algorithm to capture minor deformations on eyes and eyebrows. The feature vectors are then compared and classified using the Chi-square similarity measure. Experimental results show that leveraging spatio-temporal LBPs improves the accuracy by 12% on STOIC database.

Detecting “DeepFakes” in H.264 Video Data Using Compression Ghost Artifacts

In recent years, the number of forged videos circulating on the Internet has immensely increased. Software and services to create such forgeries have become more and more accessible to the public. In this regard, the risk of malicious use of forged videos has risen. This work proposes an approach based on the Ghost effect knwon from image forensics for detecting forgeries in videos that can replace faces in video sequences or change the mimic of a face. The experimental results show that the proposed approach is able to identify forgery in high-quality encoded video content.

Multiple Faces Tracking Using Feature Fusion and Neural Network in Video

Face tracking is one of the most challenging research topics in computer vision. This paper proposes a framework to track multiple faces in video sequences automatically and presents an improved method based on feature fusion and neural network for multiple faces tracking in a video. The proposed method mainly includes three steps. At first, it is face detection, where an existing method is used to detect the faces in the first frame. Second, faces tracking with feature fusion. Given a video that has multiple faces, at first, all faces in the first frame are detected correctly by using an existing method. Then the wavelet packet transform coefficients and color features from the detected faces are extracted. Furthermore, we design a backpropagation (BP) neural network for tracking the occasional faces. At last, a particle filter is used to track the faces. The main contributions are. Firstly, to improve face tracking accuracy, the Wavelet Packet Transform coefficients combined with traditional color features are utilized in the proposed method. It efficiently describes faces due to their discrimination and simplicity. Secondly, to solve the problem in occasional face tracking, and improved tracking method for robust occlusion tracking based on the BP neural network (PFT_WPT_BP) is proposed. Experimental results have been shown that our PFT_WPT_BP method can handle the occlusion effectively and achieve better performance over other methods.

Detection of Human Faces in Video Sequences using NGLBP Feature

Machine analysis of face detection is an interesting topic for study in Human-Computer Interaction. The existing studies show that discovering the position and scale of the face region is difficult due to significant illumination variation, noise and appearance variation in unconstrained scenarios. This paper suggests a method to detect the location of face area using recently developed YouTube Video face database. In this work, each frame is formulated by normalization technique and separated into overlapping blocks. The Gabor filter is tuned to extract the Gabor features from the individual blocks. The averaged Gabor features are manipulated and local binary pattern histogram features are extracted. The extracted patterns are passed to the classifier with training images for face region identification. Our experimental results on YouTube video face database exhibits promising results and demonstrate a significant performance improvement when compared to the existing techniques. Furthermore, our proposed work is uncaring to head poses and sturdy to variations in illumination, appearance and noisy images

A Comprehensive Experimental and Reproducible Study on Selfie Biometrics in Multistream and Heterogeneous Settings

This contribution presents a new database to address current challenges in face recognition. It contains face video sequences of 75 individuals acquired either through a laptop webcam or when mimicking the front-facing camera of a smartphone. Sequences have been acquired with a device allowing to record visual, near-infrared, and depth data at the same time. Recordings have been made across three sessions with different, challenging illumination conditions and variations in pose. Together with the database, several experimental protocols are provided and correspond to real world scenarios, when a mismatch in conditions between enrollment and probe images occurs. A comprehensive set of baseline experiments using publicly available baseline algorithms show that extreme illumination conditions and pose variations are remaining issues. However, the usage of different data domains—and their fusion—allows to mitigate such variation. Finally, experiments on heterogeneous face recognition are also presented using a state-of-the-art model based on deep neural networks, and showed better performance. When applied to other tasks, this model proved to surpass all existing baselines as well. The data, as well as the code to reproduce all experiments are made publicly available to help foster research in selfie biometrics using latest imaging devices.

Facial-expression recognition based on a low-dimensional temporal feature space

This paper suggests a facial-expression recognition in accordance with face video sequences based on a newly low-dimensional feature space proposed. Indeed, we extract a Pyramid of uniform Temporal Local Binary Pattern representation, using only XT and YT orthogonal planes (PTLBP u2). Then, a Wrapper method is applied to select the most discriminating sub-regions, and therefore, reduce the feature space that is going to be projected on a low-dimensional feature space by applying the Principal Component Analysis (PCA). Support Vector Machine (SVM) and C4.5 algorithm have been tested for the classification of facial expressions. Experiments conducted on CK + and MMI, which are the two famous facial-expression databases, have shown the effectiveness of the approach proposed under a lab-controlled environment with more than 97% of recognition rate as well as under an uncontrolled environment with more than 92%.

Recognition of peer emotions in children with ADHD: Evidence from an animated facial expressions task

A growing body of literature suggests that ADHD is associated with emotion recognition impairments that may be linked to deficient interpersonal functioning. However, our understanding of the mechanisms underlying these recognition impairments is extremely limited. Here, we used dynamic stimuli to investigate whether impaired emotion recognition in children with ADHD may be associated with impairments in perceptual sensitivity. Participants (ADHD: N = 26; Controls: N = 26) viewed video sequences of neutral faces slowly developing into one of the six basic emotional expressions (angry, happy, fearful, sad, disgusted and surprised) and were instructed to indicate via a button press the precise moment at which they were able to correctly recognize the emotional expression. The results showed that compared to controls, children with ADHD exhibited lower accuracy rates across all emotional expressions while there was no evidence for impaired perceptual sensitivity. Thus, the study provides evidence for a generalized categorization impairment across all emotional categories and is consistent with developmental delay accounts of ADHD. Future studies are needed in order to further investigate the developmental course of social cognition deficits in ADHD.

Analysis of the parameters of respiration patterns extracted from thermal image sequences

Remote estimation of vital signs is an important and active area of research. The goal of this work was to analyze the feasibility of estimating respiration parameters from video sequences of faces recorded using a mobile thermal camera. Different estimators were analyzed and experimentally verified. It was demonstrated that the respiration rate, periodicity of respiration, and presence and length of apnea periods could be reliably estimated from signals recorded using a portable thermal camera. The size of the camera and efficiency of the methods allow the implementation of this method in smart glasses.