Facial Information Research Articles

Drowsiness Prediction Based on Multiple Aspects Using Image Processing Techniques: A Review

Abstract: Clinical depression is a type of soft biometric trait that can be used to characterize a person. Because of its importance in a variety of legal situations, this mood illness can be included in forensic psychological evaluations. In recent years, research into the automatic detection of depression based on real image has yielded a variety of algorithmic approaches and auditory indicators. Machine learning algorithms have recently been used successfully in a variety of image-based applications. Automatic depression recognition - the recognition of facial expressions linked with sad behaviour – is one of the most important applications. Modern algorithms for detecting depression usually look at both geographical and temporal data separately. This method restricts the capacity to capture a wide range of face expressions as well as the use of different facial parts. This research introduces a novel machine learning strategy for accurately representing face information associated to depressive behaviours from real-world images. Our suggested architecture outperforms state-of-the-art algorithms in automatic depression recognition, according to results from two benchmark datasets.

A survey on techniques, methods and security approaches in big data healthcare

A huge percentage of people especially in developed countries spend a good chunk of their wealth in managing their health conditions. In order to adequately administer healthcare, governments and various organizations have embraced advanced technology for automating the health industry. In recent past, electronic health records have largely been managed by Enterprise Resource Planning and legacy systems. Big data framework steadily emerge as the underlying technology in healthcare, which offers solutions that limits capacity of others systems in terms of storage and reporting. Automation through cloud services supported by storage of structured and unstructured health data in heterogeneous environment has improved service delivery, efficiency, medication, diagnosis, reporting and storage in healthcare. The argument support the idea that big data healthcare still face information security concern, for instance patient image sharing, authentication of patient, botnet, correlation attacks, man-in-the-middle, Distributed Denial of Service (DDoS), blockchain payment gateway, time complexities of algorithms, despite numerous studies conducted by scholars in security management for big data in smart healthcare. Some security technique include digital image encryption, steganography, biometrics, rule-based policy, prescriptive analysis, blockchain contact tracing, cloud security, MapReduce, machine-learning algorithms, anonymizations among others. However, most of these security solutions and analysis performed on structured and semi-structured data as opposed to unstructured data. This may affect the output of medical reporting of patients’ condition particularly on wearable devices and other examinations such as computerized tomography (CT) Scans among others. A major concern is how to identify inherent security vulnerabilities in big healthcare, which generate images for transmission and storage. Therefore, this paper conducted a comparative survey of solutions that specifically safeguards structured and unstructured data using systems that run on big data frameworks. The literature highlights several security advancements in cryptography, machine learning, anonymization and protocols. Most of these security frameworks lacks implementation evidence. A number of studies did not provide comprehensive performance metrics (accuracy, error, recall, precision) of the models besides using a single algorithm without validated justification. Therefore, a critique on the contribution, performance and areas of improvements discussed and summarized in the paper.

Dynamic Face Perception: The Role of Expertise in Dual Processing of Features and Configuration

Face perception is the basis of many types of social information exchange, but there is controversy over its underlying mechanisms. Researchers have theorized two processing pathways underlying facial perception: configural processing and featural processing. Featural processing focuses on the individual features of a face, whereas configural processing focuses on the spatial relations of features. To resolve the debate on the relative contribution of the two pathways in face perception, researchers have proposed a dual processing model that the two pathways contribute to two different perceptions, detecting face-like patterns and identifying individual faces. The dual processing model is based on face perception experiments that primarily use static faces. As we mostly interact with dynamic faces in real life, the generalization of the model to dynamic faces will advance our understanding of how faces are perceived in real life. This paper proposes a refined dual processing model of dynamic face perception, in which expertise in dynamic face perception supports identifying individual faces, and it is a learned behaviour that develops with age. Specifically, facial motions account for the advantages of dynamic faces, compared to static faces. This paper highlights two intrinsic characteristics of facial motions that enable the advantages of dynamic faces in face perception. Firstly, facial motion provides facial information from various viewpoints, and thus supports the generalization of face perception to the unlearned view of faces. Secondly, distinctive motion patterns serve as a cue to the identity of the face.

A partner's smile is not per se a safety signal: Psychophysiological response patterns to instructed threat and safety.

Recent studies on fear conditioning and pain perception suggest that pictures of loved ones (e.g., a romantic partner) may serve as a prepared safety cue that is less likely to signal aversive events. Challenging this view, we examined whether pictures of smiling or angry loved ones are better safety or threat cues. To this end, 47 healthy participants were verbally instructed that specific facial expressions (e.g., happy faces) cue threat of electric shocks and others cue safety (e.g., angry faces). When facial images served as threat cues, they elicited distinct psychophysiological defensive responses (e.g., increased threat ratings, startle reflex, and skin conductance responses) compared to viewing safety cues. Interestingly, instructed threat effects occurred regardless of the person who cued shock threat (partner vs. unknown) and their facial expression (happy vs. angry). Taken together, these results demonstrate the flexible nature of facial information (i.e., facial expression and facial identity) to be easily learned as signals for threat or safety, even when showing loved ones.

Non-contact PPG signal and heart rate estimation with multi-hierarchical convolutional network

Heartbeat rhythm and heart rate (HR) are important physiological parameters of the human body. This study presents an efficient multi-hierarchical spatio-temporal convolutional network that can quickly estimate remote physiological (rPPG) signal and HR from face video clips. First, the facial color distribution characteristics are extracted using a low-level face feature generation (LFFG) module. Then, the three-dimensional (3D) spatio-temporal stack convolution module (STSC) and multi-hierarchical feature fusion module (MHFF) are used to strengthen the spatio-temporal correlation of multi-channel features. In the MHFF, sparse optical flow is used to capture the tiny motion information of faces between frames and generate a self-adaptive region of interest (ROI) skin mask. Finally, the signal prediction module (SP) is used to extract the estimated rPPG signal. The heart rate estimation results show that the proposed network overperforms the state-of-the-art methods on three datasets, 1) UBFC-RPPG, 2) COHFACE, 3) our dataset, with the mean absolute error (MAE) of 2.15, 5.57, 1.75 beats per minute (bpm) respectively.

Effects of spatially filtered fearful faces and awareness on amygdala activity in adults with autism spectrum disorder: A magnetoencephalography study

BackgroundThe amygdala is pivotal in emotional face processing. Spatial frequencies (SFs) of visual images are divided and processed via two visual pathways: low spatial frequency (LSF) information is conveyed by the magnocellular pathway, while the parvocellular pathway carries high spatial frequency information. We hypothesized that altered amygdala activity might underlie atypical social communication caused by changes in both conscious and non-conscious emotional face processing in the brain in individuals with autism spectrum disorder (ASD). MethodEighteen adults with ASD and 18 typically developing (TD) peers participated in this study. Spatially filtered fearful- and neutral-expression faces and object stimuli were presented under supraliminal or subliminal conditions, and neuromagnetic responses in the amygdala were measured using 306-channel whole-head magnetoencephalography. ResultsThe latency of the evoked responses at approximately 200 ms to unfiltered neutral face stimuli and object stimuli in the ASD group was shorter than that in the TD group in the unaware condition. Regarding emotional face processing, the evoked responses in the ASD group were larger than those in the TD group under the aware condition. The later positive shift during 200–500 ms (ARV) was larger than that in the TD group, regardless of awareness. Moreover, ARV to HSF face stimuli was larger than that to the other spatial filtered face stimuli in the aware condition. ConclusionRegardless of awareness, ARV might reflect atypical face information processing in the ASD brain.

Progressive Transformer Machine for Natural Character Reenactment

Character reenactment aims to control a target person’s full-head movement by a driving monocular sequence that is made up of the driving character video. Current algorithms utilize convolution neural networks in generative adversarial networks, which extract historical and geometric information to iteratively generate video frames. However, convolution neural networks can merely capture local information with limited receptive fields and ignore global dependencies that play a crucial role in face synthesis, leading to generating unnatural video frames. In this work, we design a progressive transformer module that introduces multi-head self-attention with convolution refinement to simultaneously capture global-local dependencies. Specifically, we utilize the non-lapping window-based multi-head self-attention mechanism with hierarchical architecture to obtain the larger receptive fields at low-resolution feature map and thus extract global information. To better model local dependencies, we introduce the convolution operation to further refine the attentional weight in the multi-head self-attention mechanism. Finally, we use several stacked progressive transformer modules with the down-sampling operation to encode information of appearance information of previously generated frames and parameterized 3D face information of the current frame. Similarly, we use several stacked progressive transformer modules with the up-sampling operation to iteratively generate video frames. In this way, it can capture global-local information to facilitate generating video frames that are globally natural while preserving sharp outlines and rich detail information. Extensive experiments on several standard benchmarks suggest that the proposed method outperforms current leading algorithms.

Encoding of dynamic facial information in the middle dorsal face area

Faces in motion reveal a plethora of information through visual dynamics. Faces can move in complex patterns while transforming facial shape, e.g., during the generation of different emotional expressions. While motion and shape processing have been studied extensively in separate research enterprises, much less is known about their conjunction during biological motion. Here, we took advantage of the discovery in brain-imaging studies of an area in the dorsal portion of the macaque monkey superior temporal sulcus (STS), the middle dorsal face area (MD), with selectivity for naturalistic face motion. To gain mechanistic insights into the coding of facial motion, we recorded single-unit activity from MD, testing whether and how MD cells encode face motion. The MD population was highly sensitive to naturalistic facial motion and facial shape. Some MD cells responded only to the conjunction of facial shape and motion, others were selective for facial shape even without movement, and yet others were suppressed by facial motion. We found that this heterogeneous MD population transforms face motion into a higher dimensional activity space, a representation that would allow for high sensitivity to relevant small-scale movements. Indeed, we show that many MD cells carry such sensitivity for eye movements. We further found that MD cells encode motion of head, mouth, and eyes in a separable manner, requiring the use of multiple reference frames. Thus, MD is a bona fide face-motion area that uses highly heterogeneous cell populations to create codes capturing even complex facial motion trajectories.

Looking at the upper facial half enlarges the range of holistic face processing

Previous studies suggested that upper and lower facial halves might be involved in the human holistic face processing differently. In this study, we replicated and extended the finding above. In Experiment 1, we used the standard composite-face task to measure holistic face processing when participants made judgements on the upper and lower facial halves separately. Results showed that the composite-face effect was stronger for the upper facial half compared to the lower half. In Experiment 2, we investigated how facial information was integrated when participants focused on different features, using the perceptual field paradigm. Results showed that: (1) more “peripheral faces” were chosen when participants fixated at the eyes than when they fixated at the mouth; (2) less “peripheral faces” were chosen for inverted faces regardless of the fixated features. Findings from both experiments together indicate that more peripheral facial information were integrated when participants focused on the upper facial half, highlighting the significance of focusing on the upper facial half in face processing.

Invertible mask network for face privacy preservation

Face privacy Preservation is research subject has attracted significant interest. Existing face privacy-preserving methods aim at erasing the semantic information of faces; however, they cannot well preserve the reusability of original facial information. To achieve the naturalness of the processed face and the recoverability of the original protected face, this paper proposes a face privacy-preserving method based on Invertible Mask Network (IMN). First, a high-definition mask face is generated by the proposed Mask Generator model. Then, the mask face is put onto a protected face to obtain the masked face, in which the masked face is visually indistinguishable from the mask face. Finally, the mask face can be removed from the masked face and the recovered face can be obtained by authorized users, where the recovered face is visually indistinguishable from the protected face. The experimental results show that the proposed method can effectively protect the privacy of the protected face in subjective perception, the protection success rate is very high by the face recognition tool, and the proposed method can almost perfectly recover the protected face from the masked face.

Should I scan my face? The influence of perceived value and trust on Chinese users’ intention to use facial recognition payment

While China has seen the widespread adoption of facial recognition payment, concerns over the potential risks impede the further growth of user acceptance. Drawing on the perspectives of perceived value and trust, we developed a research model to explore how the Chinese users’ trade-off between perceived benefits (perceived convenience and perceived novelty) and perceived risks (perceived privacy risk and perceived financial risk) influences their usage intention. Based on survey data from 1,200 Chinese mobile payment users, this study uses structural equation modeling to examine the hypotheses. Our results show that perceived convenience positively influences perceived value, while privacy risk and financial risk negatively influence perceived value and trust. Although perceived novelty has no significant effect on perceived value and trust, perceived value positively influences trust, and both of them have a positive effect on the intention to use. Furthermore, our moderation analysis indicates that the influence of perceived value on the intention to use is stronger among users who consider facial information to be more sensitive. The theoretical implications for research as well as the practical implications for practitioners are also discussed.

The sagittal evaluation and classification of midfacial morphology in Chinese Han Youth: A cross-sectional study

Midface is located in the central area of the facial region and plays a crucial role in the aesthetic evaluation of one's appearance. The East Asian, especially the Chinese, has wide and flat midfacial bone structure, low piriform aperture area compared with the prominent zygomatic and oral region, which results in an unpleasant appearance. 1 Jin H.R. Won T.B. Rhinoplasty in the Asian Patient. Clin Plast Surg. 2016; 43: 265-279 Abstract Full Text Full Text PDF PubMed Scopus (22) Google Scholar In contrast to the patients' surging demand of midface filling surgery, there is a lack of evaluation standard for midfacial depression, nor quantitative and objective grading system for midface morphology. In the current study, we collected the three-dimensional (3D) facial information of Chinese Han Youth and summarized the morphological characteristics of their midface, focusing on the concavity and convexity from the sagittal plane.

Masked Face Recognition System Based on Attention Mechanism

With the continuous development of deep learning, the face recognition field has also developed rapidly. However, with the massive popularity of COVID-19, face recognition with masks is a problem that is now about to be tackled in practice. In recognizing a face wearing a mask, the mask obscures most of the facial features of the face, resulting in the general face recognition model only capturing part of the facial information. Therefore, existing face recognition models are usually ineffective in recognizing faces wearing masks. This article addresses this problem in the existing face recognition model and proposes an improvement of Facenet. We use ConvNeXt-T as the backbone of the network model and add the ECA (Efficient Channel Attention) mechanism. This enhances the feature extraction of the unobscured part of the face to obtain more useful information, while avoiding dimensionality reduction and not increasing the model complexity. We design new face recognition models by investigating the effects of different attention mechanisms on face mask recognition models and the effects of different data set ratios on experimental results. In addition, we construct a large set of faces wearing masks so that we can efficiently and quickly train the model. Through experiments, our model proved to be 99.76% accurate for real faces wearing masks. A combined accuracy of 99.48% for extreme environments such as too high or lousy contrast and brightness.

LMFNet: A Lightweight Multiscale Fusion Network With Hierarchical Structure for Low-Quality 3-D Face Recognition

Three-dimensional (3-D) face recognition (FR) can improve the usability and user-friendliness of human–machine interaction. In general, 3-D FR can be divided into high-quality and low-quality 3-D FR according to different interaction scenarios. The low-quality data can be easily obtained, so its application prospect is more extensive. However, the challenge is how to balance the trade-offs between data accuracy and real-time performance. To solve this problem, we propose a lightweight multiscale fusion network (LMFNet) with a hierarchical structure based on single-mode data for low-quality 3-D FR. First, we design a backbone network with only five feature extraction blocks to reduce computational complexity and improve the inference speed. Second, we devise a mid-low adjacent layer with a multiscale feature fusion (ML-MSFF) module to extract the facial texture and contour information, and a mid-high adjacent layer with a multiscale feature fusion (MH-MSFF) module to obtain the discriminative information in high-level features. Then, a hierarchical multiscale feature fusion (HMSFF) module is formed by combining these two modules mentioned above to acquire the local information of different scales. Finally, we enhance the expression of features by integrating HMSFF with a global convolutional neural network for improving recognition accuracy. Experiments on Lock3DFace, KinectFaceDB, and IIIT-D datasets demonstrate that our proposed LMFNet can achieve superior performance on low-quality datasets. Furthermore, experiments on the cross-quality database based on Bosphorus and the different intensity noise low-quality datasets based on UMB-DB and Bosphorus show that our network is robust and has a high generalization ability. It satisfies the real-time requirement, which lays a foundation for a smooth and user-friendly interactive experience.

Multi‐task multi‐scale attention learning‐based facial age estimation

Face-based age estimation strongly depends on deep residual networks (ResNets), used as the backbone in the relevant research. However, ResNet-based methods ignore the importance of some large-scale facial information and other facial age attributes. Inspired by the attention mechanism, a multi-task learning framework for face-based age estimation called multi-task multi-scale attention is proposed. First, the authors embed the alternative strategy structure of dilated convolution into ResNet34 to construct a multi-scale attention module (MSA) to improve the network's receptive field, which extracts local age-sensitive information while obtaining multi-scale features. The MSA can have a larger receptive field to extract both large-scale and local detailed feature information. Second, multi-task learning network structures are built to predict gender and race, which can share rigid network parameters to improve age estimation and improve the accuracy of age estimation by other age-related parameters. Finally, the Kullback-Leibler divergence loss is adopted between a Dirac delta label and a Gaussian prediction to guide the training. The numerical tests on the MORPH Album II and Adience datasets prove the superiority of the proposed method over other state-of-the-art ones.

The Two-Stage Ensemble Learning Model Based on Aggregated Facial Features in Screening for Fetal Genetic Diseases.

With the advancement of medicine, more and more researchers have turned their attention to the study of fetal genetic diseases in recent years. However, it is still a challenge to detect genetic diseases in the fetus, especially in an area lacking access to healthcare. The existing research primarily focuses on using teenagers' or adults' face information to screen for genetic diseases, but there are no relevant directions on disease detection using fetal facial information. To fill the vacancy, we designed a two-stage ensemble learning model based on sonography, Fgds-EL, to identify genetic diseases with 932 images. Concretely speaking, we use aggregated information of facial regions to detect anomalies, such as the jaw, frontal bone, and nasal bone areas. Our experiments show that our model yields a sensitivity of 0.92 and a specificity of 0.97 in the test set, on par with the senior sonographer, and outperforming other popular deep learning algorithms. Moreover, our model has the potential to be an effective noninvasive screening tool for the early screening of genetic diseases in the fetus.

Neural correlates of conscious processing of emotional faces: Evidence from event-related potentials

There is a theoretical debate between the early and late neural correlates of consciousness (NCCs). Previous studies using neutral face stimuli supported an early NCC and suggested that visual awareness negativity (VAN) is associated with consciousness, while late positivity (LP) reflects post-perceptual activity. However, emotional faces may help to examine the relationship between LP and consciousness due to the differences in late processing between emotional and neutral faces. To explore the effects of facial emotional information on NCCs, the present study manipulated consciousness with the inattentional blindness paradigm and used happy, fearful, and neutral faces as visual stimuli. The results showed that the conscious processing of emotional faces was correlated with VAN and LP, while the conscious processing of neutral faces was associated with VAN. First, the results suggest that VAN is an NCC, and the relationship between LP and consciousness is affected by facial emotional information. Second, VAN reflects the early perceptual experience of emotional faces, whereas LP may reflect the late conscious processing of emotional faces. Furthermore, source localization analysis showed that the LPs of emotional faces were mainly located in the frontal and parietal lobes, whereas those of neutral faces showed no significant activation. This suggests that facial emotional information may affect the brain regions associated with conscious processing. Time–frequency analysis showed that conscious processing is related to the enhancement of alpha and theta oscillation, indicating that conscious processing may be associated with the suppression of irrelevant stimuli. Overall, the present study suggests that the integration of the theories that support early and late NCCs helps explain the conscious processing of emotional faces.

Perception of strength, attractiveness and aggressiveness of Maasai male faces calibrated to handgrip strength: Evidence from a European sample.

Previous research showed that male and female members of the Maasai from Northern Tanzania judge images of facial morphs calibrated to greater handgrip strength (HGS) higher on strength and attractiveness, but lower on aggressiveness than those calibrated to lower HGS. The accurate assessment of male physical strength from facial information may be adaptive as suggested by the evidence on health and fitness-related benefits linked to high muscular strength. This study extends previous work by obtaining European female (n= 220) and male (n= 51) assessments of HGS-calibrated Maasai male faces. Participants rated five facial morphs for strength, attractiveness, and aggressiveness on computer screens. Perceived physical strength increased with morphs calibrated to higher HGS. The lowest and highest HGS morphs were judged lower in attractiveness than the others, and rated aggressiveness decreased in morphs calibrated to higher HGS. Given the high similarity between the current study findings and those previously reported from intra-population assessments of Maasai faces calibrated to HGS, we suggest that strength and aggressiveness perceptions of facial features associated with male physical strength may be universal. Attractiveness assessments of strength-related information in the faces of (very) strong men were less consistent across populations, possibly attributable to cultural and ecological contexts.

Bilingual toddlers show increased attention capture by static faces compared to monolinguals.

Bilingual infants rely differently than monolinguals on facial information, such as lip patterns, to differentiate their native languages. This may explain, at least in part, why young monolinguals and bilinguals show differences in social attention. For example, in the first year, bilinguals attend faster and more often to static faces over non-faces than do monolinguals (Mercure et al., 2018). However, the developmental trajectories of these differences are unknown. In this pre-registered study, data were collected from 15- to 18-month-old monolinguals (English) and bilinguals (English and another language) to test whether group differences in face-looking behaviour persist into the second year. We predicted that bilinguals would orient more rapidly and more often to static faces than monolinguals. Results supported the first but not the second hypothesis. This suggests that, even into the second year of life, toddlers' rapid visual orientation to static social stimuli is sensitive to early language experience.

Cross-modal information fusion for voice spoofing detection

In recent years, speaker verification systems have been used in many production scenarios. Unfortunately, they are still very vulnerable to different kinds of spoofing attacks, such as speech synthesis attacks, replay attacks, etc. Researchers have proposed many methods to defend against these attacks, but in the existing methods, researchers just focus on speech features. In recent studies, researchers have found that speech contains a large amount of face information. In fact, we can determine the speaker’s gender, age, mouth shape, and other information by voice. These information can help us distinguish spoofing attacks. Inspired by this phenomenon, we propose a generalized framework named GACMNet. To cope with different attack scenarios, we instantiated two different models. Our framework is mainly divided into data pre-processing phase, feature extraction phase, feature fusion phase, and classification phase. Specifically, our framework consists of two branches. On the one hand, we extract face features in speech by a convolutional neural network. On the other hand, we use a densely connected network to extract speech features. For the more, we designed a global attention-based information fusion mechanism to distinguish the importance of each part of the features. Our solution was proven to be effective in two large scenarios. Compared to the existing methods, our model improves the tandem decision cost function (t-DCF) and equal error rate (EER) scores by 9% and 11% in the logical access scenario, respectively, our model improves the EER score by 10% in the physical access scenario.