Facial Information Research Articles

Attention-Oriented Two-Stream Convolutional Networks for the Identification of Face Spoofing

Face recognition has been extensively researched and has seen tremendous success in a variety of applications over the past few decades because the human face retains the most information for identifying individuals. Modern face recognition systems are still vulnerable to face spoofing attacks, such as the face video replay attack. Despite the fact that numerous efficient antispoofing techniques have been put forth, we discover that illuminations impair the effectiveness of many of the current techniques. It encourages us to create illumination-invariant anti-spoofing techniques. In this work, we propose a two stream convolutional neural network (TSCNN) that operates on two complementary spaces: multi-scale retinex (MSR) space (illumination invariant space) and RGB space (original imaging space). In particular, MSR is invariant to illumination but contains less detailed facial information than RGB space, which contains detailed facial textures but is sensitive to illumination. Furthermore, the high-frequency information that is discriminative for face spoofing detection can be efficiently captured by MSR images. To learn the discriminative features for anti-spoofing, the TSCNN is fed images from two spaces. We suggest an attention-based fusion technique that can successfully capture the complementarity of two features in order to smoothly fuse the features from two sources (RGB and MSR). We test the suggested framework on a number of databases, including OULU, CASIA-FASD, and REPLAY-ATTACK, and we obtain very competitive results. We perform cross-database experiments to further confirm the suggested strategies' capacity for generalization, and the outcomes demonstrate the high efficacy of our approach.

DTIA: Disruptive Text-Image Alignment for Countering Text-to-Image Diffusion Model Personalization

AbstractPersonalized content synthesis technologies based on diffusion models have achieved significant breakthroughs, allowing for the creation of specific images from just a few reference photos. However, when these technologies are used to create fake news or unsettling content targeting individuals, they pose a substantial risk to society. To address this issue, current methods generate adversarial samples by adversarially maximizing the training loss, thus disrupting any personalized generation model trained with these samples. However, these methods are not efficient and do not fully consider the intrinsic mechanisms of successful personalization attacks. In this paper, we introduce an innovative Disruptive Text-Image Alignment (DTIA) framework. Based on the analysis that existing methods succeed in their attacks due to an overfitting of text to noise, which results in inaccurate face information matching, we have designed a Text-Image Mis-Match Attack framework. This framework aims to disrupt the model’s learning of associations between input faces and specific texts, thereby reducing unnecessary computational load. We also studied how the choice of timestep in diffusion models affects adversarial attacks and proposed a step schedule strategy to enhance algorithm efficiency. Extensive experiments on facial benchmarks have demonstrated that our DTIA framework not only disrupts personalized generation models but also significantly improves model efficiency.

A Deep Learning Network for Accurate Retinal Multidisease Diagnosis Using Multiview Fusion of En Face and B-Scan Images: A Multicenter Study.

Accurate diagnosis of retinal disease based on optical coherence tomography (OCT) requires scrutiny of both B-scan and en face images. The aim of this study was to investigate the effectiveness of fusing en face and B-scan images for better diagnostic performance of deep learning models. A multiview fusion network (MVFN) with a decision fusion module to integrate fast-axis and slow-axis B-scans and en face information was proposed and compared with five state-of-the-art methods: a model using B-scans, a model using en face imaging, a model using three-dimensional volume, and two other relevant methods. They were evaluated using the OCTA-500 public dataset and a private multicenter dataset with 2330 cases; cases from the first center were used for training and cases from the second center were used for external validation. Performance was assessed by averaged area under the curve (AUC), accuracy, sensitivity, specificity, and precision. In the private external test set, our MVFN achieved the highest AUC of 0.994, significantly outperforming the other models (P < 0.01). Similarly, for the OCTA-500 public dataset, our proposed method also outperformed the other methods with the highest AUC of 0.976, further demonstrating its effectiveness. Typical cases were demonstrated using activation heatmaps to illustrate the synergy of combining en face and B-scan images. The fusion of en face and B-scan information is an effective strategy for improving the diagnostic accuracy of deep learning models. Multiview fusion models combining B-scan and en face images demonstrate great potential in improving AI performance for retina disease diagnosis.

FMIF: facial multi-feature information fusion for driver fatigue detection

FMIF: facial multi-feature information fusion for driver fatigue detection

The role of the opponent's head in perception of kick target location in martial arts.

Athletes in Martial Arts must anticipate the target of their opponent's kick or strike to avoid contact. Findings suggest that features, e.g., head and face may play a greater role in predicting opponent action intent compared to global movement information, however little research has explored the role of the head in action anticipation. The aim of this study was to examine the role of the head in predicting the target of a kicking action in martial arts. N = 76 volunteer participants (n = 32 athletes > 5 years of experience, n = 36 non-athletes with no experience) were asked to watch a series of video clips depicting various kicking techniques with differing levels of spatial occlusion of the head. These clips were also temporally occluded compelling participants to predict the landing target of each kick (i.e., head or chest). The hypothesis of the current study is that athletes would be more accurate than non-athletes, but there was no significant effect of expertise on accuracy. Both athletes and non-athletes performed well above chance level performance. Head occlusion did not significantly influence performance and did not interact with expertise, suggesting head and face information did not play a role in predicting opponent action intent. Across participants the landing target of the roundhouse kick was identified with greater accuracy than the front or the back kick. Additionally, participants identified kicks from the rear leg with greater accuracy than the front leg. These findings have significant implications for combat sports where athletes are required to anticipate the action intent of their opponent to formulate an effective defensive response.

CGFTNet:Content-Guided Frequency Domain Transform Network for Face Super-Resolution

Recent advancements in face super resolution (FSR) have been propelled by deep learning techniques using convolutional neural networks (CNN). However, existing methods still struggle with effectively capturing global facial structure information, leading to reduced fidelity in reconstructed images, and often require additional manual data annotation. To overcome these challenges, we introduce a content-guided frequency domain transform network (CGFTNet) for face super-resolution tasks. The network features a channel attention-linked encoder-decoder architecture with two key components: the Frequency Domain and Reparameterized Focus Convolution Feature Enhancement module (FDRFEM) and the Content-Guided Channel Attention Fusion (CGCAF) module. FDRFEM enhances feature representation through transformation domain techniques and reparameterized focus convolution (RefConv), capturing detailed facial features and improving image quality. CGCAF dynamically adjusts feature fusion based on image content, enhancing detail restoration. Extensive evaluations across multiple datasets demonstrate that the proposed CGFTNet consistently outperforms other state-of-the-art methods.

Detecting Missing Persons Using Face Recognition

In response to the challenge of finding missing people, we developed an advanced facial recognition model integrated with the TensorFlow machine learning platform. Traditional methods are often inefficient and slow, but our innovative solution uses facial recognition technology to optimize the search process. Missing people's facial information is fed into the system, which searches social media, CCTV footage, and public places for matches. Once a match is identified, carers and local authorities are immediately notified, enabling fast and accurate searches. Through advanced algorithms and machine learning, our solutions significantly increase your chances of success while ensuring a high level of privacy and security. Integration with the Django framework provides a powerful and scalable platform that improves the overall effectiveness and efficiency of locating missing people and reuniting them with their families.

Secure Supply Chain Information Interchange using Distributed Trust Backbone

International trade requires transparent visibility of the goods transportation. High-quality data related to containers is essential for container movement across the border speed. However, customs and port authorities face information incorrectness and inconsistency, which are significant determinants that decrease the performance of container clearance in supply chain activities. The Seamless Integrated Data Pipeline principle has been proposed to overcome the mentioned data quality shortcomings and enhance supply chain visibility. Based on the Data Pipeline idea, we proposed the Distributed Trust Backbone (DTB) as a model of secure information exchange between parties within the supply chain activity. However, the supply chain data is highly dynamic. Access control on dynamic resources is the key to enabling secure data exchange and clear visibility. We take this challenge up in this paper. We propose an access control mechanism based on the supply chain Data Pipeline concept and apply it to the DTB model. The elaboration on the concrete detail of the system is presented in this paper. The prototype has been developed and performed in the simulation tests. It reduces 58% of requesting data for supply chain activities. The results of the experiments show that our proposed method performs 100% access control to data with BigO(1) accessing the Access Control List. It can ensure that the information for decision-making in the supply chain is of high quality. The supply chain visibility is clearer and speeds up a modern information exchange system of supply chains.

A Study of Potential Applications of Student Emotion Recognition in Primary and Secondary Classrooms

Emotion recognition is critical to understanding students’ emotional states. However, problems such as crowded classroom environments, changing light, and occlusion often affect the accuracy of recognition. This study proposes an emotion recognition algorithm specifically for classroom environments. Firstly, the study adds the self-made MCC module and the Wise-IoU loss function to make object detection in the YOLOv8 model more accurate and efficient. Compared with the native YOL0v8x, it reduces the parameters by 16% and accelerates the inference speed by 20%. Secondly, in order to address the intricacies of the classroom setting and the specific requirements of the emotion recognition task, a multi-channel emotion recognition network (MultiEmoNet) has been developed. This network fuses skeletal, environmental, and facial information, and introduces a central loss function and an attention module AAM to enhance the feature extraction capability. The experimental results show that MultiEmoNet achieves a classification accuracy of 91.4% on a homemade classroom student emotion dataset, which is a 10% improvement over the single-channel classification algorithm. In addition, this study also demonstrates the dynamic changes in students’ emotions in the classroom through visual analysis, which helps teachers grasp students’ emotional states in real time. This paper validates the potential of multi-channel information-fusion deep learning techniques for classroom teaching analysis and provides new ideas and tools for future improvements to emotion recognition techniques.

Early Impairment of Face Perception in Post-Stroke Depression: An ERP Study.

Objective: Face recognition is an important cognitive function of the human brain. Post stroke depression (PSD) is a common mental complication after stroke, which has a serious impact on individual physical function recovery and quality of life. This study aims to explore the face perception characteristics of PSD through electrophysiological indicators N170 and VPP, and provide an objective basis for the early evaluation of facial cognitive dysfunction in PSD. Methods: 58 patients in the cerebral small vessel disease (CSVD) with depressive symptoms (PSD) and 188 patients in the pure CSVD (NPSD). At the same time, 30 healthy subjects were selected as the healthy controls (HC). The differences of N170 and VPP components between the three groups were compared under the stimulation of inverted faces and upright faces. Results: PSD patients exhibited significantly longer peak latency and lower amplitude of N170 and VPP under both inverted and upright face stimulation compared to HC and NPSD. These results suggest that PSD patients have defects in early face recognition, there are abnormalities in the early perception and structural encoding of face information, and both the "overall mechanism" and "feature mechanism" of face recognition are damaged. Conclusions: These findings provide neuroelectrophysiological evidence for impaired emotionless face recognition in PSD patients.

DISP-08. GBM PATIENTS FACE TTF ACCESS AND INFORMATION INEQUITY THAT MAY IMPACT PATIENT OUTCOMES AND QUALITY-OF-LIFE

Abstract We evaluated the quality and accessibility of healthcare for Glioblastoma (GBM) patients across the United States through an online survey distributed to the OurBrainBank community and partners and promoted on social media. A total of 297 patients or caregivers completed the survey (85% caregivers; 15% GBM patients). Median age at diagnosis was 57, lower than the national average. 78% described having some college education, and 69% reported commercial insurance coverage. Our preliminary findings highlight that 82% of respondents reported that their treating doctor recommended the standard-of-care for GBM consisting of tumor resection, 3-6 weeks of chemo/radiation, and 6 months of chemotherapy and sometimes Tumor-Treating-Fields (TTF), per the Comprehensive- Cancer-Network (2021) guidelines. 40% reported no discussion of TTF as an option. Of those offered TTF, nearly half (46%) received it as part of their initial treatment combined with standard chemotherapy and 36% received it as a subsequent option. For those attempting to access TTF, 29% encountered difficulties, often due to financial constraints linked to insurance denials. These findings illuminate significant concerns regarding dissemination of information on treatment options, as well as access to treatments. Access to these treatments can have a significant influence on survival and quality of life of GBM patients. The substantial number of patients not offered TTF, combined with challenges to accessing treatments due to insurance issues, highlight critical obstacles to engaging in informed and shared decision-making. Compared to the general GBM-affected population, our survey respondents tend to be younger and from urban areas, indicating a possible selection bias in survey participation. This skew could lead to an underrepresentation of the challenges encountered by patients facing additional barriers. We are adopting measures to engage a more diverse participant group, especially from rural communities Ongoing targeted recruitment will further elucidate barriers to patient-centered care in GBM, with the potential for significant improvements in survival and QOL.

Adaptive representation learning and sample weighting for low-quality 3D face recognition

3D face recognition (3DFR) algorithms have advanced significantly in the past two decades by leveraging facial geometric information, but they mostly focus on high-quality 3D face scans, thus limiting their practicality in real-world scenarios. Recently, with the development of affordable consumer-level depth cameras, the focus has shifted towards low-quality 3D face scans. In this paper, we propose a method for low-quality 3DFR. On one hand, our approach employs the normalizing flow to model an adaptive-form distribution for any given 3D face scan. This adaptive distributional representation learning strategy allows for more robust representations of low-quality 3D face scans (which may be caused by the scan noises, pose or occlusion variations, etc.). On the other hand, we introduce an adaptive sample weighting strategy to adjust the importance of each training sample by measuring both the difficulty of being recognized and the data quality. This adaptive sample weighting strategy can further enhance the robustness of the deep model and meanwhile improve its performance on low-quality 3DFR. Through comprehensive experiments, we demonstrate that our method can significantly improve the performance of low-quality 3DFR. For example, our method achieves competitive results on both the IIIT-D database and the Lock3DFace datasets, underscoring its effectiveness in addressing the challenges associated with low-quality 3D faces.

Learning face super-resolution through identity features and distilling facial prior knowledge

Deep learning techniques in electronic surveillance have shown impressive performance for super-resolution (SR) of captured low-quality face images. Most of these techniques adopt facial priors to improve the feature details in the resultant super-resolved images. However, the estimation of facial priors from the captured low-quality images is often inaccurate in real-life situations because of their tiny, noisy, and blurry nature. Thus, the fusion of such priors badly affects the performance of these models. Therefore, this work presents a teacher–student-based face SR framework that efficiently preserves the personal facial structure information in the super-resolved faces. In the proposed framework, the teacher network exploits the facial heatmap-based ground-truth-prior to learn the facial structure that is utilized by the student network. The student network is trained with the identity feature loss for maintaining the identity and facial structure information in reconstructed high-resolution (HR) face images. The performance of the proposed framework is evaluated by conducting the experimental study on standard datasets namely CelebA-HQ and LFW face. The experimental results reveal that the proposed technique conquers the existing methods for the face SR task.

Analysis of emotions of online car-hailing drivers under different driving conditions and scenarios

Emotion is an important factor that affects driving behavior, and thus, drivers’ emotions are closely related to overall traffic safety. We investigated the emotional expressions of online car-hailing drivers under two driving conditions: with passenger(s) and without passenger(s). We recruited 16 male car-hailing drivers and collected a total of 91.5 h of data using non-contact equipment. We employed FaceReader 8.0 software to analyze the collected video data and extract the drivers’ facial expression information, thereby identifying six emotions expressed by the drivers. We then compared the frequency of the occurrence of the six emotions between the two conditions. The frequency rates indicate that the drivers exhibited more emotions when no passengers were in the vehicle. The chi-square test results indicate significant differences in the drivers’ emotions under the two conditions. For example, happiness is related to chatting with passengers. Also, the drivers exhibited more aggressive driving behavior during trips without passengers, and such behavior often was accompanied by negative emotions, such as anger. We also investigated drivers’ emotions under three scenarios that often occur while online car-hailing drivers are working: driver distractions, passenger interaction, and the traffic environment. The understanding of drivers’ emotions and the relationships between those emotions and scenarios that take place under different driving conditions can facilitate the identification of drivers’ intentions and provide guidance for the development of safe driving assistance warning systems. Dangerous driving behavior can be reduced through intervention and the monitoring of drivers’ emotions for the enhanced overall safety of roadway travel.

Semantic Content in Face Representation: Essential for Proficient Recognition of Unfamiliar Faces by Good Recognizers.

Face recognition is adapted to achieve goals of social interactions, which rely on further processing of the semantic information of faces, beyond visual computations. Here, we explored the semantic content of face representation apart from visual component, and tested their relations to face recognition performance. Specifically, we propose that enhanced visual or semantic coding could underlie the advantage of familiar over unfamiliar faces recognition, as well as the superior recognition of skilled face recognizers. We asked participants to freely describe familiar/unfamiliar faces using words or phrases, and converted these descriptions into semantic vectors. Face semantics were transformed into quantifiable face vectors by aggregating these word/phrase vectors. We also extracted visual features from a deep convolutional neural network and obtained the visual representation of familiar/unfamiliar faces. Semantic and visual representations were used to predict perceptual representation generated from a behavior rating task separately in different groups (bad/good face recognizers in familiar-face/unfamiliar-face conditions). Comparisons revealed that although long-term memory facilitated visual feature extraction for familiar faces compared to unfamiliar faces, good recognizers compensated for this disparity by incorporating more semantic information for unfamiliar faces, a strategy not observed in bad recognizers. This study highlights the significance of semantics in recognizing unfamiliar faces.

A Facial Motion Retargeting Pipeline for Appearance Agnostic 3D Characters.

3D facial motion retargeting has the advantage of capturing and recreating the nuances of human facial motions and speeding up the time-consuming 3D facial animation process. However, the facial motion retargeting pipeline is limited in reflecting the facial motion's semantic information (i.e., meaning and intensity), especially when applied to nonhuman characters. The retargeting quality heavily relies on the target face rig, which requires time-consuming preparation such as 3D scanning of human faces and modeling of blendshapes. In this paper, we propose a facial motion retargeting pipeline aiming to provide fast and semantically accurate retargeting results for diverse characters. The new framework comprises a target face parameterization module based on face anatomy and a compatible source motion interpretation module. From the quantitative and qualitative evaluations, we found that the proposed retargeting pipeline can naturally recreate the expressions performed by a motion capture subject in equivalent meanings and intensities, such semantic accuracy extends to the faces of nonhuman characters without labor-demanding preparations.

댓글분석을 활용한 사이코패스 인식: 웹크롤링과 LDA를 중심으로

This study explored the perceptions of psychopath among the general public in South Korea by collecting comments through web crawling. To uncover the hidden structures and automatically extract topics, we utilized Latent Dirichlet Allocation (LDA) for topic modeling. A total of 29,126 comments were collected for the final analysis, and through the preprocessing of unstructured natural language data, 7,519 unique words were identified. The results revealed four distinct topics. Topic 1 was named 'Negative perception and demand for capital punishment' as it primarily contained negative words about psychopaths and calls for the death penalty. Topic 2, which was dominated by words related to judges' rulings, was named 'Expectations and concerns about judicial decisions.' Topic 3 was named 'Demands for personal information disclosure,' consisting mainly of words related to the disclosure of personal and facial information. Topic 4 was named 'Words associated with psychopath,' as it included words frequently mentioned in conjunction with psychopath. This research identified several misconceptions about psychopath. It was observed that the general public tends to view psychopath as a mental illness that could justify a reduced sentence, and they perceive psychopath as irredeemable. Thus, this study contributes to enhancing the understanding of psychopath by capturing the social perception and diversity related to it.

ADYOLOv5-Face: An Enhanced YOLO-Based Face Detector for Small Target Faces

Benefiting from advancements in generic object detectors, significant progress has been achieved in the field of face detection. Among these algorithms, the You Only Look Once (YOLO) series plays an important role due to its low training computation cost. However, we have observed that face detectors based on lightweight YOLO models struggle with accurately detecting small faces. This is because they preserve more semantic information for large faces while compromising the detailed information for small faces. To address this issue, this study makes two contributions to enhance detection performance, particularly for small faces: (1) modifying the neck part of the architecture by integrating a Gather-and-Distribute mechanism instead of the traditional Feature Pyramid Network to tackle the information fusion challenges inherent in YOLO-based models; and (2) incorporating an additional detection head specifically designed for detecting small faces. To evaluate the performance of the proposed face detector, we introduce a new dataset named XD-Face for the face detection task. In the experimental section, the proposed model is trained using the Wider Face dataset and evaluated on both Wider Face and XD-face datasets. Experimental results demonstrate that the proposed face detector outperforms other excellent face detectors across all datasets involving small faces and achieved improvements of 1.1%, 1.09%, and 1.35% in the AP50 metric on the WiderFace validation dataset compared to the baseline YOLOv5s-based face detector.

A Student Facial Expression Recognition Model Based on Multi-Scale and Deep Fine-Grained Feature Attention Enhancement.

In smart classroom environments, accurately recognizing students' facial expressions is crucial for teachers to efficiently assess students' learning states, timely adjust teaching strategies, and enhance teaching quality and effectiveness. In this paper, we propose a student facial expression recognition model based on multi-scale and deep fine-grained feature attention enhancement (SFER-MDFAE) to address the issues of inaccurate facial feature extraction and poor robustness of facial expression recognition in smart classroom scenarios. Firstly, we construct a novel multi-scale dual-pooling feature aggregation module to capture and fuse facial information at different scales, thereby obtaining a comprehensive representation of key facial features; secondly, we design a key region-oriented attention mechanism to focus more on the nuances of facial expressions, further enhancing the representation of multi-scale deep fine-grained feature; finally, the fusion of multi-scale and deep fine-grained attention-enhanced features is used to obtain richer and more accurate facial key information and realize accurate facial expression recognition. The experimental results demonstrate that the proposed SFER-MDFAE outperforms the existing state-of-the-art methods, achieving an accuracy of 76.18% on FER2013, 92.75% on FERPlus, 92.93% on RAF-DB, 67.86% on AffectNet, and 93.74% on the real smart classroom facial expression dataset (SCFED). These results validate the effectiveness of the proposed method.

The 2024 Richardson Lecture: Prosopagnosia - A Classic Neurologic Deficit Meets the Modern Era.

Acquired prosopagnosia is a rare disorder, but it serves as a model for impairments in expert-level visual processing. This review discusses five key observations made over the past 30 years. First, there are variants, an apperceptive type linked to damage to the inferior occipitotemporal cortex and an amnestic type associated with anterior temporal lesions, both either right or bilateral. Second, these variants are clustered in syndromes with other perceptual deficits, the apperceptive type with field defects, dyschromatopsia and topographagnosia, and the amnestic type with topographagnosia and the auditory disorders of phonagnosia and acquired amusia. Third, extensive testing often shows additional problems with recognizing exemplars of other objects, especially when degrees of expertise are taken into account. Fourth, the prosopagnosic impairment does not affect all facial information. For example, the perception of expression and lip-reading likely depends on other neural substrates than those for processing facial identity. Last, face perception in prosopagnosia is not immutable but can improve with extensive training, though as yet this does not represent a cure for the condition. Continuing work with neural networks and animal models will enhance our understanding of this intriguing condition and what it tells us about how our brains process vision.