Facial Posture Research Articles

The Experiencing Face

In this study, I uncover the moment-by-moment emergence of what I call an “experiencing face” during kisses and hugs between romantic partners. At such moments of intimate touch, the facial gestures often momentarily “shut down” to simply give way to experiencing the emotion and tactile “being-with another person” (M.H. Goodwin, 2017: 76). Focusing on the multisensoriality of the face during kisses and hugs, I reflect on some ways of face-on-communication—direct communication through faces touching. Previous EMCA studies have taken a communicative (see Ruusuvuori, 2013) approach to facial expressions. I add to the study of facial expressions by including a focus on the simultaneity of, and an interplay between, communicative and felt aspects of the face in interaction. Taking a phenomenological perspective on interaction studies based on Merleau-Ponty (1962), I approach facial postures from an intercorporeal framework (Meyer et al., 2017): facial postures are expressions of “living bodies” (Streeck, 2013) entailing felt and communicative features.

Nasofacial Groove Pedicled Flap for the Reconstruction of Lateral Alar Defect.

The repair of nasal alar defects is challenging for plastic surgeons, and there is currently no standard operation. Herein, the authors reported the clinical outcomes of a nasofacial groove pedicled flap for the reconstruction of alar defect. This retrospective study included patients who underwent the nasofacial groove pedicled flap for the reconstruction of alar defect between January 2018 and June 2020. Photographs of standard facial postures were taken before and after surgery to record the surgical results of the patients. The patient's medical history was reviewed retrospectively. Self-reported satisfaction of patients on scar morphology and reconstructive effect were evaluated with a questionnaire survey. There were 26 eligible patients enrolled, and all patients were followed up for more than 1 year after surgery. All flaps were free of ischemia and necrosis and healed well. No patient experienced restricted nostril ventilation. Eight patients underwent reoperation to trim the flap pedicle and the scar. Eight patients (8/26) reported "very satisfied," and 17 patients (17/26) reported "satisfied" with the repair effect and scar morphology. One patient went through multiple laser treatments to improve her scars but still remained visible hyperpigmentation. She was dissatisfied with postoperative flap pigmentation but was satisfied with the correction effect. The clinical results indicated that the nasal groove flap was safe for the treatment of the lateral alar defect, and the patients were satisfied with the clinical results. The authors believe that this flap can be used as an alternative method for repairing the lateral alar defect. Level -IV, therapeutic study.

Enhancing face recognition with latent space data augmentation and facial posture reconstruction

The small amount of training data for many state-of-the-art deep learning-based Face Recognition (FR) systems causes a marked deterioration in their performance. Although a considerable amount of research has addressed this issue by inventing new data augmentation techniques, using either input space transformations or Generative Adversarial Networks (GAN) for feature space augmentations, these techniques have yet to satisfy expectations. In this paper, we propose an approach named Face Representation Augmentation (FRA) for augmenting face datasets. To the best of our knowledge, FRA is the first method that shifts its focus towards manipulating the face embeddings generated by any face representation learning algorithm to create new embeddings representing the same identity and facial emotion but with an altered posture. Extensive experiments conducted in this study convince us of the efficacy of the methodology and its power to provide noiseless, completely new facial representations to improve the training procedure of any FR algorithm. Therefore, FRA can help the recent state-of-the-art FR methods by providing more data for training FR systems. The proposed method, using experiments conducted on the Karolinska Directed Emotional Faces (KDEF) dataset, improves the identity classification accuracies by 9.52 %, 10.04 %, and 16.60 %, in comparison with the base models of MagFace, ArcFace, and CosFace, respectively.

Fusion of RetinaFace and improved FaceNet for individual cow identification in natural scenes

Cows’ posture change is the fatal influencing factor for accurate identification of individual cows. To achieve non-contact, high-precision detection and identification of individual cows in farm environment, a cow individual identification method by the fusion of RetinaFace and improved FaceNet was proposed. MobileNet-enhanced RetinaFace was applied to ameliorate the impact of output channel quantity and convolution kernel dynamics using depthwise convolution combined with pointwise convolution. Regression predictions of bovine facial features and keypoints were generated under varying distances, scales and sizes. FaceNet's core feature network was enhanced through MobileNet integration, and the loss function was jointly optimized with Cross Entropy Loss and Triplet Loss to achieve a quicker and more stable convergence curve. The distances between the generated embedding vectors of cow facial features were corresponding to the similarity between cow faces, enabling accurate matching. RetinaFace exhibited detection false negative rates of 2.67%, 0.66%, 2.67%, and 3.33% under conditions of occlusion, no occlusion, low light, and bright light for cow facial detection. For cow facial pattern detection, the false negative rates for black and white patterns, pure black and pure white were 1.33%, 6.00% and 8.00%, respectively. Regarding cow facial posture changes, the false negative rates for face upward, bowing down, profile, and normal posture were 1.33%, 1.33%, 4.00% and 0.66%, respectively. Improved FaceNet model achieved an accuray of 99.50% on training set and 83.60% on test set. In comparison to YOLOX, the recognition model presented in this research demonstrated increased accuracy in cow facial detection under occlusion, no occlusion and strong lighting conditions by 2.67%, 0.40%, and 0.40%, respectively. Moreover, the accuracy for patterns with pure black and pure white tones surpassed that of YOLOX by 1.06% and 5.71%, correspondingly. Additionally, the accuracy rates for face upward, bowing down, profile and normal posture were higher than YOLOX by 2.00%, 3.34%, 2.66% and 0.40%, respectively. The proposed model demonstrates the proficiency in accurately identifying individual cows in natural scenes.

Analysis and Study of Fatigue Driving Detection System based on Multimodal Fusion

Fatigue driving always endangers road traffic safety and is one of the main causes of traffic accidents. How to effectively monitor and determine the fatigue driving state, so as to realize the fatigue early warning, has become a hot spot of scientific research. The traditional detection methods are mostly based on a certain human physiological parameters, through the monitoring of the physiological parameters changes, to complete the determination of the fatigue state. Compared with the traditional detection methods, the fatigue detection method based on computer vision has the advantages of accurate and reliable detection results, detection not relying on wearable devices, and good human-computer interaction experience. However, in the specific algorithm application scenarios, there are often interference from environmental factors such as partial occlusion of the inspected image, low image resolution, complex illumination of the detection environment, and unstable acquisition equipment. In the face of many challenges, how to effectively improve the detection accuracy of fatigue state, and the algorithm has the practical application space, still needs to be explored. The traditional fatigue detection method based on facial information often detects and analyzes a certain subordination. Due to the detection of individual differences and the fuzzy definition of fatigue state, the algorithm has the problems of high recognition error rate and poor robustness. Moreover, the single attribute model separates the potential integration and exclusion relationship between facial attributes. At the same time, the huge model size and complex model structure also limit the actual implementation of the algorithm. In view of these urgent problems, this paper first proposes the fusion model of face detection and head posture analysis based on the multi-task convolutional neural network (MTCNN). The model uses the architecture strategy of multi-task cascade to complete the complete facial posture analysis task while completing the facial detection task. Secondly, based on the lightweight convolutional neural network (SqueezeNet) of the network cascade, the facial key point detection network is designed to obtain the 72-point key point coordinates of the face. For the obtained facial attribute information, extract fatigue features and propose a fatigue determination method of multimodal fusion. Finally, combined with the research results, the construction of the online fatigue detection and early warning system platform was completed.

A longitudinal study of facial asymmetry in a normal birth cohort up to 6 years of age and the predisposing factors.

This prospective, population-based cohort study aimed to investigate the development of facial asymmetry up to 6 years of age using a three-dimensional (3D) soft tissue imaging method in a normal population. In addition, the study sought to identify potential predisposing factors to facial asymmetry. A total of 102 newborns were enrolled in the study at birth. 3D stereophotogrammetric images of the head and face were analysed at the ages of 12 months (T1), 3 years (T2), and 6 years (T3). The surface-based analysis involved the calculation of the average distance (mm) and the symmetry percentage (%) between the original and mirrored surfaces. For landmark-based analysis, the distance of facial landmarks to the facial midline was examined. The final analysis included 70 (68.6%) subjects. Surface-based analysis showed a significant improvement of facial symmetry from T1 to T3 in all facial areas. Landmark-based analysis showed that upper facial landmarks were located, on average, slightly on the left and lower facial landmarks slightly on the right in relation to the facial midline (P < 0.001). The size of the study population was limited. Facial posture may affect the reliability of the results, especially in younger children. Facial asymmetry is detectable in early childhood and tends to reduce with age in young children. The lower face deviates slightly to the right, and the upper face to the left in relation to the facial midline. Possible predisposing factors for facial asymmetry at the age of 6 years include deformational plagiocephaly, sleeping position, and previous facial asymmetry.

Analysis of Various Facial Expressions of Horses as a Welfare Indicator Using Deep Learning.

This study aimed to prove that deep learning can be effectively used for identifying various equine facial expressions as welfare indicators. In this study, a total of 749 horses (healthy: 586 and experiencing pain: 163) were investigated. Moreover, a model for recognizing facial expressions based on images and their classification into four categories, i.e., resting horses (RH), horses with pain (HP), horses immediately after exercise (HE), and horseshoeing horses (HH), was developed. The normalization of equine facial posture revealed that the profile (99.45%) had higher accuracy than the front (97.59%). The eyes-nose-ears detection model achieved an accuracy of 98.75% in training, 81.44% in validation, and 88.1% in testing, with an average accuracy of 89.43%. Overall, the average classification accuracy was high; however, the accuracy of pain classification was low. These results imply that various facial expressions in addition to pain may exist in horses depending on the situation, degree of pain, and type of pain experienced by horses. Furthermore, automatic pain and stress recognition would greatly enhance the identification of pain and other emotional states, thereby improving the quality of equine welfare.

Enhanced Deep Learning Hybrid Model of CNN Based on Spatial Transformer Network for Facial Expression Recognition

One of the most common approaches through which people communicate is facial expressions. A large number of features documented in the literature were created by hand, with the goal of overcoming specific challenges such as occlusions, scale, and illumination variations. These classic methods are then applied to a dataset of facial images or frames in order to train a classifier. The majority of these studies perform admirably on datasets of images shot in a controlled environment, but they struggle with more difficult datasets (FER-2013) that have higher image variation and partial faces. The nonuniform features of the human face as well as changes in lighting, shadows, facial posture, and direction are the key obstacles. Techniques of deep learning have been studied as a set of methodologies for gaining scalability and robustness on new forms of data. In this paper, we look at how well-known deep learning techniques (e.g. GoogLeNet, AlexNet) perform when it comes to facial expression identification, and propose an enhanced hybrid deep learning model based on STN for facial emotion recognition, which gives the best feature extraction and classification in one go and maximizes the accuracy for a large number of samples on FERG, JAFFE, FER-2013, and CK+ datasets. It is capable of focusing on the main parts of the face and attaining extensive development over preceding fashions on the FERG, JAFFE, CK+ datasets, and the more challenging one namely FER-2013.

Timing of smile suppression during the articulation of labials in smiled speech

Past studies have shown that opposing forces on the lips can be reconciled by suppressing a smile during speech-related lip closure movements [Liu et al., 2020, ISSP]. However, the exact timing of this suppression on the smiling posture remains unknown. The purpose of this study was to determine the onset and duration of smile suppression during labial production in smiled speech. We collected video footage of participants reading sentences that contain labial sounds (/m, f, v, b, p, w/) under three facial posture conditions (neutral, smile, laugh), and extracted short video clips of the labial sound productions. We then analyzed the extent of lip spreading in each facial posture condition using OpenFace 2.0 [Baltrušaitis et al., 2018, IEEE]. Our preliminary results show that smile is suppressed in the smiling and laughing conditions, with attenuation beginning approximately 300ms prior to labial sound production, and continuing for about 600ms regardless of conditions. Further analysis will investigate the duration of suppression after the offset of labials and will be conducted on more participants.

Online English Teaching Quality Assessment Based on K-Means and Improved SSD Algorithm

Classroom teaching quality is a key content to measure the teaching level, and the teaching effect can be intuitively reflected from the students’ listening state. In order to improve the teaching quality, this paper proposes an online English teaching quality evaluation model based on K-means and an improved SSD algorithm. In the SSD algorithm, the backbone network is replaced by DenseNet with a dense connection to improve detection accuracy. The network structure of quadratic regression is designed to solve the problem of unbalance between positive and negative samples in the default box of the candidate region. A feature graph scaling method is used to fuse feature graphs without introducing additional parameters. The number of default boxes and the optimal aspect ratio were obtained by k-means clustering analysis. Finally, the state of students in the teaching process is predicted through the dual-mode recognition model of facial expression and posture, and the state of students in class is judged. Experimental comparison and analysis were conducted on the public data set and a self-built classroom teaching video data set. Experiments show that compared with other comparison algorithms, the algorithm presented in this paper performs better in the index of detection accuracy.

Facial posture as a phonetic prime for memory retrieval

Facial postures influence the perception of speech towards sounds with similar motor configurations [Yeung and Scott, J. Exper. Psych. 150, 983 (2021)]. However, it remains unknown whether facial posture can serve as a phonetic prime for the recall of speech. The present study tests whether maintaining a smile improves the recall of speech sounds with similar kinematics and somatosensory input (e.g., high front vowel /i/) [Ogane etal., J. Acoust. Soc. Am. 148, EL279–EL284 (2020)]. In the training phase, a list of monosyllabic words containing /i/, /a/, and /u/ vowels is presented aurally to participants maintaining a neutral expression. In the recall phase, participants are asked to recall these words in one of two conditions, maintaining either a smile or a neutral expression. Improved recall for tokens containing /i/ would suggest facial postures act as phonetic primes for memory retrieval. The experiment is ongoing; results will be presented and discussed. [Work by the NIH and NSERC].

MEDICAL EDUCATOR'S NONVERBAL BEHAVIOR AND ITS IMPACT ON STUDENTS AND EDUCATIONAL ENVIRONMENT

BACKGROUND &amp; OBJECTIVE: Nonverbal communication has been overlooked in medical education. The purpose of this study was to explore student-teacher perceptions regarding impact of nonverbal behavior of medical educators on educational environment.&#x0D; METHODOLOGY: A qualitative exploratory study was conducted among purposively chosen final year students (n=150) and medical teachers (n=73) from three dental colleges of Islamabad. A web-based survey concerning three domains of kinesics (Facial expressions, Gestures, Head movement and postures) was developed and validated before execution. Comparison of participant’s responses to closed-ended questions was done by applying Fisher’s Exact Test and open-ended questions were thematically analyzed.&#x0D; RESULTS: Response rate of students and teachers was 46% and 63% respectively. Five out of nine closed-ended questions gave statistically significant difference among student’s and teacher’s responses (p≤0.05). Twenty-seven subthemes emerged from three domains of kinesics. Students revealed that happy mood of their teachers reduces boredom in the classroom and a teacher’s smile lets them freely express anything, whereas anger produces fear among them. According to the educators a happy instructor inspires students to study and makes the environment friendly and an undemanding smile helps build a rapport with students. Moreover, positive gestures used by teachers encourage students to clarify concepts and upright posture brings energy and motivation to the classroom.&#x0D; CONCLUSION: A teacher’s nonverbal behavior has a profound effect on educational environment. A smile on a teacher’s face creates a friendly and fearless environment whereas anger creates barriers to communication. A happy teacher can generate a pleasant and motivating learning environment.

Analysis of Facial Expressions in Brazilian Sign Language (Libras)

Brazilian Sign Language (in Portuguese, Libras) is a visuospatial linguistic system adopted by the Brazilian deaf communities as the primary form of communication. Libras are a language of minority groups, thus their research and production of teaching materials do not receive the same incentive to progress or improve as oral languages. This complex language employs signs composed of forms and hands movements combined with facial expressions and postures of the body. Facial expressions rarely appear in sign language literature, despite their being essential to this form of communication. Thereby, this research objectives are to present and discuss sub-categories of the grammatical facial expressions of Libras, with two specific objectives: (1) the building of an annotated video corpus comprehending all the categories identified in the literature of facial expressions in Brazilian sign language; (2) the application of Facial Action Coding System (FACS) (which has its origins as an experimental model in psychology) as a tool for annotating facial expressions in sign language. Ruled by a qualitative approach, the video corpus was carried out with nineteen Libras users (sixteen deaf and three hearing participants) who translated forty- three phrases from Portuguese to Libras. The records were later transcribed with the Eudico Linguistic Annotator software tool. From the analysis of the literature review, it was observed the need to classify facial expression as subcategories of lexical, as intensity, homonyms, and norm. It is believed that it is necessary to expand the studies on facial expressions, favoring their documentation and the description of their linguistic functions. Advances in this sense can contribute to the learning of Libras by deaf students and also by listeners who propose to act as teachers or as translators and interpreters of this language system.

Navigating from live to virtual social interactions: looking at but not manipulating smartphones provokes a spontaneous mimicry response in the observers

By gathering data on people during their ordinary daily activities, we tested if looking at, but not manipulating, smartphones led to a mimicry response in the observer. Manipulating and looking at the device (experimental condition), more than its mere manipulation (control condition), was critical to elicit a mimicry response in the observer. Sex, age and relationship quality between the experimenter and the observer had no effect on the smartphone mimicry response that tended to decrease during social meals. Due to the role of food as a tool in increasing social affiliation, it is possible that during communal eating, people engage in other forms of mimicry involving facial expressions and postures rather than the use of objects. Understanding the ethological mechanisms of the use of smartphones at everyday-social scale could unveil the processes at the basis of the widespread/increasing use of these devices at a large scale.

Hybrid Attention Cascade Network for Facial Expression Recognition.

As a sub-challenge of EmotiW (the Emotion Recognition in the Wild challenge), how to improve performance on the AFEW (Acted Facial Expressions in the wild) dataset is a popular benchmark for emotion recognition tasks with various constraints, including uneven illumination, head deflection, and facial posture. In this paper, we propose a convenient facial expression recognition cascade network comprising spatial feature extraction, hybrid attention, and temporal feature extraction. First, in a video sequence, faces in each frame are detected, and the corresponding face ROI (range of interest) is extracted to obtain the face images. Then, the face images in each frame are aligned based on the position information of the facial feature points in the images. Second, the aligned face images are input to the residual neural network to extract the spatial features of facial expressions corresponding to the face images. The spatial features are input to the hybrid attention module to obtain the fusion features of facial expressions. Finally, the fusion features are input in the gate control loop unit to extract the temporal features of facial expressions. The temporal features are input to the fully connected layer to classify and recognize facial expressions. Experiments using the CK+ (the extended Cohn Kanade), Oulu-CASIA (Institute of Automation, Chinese Academy of Sciences) and AFEW datasets obtained recognition accuracy rates of 98.46%, 87.31%, and 53.44%, respectively. This demonstrated that the proposed method achieves not only competitive performance comparable to state-of-the-art methods but also greater than 2% performance improvement on the AFEW dataset, proving the significant outperformance of facial expression recognition in the natural environment.

Non-Verbal Communication by Autistic Children

This thesis entitled ‘Non-Verbal Communication by Autistic Children’ was conducted with the aim to find out the component s of kinesics as a part of nonverbal communication expressed by autistic children when interacting with their teachers at S LB ABC Taman Pendidikan Islam. Beside that this study also intended to know the meanings of those kinesics component s expressed by autistic children when interacting with their teachers. This research used a descriptive qualitative approach by applying kinesics theory by Ray Birdwhistell. The data collection techniques were conducted by using non-participant observation techniques and documentation. The documents used to support this study were in the form of photographs and video recorder that showed the communication done by the autistic children with their teachers during the learning process in the class. The results showed that although autistic children cannot interact clearly through verbal communication, they can express their desires through non verbal communication. They can communicate by expressing it through facial expressions, eye contact, body movement , posture and touch. T he meaning of the non-verbal communication expressed by each autistic child has a different meaning. This study also intended to give worthy contribution to all people who want to understand the non-verbal communication of autistic children.

Errors in identifying emotion in body postures and facial expressions among pediatric patients with bipolar disorder

ABSTRACT Introduction Facial emotion processing impairments have been consistently demonstrated among patients with pediatric bipolar disorder (PBD). However, less is known about other domains of nonverbal emotion recognition in this group. Therefore, the aim of the current study was to investigate emotion identification for body postures in addition to facial expressions among PBD patients and matched healthy comparison (HC) subjects. Method Youth with PBD (n = 25) and HC subjects (n = 25), equal on intellectual and demographic variables, completed the body postures and faces subtests of the Diagnostic Analysis of Nonverbal Accuracy-2 scale (DANVA-2) along with diagnostic and clinical assessments. Results Compared to HC, PBD patients made significantly more errors identifying emotion in both body postures and faces. Poorer performance on the faces subtest was associated with more problematic peer functioning. Greater manic symptoms were associated with poorer performance on negative emotional stimuli but not positive. Conclusions In addition to impairments in facial emotion processing, youth with PBD also show impairments in the ability to correctly identify emotion from body postures. In this group, greater manic symptomatology appears to impair processing for negative facial expressions and body postures, but not positive ones. Interestingly, facial emotion processing impairments appear to be a better indicator of real-world psychosocial difficulties among PBD patients compared to body postures. Psychosocial treatments for youth with PBD should focus on improving the ability to accurately identify and respond to various facial and body postural cues, as well as symptom management and emotion regulation strategies.

Multi-Pose Face Recognition Based on Deep Learning in Unconstrained Scene

At present, deep learning drives the rapid development of face recognition. However, in the unconstrained scenario, the change of facial posture has a great impact on face recognition. Moreover, the current model still has some shortcomings in accuracy and robustness. The existing research has formulated two methods to solve the above problems. One method is to model and train each pose separately. Then, a fusion decision will be made. The other method is to make “frontal” faces on the image or feature level and transform them into “frontal” face recognition. Based on the second idea, we propose a profile to the frontal revise mapping (PTFRM) module. This module realizes the revision of arbitrary poses on the feature level and transforms the multi-pose features into an approximate frontal representation to enhance the recognition ability of the existing recognition models. Finally, we evaluate the PTFRM on unconstrained face validation benchmark datasets such as Labeled Faces in the Wild (LFW), Celebrities in Frontal Profile (CFP), and IARPA Janus Benchmark A(IJB-A). Results show that the chosen method for this study achieves good performance.

The 3D Emotion Recognition Using SVM and HoG Features

Emotion recognition is becoming commercially popular due to the major role of analytics in various aspects of marketing and strategy management. Several papers have been proposed in emotion recognition. They are mainly classified in the past under 2D and 3D emotion recognition, out of which 2D emotion recognition has been more popular. Various aspects like facial posture, light intensity variations and sensor-independent recognition have been studied by different authors in the past. However, in reality, 3D emotion recognition has been found to be more efficient which has a broader area of use. In this paper, a 3D tracking plane with 2D feature points has enabled us to recognize emotions by statistical voting method from all planes having over threshold number of points in their respective contour area. The proposed technique’s results are comparable to existing methods in terms of time, space complexity and accuracy improvement.

Efficient recognition of facial expressions does not require motor simulation.

What mechanisms underlie facial expression recognition? A popular hypothesis holds that efficient facial expression recognition cannot be achieved by visual analysis alone but additionally requires a mechanism of motor simulation - an unconscious, covert imitation of the observed facial postures and movements. Here, we first discuss why this hypothesis does not necessarily follow from extant empirical evidence. Next, we report experimental evidence against the central premise of this view: we demonstrate that individuals can achieve normotypical efficient facial expression recognition despite a congenital absence of relevant facial motor representations and, therefore, unaided by motor simulation. This underscores the need to reconsider the role of motor simulation in facial expression recognition.