Global Facial Features Research Articles

Improved Facial Expression Recognition Algorithm Based on Local Feature Enhancement and Global Information Association

Facial expression recognition is the key area of research in computer vision, enabling intelligent devices to understand human emotions and intentions. However, recognition of facial expressions in natural scenes presents challenges due to environmental factors like occlusion and pose variations. To address this, we propose a novel approach that combines local feature enhancement and global information correlation. This method allows the model to learn both local and global facial features along with contextual information. By enhancing salient local features and exploring multi-scale facial expression features, our model effectively mitigates the impact of occlusion and pose variations, improving recognition accuracy. Experimental results demonstrate that our adapted model outperforms alternative algorithms in recognizing facial expressions under challenging environments, achieving recognition accuracies of 85.07% and 99.35% on the RAF-DB and CK+ datasets, respectively.

FrseGAN: Free‐style editable facial makeup transfer based on GAN combined with transformer

AbstractMakeup in real life varies widely and is personalized, presenting a key challenge in makeup transfer. Most previous makeup transfer techniques divide the face into distinct regions for color transfer, frequently neglecting details like eyeshadow and facial contours. Given the successful advancements of Transformers in various visual tasks, we believe that this technology holds large potential in addressing pose, expression, and occlusion differences. To explore this, we propose novel pipeline which combines well‐designed Convolutional Neural Network with Transformer to leverage the advantages of both networks for high‐quality facial makeup transfer. This enables hierarchical extraction of both local and global facial features, facilitating the encoding of facial attributes into pyramid feature maps. Furthermore, a Low‐Frequency Information Fusion Module is proposed to address the problem of large pose and expression variations which exist between the source and reference faces by extracting makeup features from the reference and adapting them to the source. Experiments demonstrate that our method produces makeup faces that are visually more detailed and realistic, yielding superior results.

Local Eyebrow Feature Attention Network for Masked Face Recognition

During the COVID-19 coronavirus epidemic, wearing masks has become increasingly popular. Traditional occlusion face recognition algorithms are almost ineffective for such heavy mask occlusion. Therefore, it is urgent to improve the recognition performance of the existing face recognition technology on masked faces. Due to the limited visible feature points of the masked face image relative to the normal face image, we have to exploit the identification potential of eyebrow (referring to eyes and brows) features. This article proposes a local eyebrow feature attention network for masked face recognition, which consists of feature extraction, eyebrow region pooling, and feature fusion. To highlight the eyebrow region, we first use the eyebrow region pooling to separate the local features of eyebrows from the learned overall facial features. We then make full use of the symmetry of left and right eyebrows to emphasize their discriminant ability, due to the inadequate fine information of the low-resolution eyebrows. In particular, in view of the symmetrical similarity between eyebrow pairs and the subordinate relationship between facial components and the whole, we propose a feature fusion model based on graph convolutional network (GCN) to learn the feature association structure of eye features, brow features, and global facial features. We construct the benchmark datasets for masked face recognition to validate our approach, including real-world masked face recognition dataset (RMFRD) and synthetic masked face recognition dataset (SMFRD). Extensive experimental results on both public datasets and our built masked face datasets show that our approach significantly outperforms the state-of-the-arts.

SILP-autoencoder for face de-occlusion

Recognizing faces with partial occlusion is a challenging problem in many real-world applications. Although various methods have been proposed to deal with the facial image de-occlusion tasks, most of them only concern the local features of occluded images, obviously ignoring the global facial expressions and structural prior information. In this paper, we propose a novel end-to-end SILP-Autoencoder to effectively restore partial occluded faces. To improve the recovery quality and occlusion removal robustness, our framework mainly consists of two components, Laplacian prior subnetwork, and left-and-right symmetric match module (LR-match module), which preserve the global facial expression features and fully make use of the symmetrical characteristics of facial regions and structures respectively. Based on the above characteristics, a composite loss function is designed to achieve end-to-end training of the entire network. Extensive experiments on the face expression datasets with various shaded areas suggest that our approach achieves superior performance against the state-of-the-art methods. In particular, our method is more useful for facial detail recovery and distortion expression suppression.

Localization and classification of human facial emotions using local intensity order pattern and shape-based texture features

Facial emotion recognition system (FERS) recognize the person’s emotions based on various image processing stages including feature extraction as one of the major processing steps. In this study, we presented a hybrid approach for recognizing facial expressions by performing the feature level fusion of a local and a global feature descriptor that is classified by a support vector machine (SVM) classifier. Histogram of oriented gradients (HoG) is selected for the extraction of global facial features and local intensity order pattern (LIOP) to extract the local features. As HoG is a shape-based descriptor, with the help of edge information, it can extract the deformations caused in facial muscles due to changing emotions. On the contrary, LIOP works based on the information of pixels intensity order and is invariant to change in image viewpoint, illumination conditions, JPEG compression, and image blurring as well. Thus both the descriptors proved useful to recognize the emotions effectively in the images captured in both constrained and realistic scenarios. The performance of the proposed model is evaluated based on the lab-constrained datasets including CK+, TFEID, JAFFE as well as on realistic datasets including SFEW, RaF, and FER-2013 dataset. The optimal recognition accuracy of 99.8%, 98.2%, 93.5%, 78.1%, 63.0%, 56.0% achieved respectively for CK+, JAFFE, TFEID, RaF, FER-2013 and SFEW datasets respectively.

LGAttNet: Automatic micro-expression detection using dual-stream local and global attentions

Research in the field of micro-expressions has gained significance in recent years. Many researchers have concentrated on classifying micro-expressions in different discrete emotion classes, while detecting the presence of micro-expression in the video frames is considered as a pre-requisite step in the recognition process. Hence, there is a need to introduce more advanced detection models for micro-expressions. In order to address this, we propose a dual attention network based micro-expression detection architecture called LGAttNet. LGAttNet is one of the first to utilize a dual attention network grouped with 2-dimensional convolutional neural network to perform frame-wise automatic micro-expression detection. This method divides the feature extraction and enhancement task into two different convolutional neural network modules; sparse module and feature enhancement module. One of the key modules in our approach is the attention network which extracts local and global facial features, namely local attention module and global attention module. The attention mechanism adopts the human characteristic of focusing on the specific regions of micro-movements, which enables the LGAttNet to concentrate on particular facial regions along with the full facial features to identify the micro-expressions in the frames. Experiments performed on widely used publicly available databases demonstrate the robustness and superiority of our LGAttNet when compared to state-of-the-art approaches.

Face and occlusion Recognition Algorithm based on Global and Local

Nowadays, the face recognition direction of unobstructed has been greatly developed, but there is still a huge deficiency in face recognition with occlusion. In order to improve the recognition rate under occlusion, this paper proposes a global and local occlusion recognition algorithm using deep learning for feature learning. Due to occlusion, local significant features will be missing, the impact will cause the accuracy of global recognition to decrease. Therefore, SIFT and SVM algorithms are used to discriminate partial face occlusion, and cut off the occluded local face to form a global face. Global facial features and unmasked local feature are extracted together. In the feature extraction, the LeNet-5 lightweight convolutional neural network and the improved LeNet-5 lightweight convolutional neural network are combined with SoftMax multi-classification. The final classification results use the improved hierarchical voting method to make final decisions on the face, which speeds up the training and prediction speed of the multi-classifier. This experiment will perform occlusion face verification on the AR dataset. The final experimental results show that the method greatly improves the accuracy of occlusion face recognition and has strong robustness to occlusion.

What Is a Face? Critical Features for Face Detection.

Faces convey very rich information that is critical for intact social interaction. To extract this information efficiently, faces should be easily detected from a complex visual scene. Here, we asked which features are critical for face detection. To answer this question, we presented non-face objects that generate a strong percept of a face (i.e., Pareidolia). One group of participants rated the faceness of this set of inanimate images. A second group rated the presence of a set of 12 local and global facial features. Regression analysis revealed that only the eyes or mouth significantly contributed to faceness scores. We further showed that removing eyes or mouth, but not teeth or ears, significantly reduced faceness scores. These findings show that face detection depends on specific facial features, the eyes and the mouth. This minimal information leads to over-generalization that generates false face percepts but assures that real faces are not missed.

NIRExpNet: Three-Stream 3D Convolutional Neural Network for Near Infrared Facial Expression Recognition

Facial expression recognition (FER) under active near-infrared (NIR) illumination has the advantages of illumination invariance. In this paper, we propose a three-stream 3D convolutional neural network, named as NIRExpNet for NIR FER. The 3D structure of NIRExpNet makes it possible to extract automatically, not just spatial features, but also, temporal features. The design of multiple streams of the NIRExpNet enables it to fuse local and global facial expression features. To avoid over-fitting, the NIRExpNet has a moderate size to suit the Oulu-CASIA NIR facial expression database that is a medium-size database. Experimental results show that the proposed NIRExpNet outperforms some previous state-of-art methods, such as Histogram of Oriented Gradient to 3D (HOG 3D), Local binary patterns from three orthogonal planes (LBP-TOP), deep temporal appearance-geometry network (DTAGN), and adapt 3D Convolutional Neural Networks (3D CNN DAP).

Face alignment using a deep neural network with local feature learning and recurrent regression

Abstract We propose a face alignment method that uses a deep neural network employing both local feature learning and recurrent regression. This method is primarily based on a convolutional neural network(CNN), which automatically learns local feature descriptors from the local facial landmark dataset that we created. Our research is motivated by the belief that investigating a face from its low-level component features would produce more competitive face alignment results, just as a CNN is normally trained to automatically learn a feature hierarchy from the lowest to the highest levels of abstraction. Moreover, by separately training the feature extraction layers and the regression layers, we impose an explicit functional discrimination between the feature extraction and regression tasks. First, we train a feature extraction network that is used to classify the landmark patches in the dataset. Using this pre-trained feature extraction network, we build a face alignment network, which uses an entire face image rather than the local landmark patch as input, thus generating the global facial features. The subsequent local feature extraction layer extracts the local feature set from this global feature, finally generating the local feature descriptors, in which space the network learns a generic descent direction from the currently estimated landmark positions to the ground truth via linear regression applied recurrently. Head pose estimation network also applied to provide a good initial estimate to the local feature extraction layer for accurate convergence. We found that learning of the good local landmark features in pursuit of good landmark classification also leads to a higher face alignment accuracy and achieves state-of-the-art performance on several public benchmark dataset. It signifies the importance of learning not only the global features but the local features for face alignment. We further verify our method's effectiveness when applied to related problems such as head pose estimation, facial landmark tracking, and invisible landmark detection. We believe that good local learning enables a deeper understanding of the face or object resulting in higher performance.

An approach for facial expression classification

In this paper, a new method for facial expression classification is proposed, which uses the histograms of oriented gradients (HOG) algorithm to extract facial expression features and the sparse representation classifier (SRC) to classify facial expressions with a large variation of poses. The HOG algorithm was selected due to its effectiveness in picking up both local and global facial expression features in different orientations and scales, and the SRC was chosen because of its proven effectiveness in face recognition. A novelty of the proposed approach is that given a facial image for classification, its pose is determined first to select a pose-dependent dictionary for the SRC procedure. The paper also discusses ways of selecting parameters for improving the effectiveness of the HOG algorithm. The proposed method was applied to two multi-pose facial expression databases and satisfactory results were obtained for the majority of facial expressions under various poses.

An approach for facial expression classification

In this paper, a new method for facial expression classification is proposed, which uses the histograms of oriented gradients (HOG) algorithm to extract facial expression features and the sparse representation classifier (SRC) to classify facial expressions with a large variation of poses. The HOG algorithm was selected due to its effectiveness in picking up both local and global facial expression features in different orientations and scales, and the SRC was chosen because of its proven effectiveness in face recognition. A novelty of the proposed approach is that given a facial image for classification, its pose is determined first to select a pose-dependent dictionary for the SRC procedure. The paper also discusses ways of selecting parameters for improving the effectiveness of the HOG algorithm. The proposed method was applied to two multi-pose facial expression databases and satisfactory results were obtained for the majority of facial expressions under various poses.

Eye centre localisation: an unsupervised modular approach

Purpose This paper aims to introduce an unsupervised modular approach for eye centre localisation in images and videos following a coarse-to-fine, global-to-regional scheme. The design of the algorithm aims at excellent accuracy, robustness and real-time performance for use in real-world applications. Design/methodology/approach A modular approach has been designed that makes use of isophote and gradient features to estimate eye centre locations. This approach embraces two main modalities that progressively reduce global facial features to local levels for more precise inspections. A novel selective oriented gradient (SOG) filter has been specifically designed to remove strong gradients from eyebrows, eye corners and self-shadows, which sabotage most eye centre localisation methods. The proposed algorithm, tested on the BioID database, has shown superior accuracy. Findings The eye centre localisation algorithm has been compared with 11 other methods on the BioID database and six other methods on the GI4E database. The proposed algorithm has outperformed all the other algorithms in comparison in terms of localisation accuracy while exhibiting excellent real-time performance. This method is also inherently robust against head poses, partial eye occlusions and shadows. Originality/value The eye centre localisation method uses two mutually complementary modalities as a novel, fast, accurate and robust approach. In addition, other than assisting eye centre localisation, the SOG filter is able to resolve general tasks regarding the detection of curved shapes. From an applied point of view, the proposed method has great potentials in benefiting a wide range of real-world human-computer interaction (HCI) applications.

MG-106 Global developmental delay and characteristic facial features associated with pacs1 gene mutation – report of two cases

Intellectual disability (ID) affects 1%–3% of the population. While it has a strong genetic component, finding a genetic diagnosis remains challenging. Given the high rate of de novo events in ID, family-based sequencing may be an important tool. In 2012 Schuurs-Hoeijmakers et al ., reported two children with ID and characteristic features associated with a heterozygote mutation in PACS1. We report the same mutation elucidated by whole exome sequencing (WES) in two additional children. Patient 1 was born at term to a 30-year-old primigravida from Bangladesh. Her birth weight and length were 3–10th% and head circumference 50–75th%. ID presented in the first year of life. She had sparse hair, a high forehead, frontal bossing, hypertelorism, deep-set eyes, a broad nasal root, full lips and a wide mouth. She had marked hypotonia, decreased muscle bulk and hyperextensible joints. WES revealed a PACS1 mutation. (NM_018026)exon4:c. C607T:p. R203W. Patient 2 was born at term to a 34-year-old primigravida from China. Birth weight and length were 10–25th% and head circumference 25–50th%. At birth, an anoplasty was performed for an ectopic anus. A right duplex kidney and undescended testes were identified. ID presented before age one. He had a short forehead, bushy eyebrows, short nose, large mouth, uplifting earlobes, bilateral single palmar creases, widely spaced nipples and an umbilical hernia. WES revealed the same mutation. Our two cases highlight the clinical utility of WES in helping establish a diagnosis of an unfamiliar clinical syndrome and supports the discovery of a now recognisable syndrome due to mutation in PACS1.

Facial Age Estimation based on Decision Level Fusion of AAM, LBP and Gabor Features

In this paper a new hierarchical age estimation method based on decision level fusion of global and local features is proposed. The shape and appearance information of human faces which are extracted with active appearance models (AAM) are used as global facial features. The local facial features are the wrinkle features extracted with Gabor filters and skin features extracted with local binary patterns (LBP). Then feature classification is performed using a hierarchical classifier which is the combination of an age group classification and detailed age estimation. In the age group classification phase, three distinct support vector machines (SVM) classifiers are trained using each feature vector. Then decision level fusion is performed to combine the results of these classifiers. The detailed age of the classified image is then estimated in that age group, using the aging functions modeled with global and local features, separately. Aging functions are modeled with multiple linear regression. To make a final decision, the results of these aging functions are also fused in decision level. Experimental results on the FG-NET and PAL aging databases have shown that the age estimation accuracy of the proposed method is better than the previous methods.

Age Classification Based on Feature Fusion

Age classification based on facial images is attracting wide attention with its broad application to human-computer interaction (HCI). Since human senescence is a tremendously complex process, age classification is still a highly challenging issue. In our study, Local Directional Pattern (LDP) and Gabor wavelet transform were used to extract global and local facial features, respectively, that were fused based on information fusion theory. The Principal Component Analysis (PCA) method was used for dimensionality reduction of the fused features, to obtain a lower-dimensional age characteristic vector. A Support Vector Machine (SVM) multi-class classifier with Error Correcting Output Codes (ECOC) was proposed in the paper. This was aimed at multi-class classification problems, such as age classification. Experiments on a public FG-NET age database proved the efficiency of our method.

A kind of Face Recognition Method Based on CCA Feature Information Fusion

In order to achieve more local facial feature, a kind of sub image face recognition method based on RS-Sp CCA feature information fusion is proposed in this paper. According to take samples for the local facial feature of sub image and use CCA to fuse the global facial feature and the local facial feature information after sampling, the global feature of image can be fully used to construct much more different kinds of component classifiers. Then, make experimental analysis on databases of 3 standard facial data sets. At last, the results show that sub image face recognition method based on RS-Sp CCA feature information fusion is better than the simple feature sampling method and feature fusion method. Besides, it can efficiently improve the face recognition rate.

Robust gender classification using a precise patch histogram

This study proposed a precise facial feature extraction method to improve the accuracy of gender classification under pose and illumination variations. We used the active appearance model (AAM) to align the face image. Images were modeled by the patches around the coordinates of certain landmarks. Using the proposed precise patch histogram (PPH) enabled us to improve the accuracy of the global facial features. The system is composed of three phases. In the training phase, non-parametric statistics were used to describe the characteristics of the training images and to construct the patch library. In the inference phase, the choice of feature patch from the library needed to approximate the patch of the testing image was based on the maximum a posteriori estimation. In the estimation phase, a Bayesian framework with portion-oriented posteriori fine-tuning was employed to determine the classification decision. In addition, we developed the dynamic weight adaptation to obtain a more convincing performance. The experimental results demonstrated the robustness of the proposed method.

Integration of Global and Local Feature for Face Recognition

This paper proposes to combine the global and local facial features in both serial and parallel manner.Firstly,global features are used for coarse classification.Then,global and local features are integrated for fine classification.In the proposed method,global and local features are extracted by Discrete Fourier Transform(DFT) and Gabor Wavelets Transform(GWT) respectively.Experiments on two large scale face databases(FERET and FRGC v2.0) validate that the proposed method can not only greatly increase the system accuracy but also improve the system speed.

A 15q24 microduplication, reciprocal to the recently described 15q24 microdeletion, in a boy sharing clinical features with 15q24 microdeletion syndrome patients

A 15q24 microduplication, reciprocal to the minimal critical region for the recently described 15q24 microdeletion syndrome, was found in a 2-year-old boy by 244k Agilent oligoarray CGH analysis. The boy had global developmental delay and dysmorphic facial features, digital and genital abnormalities. The duplication was inherited from a healthy father, but was considered clinically significant, as the patient shared clinical features with 15q24 microdeletion syndrome patients. To our knowledge this is the first report of a patient with a 15q24 microduplication.