Dynamic Expression Recognition Research Articles

Dynamic facial expression recognition of sprinters based on multi-scale detail enhancement

In order to solve the problems of low average gradient and long recognition time in traditional facial expression recognition method, a multi-scale detail enhancement method for facial dynamic expression recognition of sprint athletes is proposed. A principal component analysis method was used to establish the facial expression feature subspace of sprinters, to project and reduce the dimension of the facial dynamic expression feature vector of sprinters, and to obtain the low frequency information and high frequency information of the facial image of sprinters by bilateral filtering. The multi-scale details of expression are enhanced by using side suppression network model and improving image S curve. The feature vector of facial dynamic expression is input into support vector machine to recognise the facial dynamic expression of sprinter. Experimental results show that the average value of annoying gradient is about 98 and the shortest time s. is about 1.9.

Shape-invariant encoding of dynamic primate facial expressions in human perception.

Dynamic facial expressions are crucial for communication in primates. Due to the difficulty to control shape and dynamics of facial expressions across species, it is unknown how species-specific facial expressions are perceptually encoded and interact with the representation of facial shape. While popular neural network models predict a joint encoding of facial shape and dynamics, the neuromuscular control of faces evolved more slowly than facial shape, suggesting a separate encoding. To investigate these alternative hypotheses, we developed photo-realistic human and monkey heads that were animated with motion capture data from monkeys and humans. Exact control of expression dynamics was accomplished by a Bayesian machine-learning technique. Consistent with our hypothesis, we found that human observers learned cross-species expressions very quickly, where face dynamics was represented largely independently of facial shape. This result supports the co-evolution of the visual processing and motor control of facial expressions, while it challenges appearance-based neural network theories of dynamic expression recognition.

Dense and Sparse 3D Deformation Signatures for 3D Dynamic Face Recognition

This work analyses dense and sparse 3D Deformation Signatures to represent 3D temporal deformation instances. The signatures are employed in dynamic 3D face recognition, however, they are applicable in other domains. This is demonstrated for dynamic expression recognition. The pushing need for non-intrusive bio-metric measurements made face and its expressions recognition dominant players in domains like entertainment, surveillance and security. The proposed signature can be computed from 2D, 3D or hybrid input by means of robust 3D fitting. It is computed given a non-linear 6D space representation which guarantees by construction physically plausible 3D deformations. A unique deformation indicator is computed per triangle in a triangulated mesh as a ratio derived from scale and in-plane deformation in the canonical space. These indicators are concatenated densely or sparsely to form the signature. It is then used to learn the 3D deformation space from the temporal facial signals. Two dynamic datasets were examined for evaluation. The reported 1-Rank recognition accuracy outperforms the existing literature. Democratising the recognition step results in 100% accuracy as demonstrated by the reported confusion matrices. In an open-world setting in the face recognition context, an accuracy of 100% was achieved in detecting intruders. The signature robustness has been further validated in face expressions recognition from a very challenging highly 3D dynamic dataset.

Multi-attention based Deep Neural Network with hybrid features for Dynamic Sequential Facial Expression Recognition

In interpersonal communication, the expression is an import way to express one’s emotions. In order to make computers understand facial expressions like human beings, a large number of researchers have put a lot of time and energy into it. But for now, most of the work of dynamic sequence facial expression recognition fails to make full use of the combined advantages of shallow features (prior knowledge) and depth features (high-level semantic). Therefore, this paper implements a dynamic sequence facial expression recognition system that integrates shallow features and deep features with the attention mechanism. In order to extract the shallow features, an Attention Shallow Model (ASModel) is proposed by using the relative position of facial landmarks and the texture characteristics of the local area of the face to describe the Action Units of the Facial Action Coding System. And with the advantage of the deep convolutional neural network in expressing high-level features, a Attention Deep Model (ADModel) is also designed to extract deep features on sequence facial images. Finally, the ASModel and the ADModel are integrated to a Multi-attention Shallow and Deep Model (MSDModel) to complete the dynamic sequence facial expression recognition. There are three kinds of attention mechanism introduced, such as Self-Attention (SA), Weight-Attention (WA), and Convolution-Attention (CA). We verify our dynamic expression recognition system on three publicly available databases include CK+, MMI, and Oulu-CASIA and get superior performance than other state-of-art results.

Dynamic Facial Expression Recognition Based on ResNet and LSTM

The face image data collected by ordinary face recognition system is usually static. In real life, to fully capture the state information of human emotion change, we need to collect dynamic face expression change data, which needs to capture a video sequence to reflect the process of emotion change. In this paper, we combine ResNet residual neural network and long-term memory network (LSTM) to extract features from video sequences and then recognize them. In the residual neural network, the residual element has a very good effect on the training and optimization process of deep CNN model, and shortens the convergence time of the model. Combined with the LSTM network, the captured dynamic sequence data is incorporated into the CNN model. Compared with the static single frame expression recognition system, the research of dynamic expression recognition has more application value.

Dynamic Expression Recognition Based on Hybrid Features and Optimized Extreme Learning Machine Model

Facial expression recognition is an academic subject, and it can be applied not only in the field of computer vision but also in psychology and sociology. This paper proposed a robust dynamic facial expression recognition method which using hybrid features and the PSOELM (Particle Swarm Optimization-Extreme Learning Machine) model. At first, six PSOELM models were trained on the training set of six basic expressions, respectively. Then by extracting the hybrid features which fusing geometric features and texture features, the dynamic expressions were recognized by PSO-ELM models. These experiments were performed on the Cohn-Kanade+ face database. The results show that this method has a better recognition effect on dynamic facial expression recognition.

Learning Expressionlets via Universal Manifold Model for Dynamic Facial Expression Recognition.

Facial expression is a temporally dynamic event which can be decomposed into a set of muscle motions occurring in different facial regions over various time intervals. For dynamic expression recognition, two key issues, temporal alignment and semantics-aware dynamic representation, must be taken into account. In this paper, we attempt to solve both problems via manifold modeling of videos based on a novel mid-level representation, i.e., expressionlet. Specifically, our method contains three key stages: 1) each expression video clip is characterized as a spatial-temporal manifold (STM) formed by dense low-level features; 2) a universal manifold model (UMM) is learned over all low-level features and represented as a set of local modes to statistically unify all the STMs; and 3) the local modes on each STM can be instantiated by fitting to the UMM, and the corresponding expressionlet is constructed by modeling the variations in each local mode. With the above strategy, expression videos are naturally aligned both spatially and temporally. To enhance the discriminative power, the expressionlet-based STM representation is further processed with discriminant embedding. Our method is evaluated on four public expression databases, CK+, MMI, Oulu-CASIA, and FERA. In all cases, our method outperforms the known state of the art by a large margin.

Individual differences in antisocial and prosocial traits predict perception of dynamic expression

Successful everyday, social interactions are mediated by the accurate perception of dynamic facial expressions. For example, the decoding of distress cues have been shown to inhibit antisocial behaviour while simultaneously eliciting empathetic responses. The import of this recognition-behaviour connection has been revealed by psychiatric research demonstrating that disorders characterized by interpersonal deficits are associated with impairments in processing facial affect. This area of research has generated speculation that the neurocognitive mechanisms specific to social behaviour are also pivotally involved in expression recognition, but it is unknown whether this relationship also holds in nonclinical populations. To explore this question, we examined how individual differences in antisocial traits relate to the recognition of facial affect. Antisocial behaviour was assessed using the Inventory for Callous-Unemotional Traits (ICU, Kimonis et al., 2008), a scale which designates a subgroup of antisocial individuals who are more likely to show deficits in processing emotional stimuli relative to other antisocial individuals. Perceptual sensitivity to facial affect was probed using a Dynamic Expression Recognition Task (DERT, Deriso et al., 2012) in which participants were shown 75ms, 150ms, or 225ms reveals of a dynamic face morph progressing from neutral to one of sad, happy, angry, fear, surprise, or disgust. The main finding was that participants who scored higher in callous-unemotional traits were significantly less accurate in expression recognition compared to participants who scored low on the ICU. This result supports the hypothesis for a common mechanism underlying affect recognition and antisocial behaviour across clinical and nonclinical groups, while also highlighting the diagnostic potential of facial affect processing. Meeting abstract presented at VSS 2015. Language: en

Towards a dynamic expression recognition system under facial occlusion

Facial occlusion is a challenging research topic in facial expression recognition (FER). This has resulted in the need to develop some interesting facial representations and occlusion detection methods in order to extend the FER to uncontrolled environments. It should be noted that most of the previous work focuses on these two issues separately, and on static images. We are thus motivated to propose a complete system consisting of facial representations, occlusion detection, and multiple feature fusion in video sequences. For achieving a robust facial representation, we propose an approach deriving six feature vectors from eyes, nose and mouth components to form a facial representation. These features with temporal cues are generated by the dynamic texture and structural shape feature descriptors. On the other hand, occlusion detection is still mainly realized by the traditional classifiers or model comparison. Recently, sparse representation has been proposed as an efficient method against occlusion, while it is correlated with facial identity in FER, unless using an appropriate facial representation. Thus, we present an evaluation demonstrating that the proposed facial representation is independent of facial identity. Inspired by Mercier et al. (2007), we then exploit the use of the sparse representation and residual statistics to occlusion detection of the image sequences. As concatenating six feature vectors into one causes the curse of dimensionality, we propose multiple feature fusion consisting of fusion module and weight learning. Experimental results on the Extended Cohn–Kanade database and its simulated database demonstrate that our framework outperforms the state-of-the-art methods for FER in normal videos, and especially, in partial occlusion videos.