Movement Of Facial Features Research Articles

Exploiting Complementary Dynamic Incoherence for DeepFake Video Detection

Recently, manipulated videos based on DeepFake technology have spread widely on social media, causing concerns about the authenticity of video content and personal privacy protection. Although existing DeepFake detection methods achieve remarkable progress in some specific scenarios, their detection performance usually drops drastically when detecting unseen manipulation methods. Compared with static information such as human face, dynamic information depicting the movements of facial features is more difficult to forge without leaving visual or statistical traces. Hence, in order to achieve better generalization ability, we focus on dynamic information analysis to disclose such traces and propose a novel Complementary Dynamic Interaction Network (CDIN). Inspired by the DeepFake detection methods based on mouth region analysis, both the global (entire face) and local (mouth region) dynamics are analyzed with properly designed network branches, respectively, and their feature maps at various levels are communicated with each other using a newly proposed Complementary Cross Dynamics Fusion Module (CCDFM). With CCDFM, the global branch will pay more attention to anomalous mouth movements and the local branch will gain more information about the global context. Finally, a multi-task learning scheme is designed to optimize the network with both the global and local information. Extensive experiments have demonstrated that our approach achieves better detection results compared with several SOTA methods, especially in detecting video forgeries manipulated by unseen methods.

Optical Flow-Based Algorithm Analysis to Detect Human Emotion from Eye Movement-Image Data

One of the popular methods for the recognition of human emotions such as happiness, sadness and shock is based on the movement of facial features. Motion vectors that show these movements can be calculated by using optical flow algorithms. In this method, for detecting emotions, the resulted set of motion vectors is compared with a standard facial movement template caused by human emotional changes. In this paper, a new method is introduced to compute the quantity of likeness towards a particular emotion to make decisions based on the importance of obtained vectors from an optical flow approach. The current study uses a feature point tracking technique separately applied to the five facial image regions (eyebrows, eyes, and mouth) to identify basic emotions. Primarily, this research will be focusing on eye movement regions. For finding the vectors, one of the efficient optical flow methods is using the pre-experiment as explained further below.

Face Image Animation with Adversarial Learning and Motion Transfer

Significant advances have been made in facial image animation from a single image. Nonetheless, generating convincing facial feature movements remains a complex challenge in computer graphics. The purpose of this study is to develop an efficient and effective approach for transferring motion from a source video to a single facial image by governing the position and expression of the face in the video to generate a new video imitating the source image. Compared to prior methods that focus solely on manipulating facial expressions, this model has been trained to distinguish the moving foreground from the background image and to create motions such as facial rotation and translation as well as small local motions such as gaze shift. The pro-posed technique uses generative adversarial networks GANs with a motion transfer model. The network forecasts photo-realistic video frames for a given target image using synthetic input in renderings from a parametric face model. The authenticity in this postprocessing conversion is attained by precise image manipulation. Thorough adversarial training is used to produce greater accuracy in this postprocessing conver-sion. Although more improvements to face landmark identification on videos and face super-resolution techniques have been made to improve the results, the pro-posed technique can provide more coherent videos with improved visual quality, resulting in more aligned landmark sequences for training. In addition, experiments indicate that we obtain superior results compared to those obtained by the state-of-the-art image-driven technique with PSNR 30.74 and SSIM 0.90

Facial Feature Movements Caused by Various Emotions: Differences According to Sex

Facial muscle micro movements for eight emotions were induced via visual and auditory stimuli and were verified according to sex. Thirty-one main facial features were chosen from the Kinect API out of 121 initially obtained facial features; the average change of pixel value was measured after image alignment. The proposed method is advantageous as it allows for comparisons. Facial micro-expressions are analyzed in real time using 31 facial feature points. The amount of micro-expressions for the various emotion stimuli was comparatively analyzed for differences according to sex. Men’s facial movements were similar for each emotion, whereas women’s facial movements were different for each emotion. The six feature positions were significantly different according to sex; in particular, the inner eyebrow of the right eye had a confidence level of p < 0.01. Consequently, discriminative power showed that men’s ability to separate one emotion from the others was lower compared to women’s ability in terms of facial expression, despite men’s average movements being higher compared to women’s. Additionally, the asymmetric phenomena around the left eye region of women appeared more strongly in cases of positive emotions.

Dynamic Facial Expression Recognition With Atlas Construction and Sparse Representation

In this paper, a new dynamic facial expression recognition method is proposed. Dynamic facial expression recognition is formulated as a longitudinal groupwise registration problem. The main contributions of this method lie in the following aspects: 1) subject-specific facial feature movements of different expressions are described by a diffeomorphic growth model; 2) salient longitudinal facial expression atlas is built for each expression by a sparse groupwise image registration method, which can describe the overall facial feature changes among the whole population and can suppress the bias due to large intersubject facial variations; and 3) both the image appearance information in spatial domain and topological evolution information in temporal domain are used to guide recognition by a sparse representation method. The proposed framework has been extensively evaluated on five databases for different applications: the extended Cohn-Kanade, MMI, FERA, and AFEW databases for dynamic facial expression recognition, and UNBC-McMaster database for spontaneous pain expression monitoring. This framework is also compared with several state-of-the-art dynamic facial expression recognition methods. The experimental results demonstrate that the recognition rates of the new method are consistently higher than other methods under comparison.

Lip Reading Without Awareness

Lip Reading Without Awareness

The contribution of different cues of facial movement to the emotional facial expression adaptation aftereffect

Probing emotional facial expression recognition with the adaptation paradigm is one way to investigate the processes underlying emotional face recognition. Previous research suggests that these processes are tuned to dynamic facial information (facial movement). Here we examined the tuning of processes involved in the recognition of emotional facial expressions to different sources of facial movement information. Specifically we investigated the effect of the availability of rigid head movement and intrinsic facial movements (e.g., movement of facial features) on the size of the emotional facial expression adaptation effect. Using a three-dimensional (3D) morphable model that allowed the manipulation of the availability of each of the two factors (intrinsic facial movement, head movement) individually, we examined emotional facial expression adaptation with happy and disgusted faces. Our results show that intrinsic facial movement is necessary for the emergence of an emotional facial expression adaptation effect with dynamic adaptors. The presence of rigid head motion modulates the emotional facial expression adaptation effect only in the presence of intrinsic facial motion. In a second experiment we show these adaptation effects are difficult to explain by merely the perceived intensity and clarity (uniqueness) of the adaptor expressions. Together these results suggest that processes encoding facial expressions are differently tuned to different sources of facial movements.

Automatically Analyzing Facial-Feature Movements to Identify Human Errors

Using facial feature points automatically extracted from short video segments, researchers couple computer vision with machine learning to predict performance over an entire task and at any given instant within the task.

Expressive Facial Gestures From Motion Capture Data

AbstractHuman facial gestures often exhibit such natural stochastic variations as how often the eyes blink, how often the eyebrows and the nose twitch, and how the head moves while speaking. The stochastic movements of facial features are key ingredients for generating convincing facial expressions. Although such small variations have been simulated using noise functions in many graphics applications, modulating noise functions to match natural variations induced from the affective states and the personality of characters is difficult and not intuitive. We present a technique for generating subtle expressive facial gestures (facial expressions and head motion) semi‐automatically from motion capture data. Our approach is based on Markov random fields that are simulated in two levels. In the lower level, the coordinated movements of facial features are captured, parameterized, and transferred to synthetic faces using basis shapes. The upper level represents independent stochastic behavior of facial features. The experimental results show that our system generates expressive facial gestures synchronized with input speech.

Pose Robust Face Tracking by Combining Active Appearance Models and Cylinder Head Models

The active appearance models (AAMs) provide the detailed descriptive parameters that are useful for various autonomous face analysis problems. However, they are not suitable for robust face tracking across large pose variation for the following reasons. First, they are suitable for tracking the local movements of facial features within a limited pose variation. Second, they use gradient-based optimization techniques for model fitting and the fitting performance is thus very sensitive to initial model parameters. Third, when their fitting is failed, it is difficult to obtain appropriate model parameters to re-initialize them. To alleviate these problems, we propose to combine the active appearance models and the cylinder head models (CHMs), where the global head motion parameters obtained from the CHMs are used as the cues of the AAM parameters for a good fitting or re-initialization. The good AAM parameters for robust face tracking are computed in the following manner. First, we estimate the global motion parameters by the CHM fitting algorithm. Second, we project the previously fitted 2D shape points onto the 3D cylinder surface inversely. Third, we transform the inversely projected shape points by the estimated global motion parameters. Fourth, we project the transformed 3D points onto the input image and computed the AAM parameters from them. Finally, we treat the computed AAM parameters as the initial parameters for the fitting. Experimental results showed that face tracking combining AAMs and CHMs is more pose robust than that of AAMs in terms of 170% higher tracking rate and the 115% wider pose coverage.

Reshaping the binding problem of form and motion vision

The primary visual cortex (or V1) is the first cortical processing stage in primates and receives visual information from the retina via several parallel pathways. Cortical regions beyond V1 are segregated into two major streams: dorsal and ventral. The dorsal route takes information from V1 to the posterior parietal cortex, while the ventral pathway takes a route from V1 to the inferotemporal cortex. The ventral visual stream is tuned for object identification, while the dorsal stream is tuned to detect where objects are located and in which direction they move (Ungerleider & Mishkin, 1982; Ungerleider & Pasternak, 2003). The apparent functional separation between the two paths has created a theoretical challenge. How can the information in the two systems be brought together across cortical space: the visual binding problem (Roskies, 1999). A good example of this binding problem relates to form and motion processing. It is generally believed that motion direction is coded by the dorsal stream and only combined with form processing from the ventral stream at higher cortical levels (e.g. Oram & Perret, 1994). The segregation into form and motion paths begins in V1 where only certain horizontal layers have been shown to have large proportions of direction-selective cells (Hawken et al. 1988). V1 has six layers of neural tissue each being subdivided into functionally distinct sublayers (Fig. 1; Lund, 1988). In primates only layers 4B, 4Cα and 6 have been shown to contain a high proportion of direction-selective cells. Layer 4Cα receives input from the retina via the dorsal lateral geniculate nucleus and transfers signals to layers 4B and 6, which in turn send information to the dorsal stream (Movshon & Newsome, 1996). There has been little evidence of highly direction-selective signals being sent to the ventral pathway. Schematic diagram of the direction-selective (DS) pathways in V1 However, the study by Gur & Snodderly (2007) in this issue of The Journal of Physiology reveals a new pathway for direction-selective neurons that targets the ventral stream (also see Gur et al. 2005). The route begins in the primary visual cortex in sublayer 4Cα (Fig. 1) from where it travels a short distance to a neighbouring sublayer, 4 Cm, which is a thin sheet of cell bodies and dendrites wedged in the middle (m) of layer 4C (Yabuta & Callaway, 1998). The axons of neurons in layer 4Cm travel upwards through V1 and innervate direction-selective neurons in layer 3 (Gur & Snodderly, 2007), from where information is sent to the ventral stream (Ungerleider & Pasternak, 2003). It appears that the ventral visual pathway has no need to extract motion information from the dorsal stream across wide reaches of cortical space. Rather, the ventral path is supplied by its own motion input at source. This is significant because trying to combine form and motion signals that have already travelled through many separate cortical areas, each adding variable transmission delays, is a complex task. Direction-selective neurons in V1 that send motion information to the ventral stream are highly discriminating. They must be activated with a stimulus of exactly the correct orientation, moving in the correct direction and doing so within a very restricted patch of visual space. It is presumed that these cells are involved in fine motion discrimination during fixation, e.g. detecting the movement of facial features. Gur & Snodderly (2007) have also found consistent differences between the directional properties of cells in the other direction-selective V1 layers (4B and 6), which are destined for the dorsal pathway. Taken as a whole it appears that three distinct motion pathways exist in V1, two targeting the dorsal pathway, and one targeting ventral destinations (Fig. 1). Gur & Snodderly (2007) have uncovered a functional pathway by which directional information reaches the ventral stream without first travelling along the dorsal stream. This finding reduces the spatial binding problem by removing the need to combine form and motion information from widely separate brain regions. A binding issue still remains as form and motion need to be united, but at least this can happen in neighbouring cells that have experienced similar transmission delays.

Active mesh-a feature seeking and tracking image sequence representation scheme

This paper introduces a representation scheme for image sequences using nonuniform samples embedded in a deformable mesh structure. It describes a sequence by nodal positions and colors in a starting frame, followed by nodal displacements in the following frames. The nodal points in the mesh are more densely distributed in regions containing interesting features such as edges and corners; and are dynamically updated to follow the same features in successive frames. They are determined automatically by maximizing feature (e.g., gradient) magnitudes at nodal points, while minimizing interpolation errors within individual elements, and matching errors between corresponding elements. In order to avoid the mesh elements becoming overly deformed, a penalty term is also incorporated, which measures the irregularity of the mesh structure. The notions of shape functions and master elements commonly used in the finite element method have been applied to simplify the numerical calculation of the energy functions and their gradients. The proposed representation is motivated by the active contour or snake model proposed by Kass, Witkin, and Terzopoulos (1988). The current representation retains the salient merit of the original model as a feature tracker based on local and collective information, while facilitating more accurate image interpolation and prediction. Our computer simulations have shown that the proposed scheme can successfully track facial feature movements in head-and-shoulder type of sequences, and more generally, interframe changes that can be modeled as elastic deformation. The treatment for the starting frame also constitutes an efficient representation of arbitrary still images.