Dynamic Face Perception Research Articles

EEG correlates of static and dynamic face perception: the role of naturalistic motion

Much of our understanding of how the brain processes dynamic faces comes from research that compares static photographs to dynamic morphs, which exhibit simplified, computer-generated motion. By comparing static, video recorded, and dynamic morphed expressions, we aim to identify the neural correlates of naturalistic facial dynamism, using time-domain and time-frequency analysis. Dynamic morphs were made from the neutral and peak frames of video recorded transitions of happy and fearful expressions, which retained expression change and removed asynchronous and non-linear features of naturalistic facial motion. We found that dynamic morphs elicited increased N400 amplitudes and lower LPP amplitudes compared to other stimulus types. Video recordings elicited higher LPP amplitudes and greater frontal delta activity compared to other stimuli. Thematic analysis of participant interviews using a large language model revealed that participants found it difficult to assess the genuineness of morphed expressions, and easier to analyse the genuineness of happy compared to fearful expressions. Our findings suggest that animating real faces with artificial motion may violate expectations (N400) and reduce the social salience (LPP) of dynamic morphs. Results also suggest that delta oscillations in the frontal region may be involved with the perception of naturalistic facial motion in happy and fearful expressions. Overall, our findings highlight the sensitivity of neural mechanisms required for face perception to subtle changes in facial motion characteristics, which has important implications for neuroimaging research using faces with simplified motion.

The effect of processing partial information in dynamic face perception

Face perception is a major topic in vision research. Most previous research has concentrated on (holistic) spatial representations of faces, often with static faces as stimuli. However, faces are highly dynamic stimuli containing important temporal information. How sensitive humans are regarding temporal information in dynamic faces is not well understood. Studies investigating temporal information in dynamic faces usually focus on the processing of emotional expressions. However, faces also contain relevant temporal information without any strong emotional expression. To investigate cues that modulate human sensitivity to temporal order, we utilized muted dynamic neutral face videos in two experiments. We varied the orientation of the faces (upright and inverted) and the presence/absence of eye blinks as partial dynamic cues. Participants viewed short, muted, monochromic videos of models vocalizing a widely known text (National Anthem). Videos were played either forward (in the correct temporal order) or backward. Participants were asked to determine the direction of the temporal order for each video, and (at the end of the experiment) whether they had understood the speech. We found that face orientation, and the presence/absence of an eye blink affected sensitivity, criterion (bias) and reaction time: Overall, sensitivity was higher for upright compared to inverted faces, and in the condition where an eye blink was present compared to the condition without an eye blink. Reaction times were mostly faster in the conditions with higher sensitivity. A bias to report inverted faces as ‘backward’ observed in Experiment I, where upright and inverted faces were presented randomly interleaved within each block, was absent when presenting upright and inverted faces in different blocks in Experiment II. Language comprehension results revealed that there was higher sensitivity when understanding the speech compared to not understanding the speech in both experiments. Taken together, our results showed higher sensitivity with upright compared to inverted faces, suggesting that the perception of dynamic, task-relevant information was superior with the canonical orientation of the faces. Furthermore, partial information coming from eye blinks, in addition to mouth movements, seemed to play a significant role in dynamic face perception, both when faces were presented upright and inverted. We suggest that studying the perception of facial dynamics beyond emotional expressions will help us to better understand the mechanisms underlying the temporal integration of facial information from different -partial and holistic- sources, and that our results show how different strategies, depending on the available information, are employed by human observers when judging the temporal order of faces.

Dynamic Face Perception: The Role of Expertise in Dual Processing of Features and Configuration

Face perception is the basis of many types of social information exchange, but there is controversy over its underlying mechanisms. Researchers have theorized two processing pathways underlying facial perception: configural processing and featural processing. Featural processing focuses on the individual features of a face, whereas configural processing focuses on the spatial relations of features. To resolve the debate on the relative contribution of the two pathways in face perception, researchers have proposed a dual processing model that the two pathways contribute to two different perceptions, detecting face-like patterns and identifying individual faces. The dual processing model is based on face perception experiments that primarily use static faces. As we mostly interact with dynamic faces in real life, the generalization of the model to dynamic faces will advance our understanding of how faces are perceived in real life. This paper proposes a refined dual processing model of dynamic face perception, in which expertise in dynamic face perception supports identifying individual faces, and it is a learned behaviour that develops with age. Specifically, facial motions account for the advantages of dynamic faces, compared to static faces. This paper highlights two intrinsic characteristics of facial motions that enable the advantages of dynamic faces in face perception. Firstly, facial motion provides facial information from various viewpoints, and thus supports the generalization of face perception to the unlearned view of faces. Secondly, distinctive motion patterns serve as a cue to the identity of the face.

Neural correlates of integration processes during dynamic face perception

Integrating the spatiotemporal information acquired from the highly dynamic world around us is essential to navigate, reason, and decide properly. Although this is particularly important in a face-to-face conversation, very little research to date has specifically examined the neural correlates of temporal integration in dynamic face perception. Here we present statistically robust observations regarding the brain activations measured via electroencephalography (EEG) that are specific to the temporal integration. To that end, we generate videos of neutral faces of individuals and non-face objects, modulate the contrast of the even and odd frames at two specific frequencies (f_1 and f_2) in an interlaced manner, and measure the steady-state visual evoked potential as participants view the videos. Then, we analyze the intermodulation components (IMs: (nf_1pm mf_2), a linear combination of the fundamentals with integer multipliers) that consequently reflect the nonlinear processing and indicate temporal integration by design. We show that electrodes around the medial temporal, inferior, and medial frontal areas respond strongly and selectively when viewing dynamic faces, which manifests the essential processes underlying our ability to perceive and understand our social world. The generation of IMs is only possible if even and odd frames are processed in succession and integrated temporally, therefore, the strong IMs in our frequency spectrum analysis show that the time between frames (1/60 s) is sufficient for temporal integration.

Neural Correlates of Integration Processes during Dynamic Face Perception

Neural Correlates of Integration Processes during Dynamic Face Perception

Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections

Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.

Perceiving social interactions in the posterior superior temporal sulcus

Primates are highly attuned not just to social characteristics of individual agents, but also to social interactions between multiple agents. Here we report a neural correlate of the representation of social interactions in the human brain. Specifically, we observe a strong univariate response in the posterior superior temporal sulcus (pSTS) to stimuli depicting social interactions between two agents, compared with (i) pairs of agents not interacting with each other, (ii) physical interactions between inanimate objects, and (iii) individual animate agents pursuing goals and interacting with inanimate objects. We further show that this region contains information about the nature of the social interaction-specifically, whether one agent is helping or hindering the other. This sensitivity to social interactions is strongest in a specific subregion of the pSTS but extends to a lesser extent into nearby regions previously implicated in theory of mind and dynamic face perception. This sensitivity to the presence and nature of social interactions is not easily explainable in terms of low-level visual features, attention, or the animacy, actions, or goals of individual agents. This region may underlie our ability to understand the structure of our social world and navigate within it.

Neural Correlates of Dynamic Face Perception

Neural Correlates of Dynamic Face Perception

Magnetoencephalographic study on facial movements.

In this review, we introduced our three studies that focused on facial movements. In the first study, we examined the temporal characteristics of neural responses elicited by viewing mouth movements, and assessed differences between the responses to mouth opening and closing movements and an averting eyes condition. Our results showed that the occipitotemporal area, the human MT/V5 homologue, was active in the perception of both mouth and eye motions. Viewing mouth and eye movements did not elicit significantly different activity in the occipitotemporal area, which indicated that perception of the movement of facial parts may be processed in the same manner, and this is different from motion in general. In the second study, we investigated whether early activity in the occipitotemporal region evoked by eye movements was influenced by the facial contour and/or features such as the mouth. Our results revealed specific information processing for eye movements in the occipitotemporal region, and this activity was significantly influenced by whether movements appeared with the facial contour and/or features, in other words, whether the eyes moved, even if the movement itself was the same. In the third study, we examined the effects of inverting the facial contour (hair and chin) and features (eyes, nose, and mouth) on processing for static and dynamic face perception. Our results showed the following: (1) In static face perception, activity in the right fusiform area was affected more by the inversion of features while that in the left fusiform area was affected more by a disruption in the spatial relationship between the contour and features; and (2) In dynamic face perception, activity in the right occipitotemporal area was affected by the inversion of the facial contour.

Do the eyes really have it? Dynamic allocation of attention when viewing moving faces

What controls gaze allocation during dynamic face perception? We monitored participants' eye movements while they watched videos featuring close-ups of pedestrians engaged in interviews. Contrary to previous findings using static displays, we observed no general preference to fixate eyes. Instead, gaze was dynamically directed to the eyes, nose, or mouth in response to the currently depicted event. Fixations to the eyes increased when a depicted face made eye contact with the camera, while fixations to the mouth increased when the face was speaking. When a face moved quickly, fixations concentrated on the nose, suggesting that it served as a spatial anchor. To better understand the influence of auditory speech during dynamic face perception, we presented participants with a second version of the same video, in which the audio speech track had been removed, leaving just the background music. Removing the speech signal modulated gaze allocation by decreasing fixations to faces generally and the mouth specifically. Since the task was to simply rate the likeability of the videos, the decrease of attention allocation to the mouth region implies a reduction of the functional benefits of mouth fixations given that speech comprehension was not required. Together, these results argue against a general prioritization of the eyes and support a more functional, information-seeking use of gaze allocation during dynamic face viewing.

Dynamic Facial Expressions Evoke Distinct Activation in the Face Perception Network: A Connectivity Analysis Study

Very little is known about the neural structures involved in the perception of realistic dynamic facial expressions. In the present study, a unique set of naturalistic dynamic facial emotional expressions was created. Through fMRI and connectivity analysis, a dynamic face perception network was identified, which is demonstrated to extend Haxby et al.'s [Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. The distributed human neural system for face perception. Trends in Cognitive Science, 4, 223-233, 2000] distributed neural system for face perception. This network includes early visual regions, such as the inferior occipital gyrus, which is identified as insensitive to motion or affect but sensitive to the visual stimulus, the STS, identified as specifically sensitive to motion, and the amygdala, recruited to process affect. Measures of effective connectivity between these regions revealed that dynamic facial stimuli were associated with specific increases in connectivity between early visual regions, such as the inferior occipital gyrus and the STS, along with coupling between the STS and the amygdala, as well as the inferior frontal gyrus. These findings support the presence of a distributed network of cortical regions that mediate the perception of different dynamic facial expressions.

Effects of inverting contour and features on processing for static and dynamic face perception: An MEG study

We investigated the effects of inverting facial contour (hair and chin) and features (eyes, nose and mouth) on processing for static and dynamic face perception using magnetoencephalography (MEG). We used apparent motion, in which the first stimulus (S1) was replaced by a second stimulus (S2) with no interstimulus interval and subjects perceived visual motion, and presented three conditions as follows: (1) U&U: Upright contour and Upright features, (2) U&I: Upright contour and Inverted features, and (3) I&I: Inverted contour and Inverted features. In static face perception (S1 onset), the peak latency of the fusiform area's activity, which was related to static face perception, was significantly longer for U&I and I&I than for U&U in the right hemisphere and for U&I than for U&U and I&I in the left. In dynamic face perception (S2 onset), the strength (moment) of the occipitotemporal area's activity, which was related to dynamic face perception, was significantly larger for I&I than for U&U and U&I in the right hemisphere, but not the left. These results can be summarized as follows: (1) in static face perception, the activity of the right fusiform area was more affected by the inversion of features while that of the left fusiform area was more affected by the disruption of the spatial relation between the contour and features, and (2) in dynamic face perception, the activity of the right occipitotemporal area was affected by the inversion of the facial contour.

Identifying regions that carry the best information about global facial configurations

Regions of the face are not equally important in conveying information about configural change. The bubbles spatial occlusion technique has proved to be a good method for revealing which areas carry diagnostic facial information for different perceptual categorization tasks. We have applied it here within a performance-driven mimicry system implemented using a computer-generated model of the face designed to automatically retarget the behavior of one face onto another face. Our bubbles technique, mapping an occluded face into a PCA model of the same face, revealed the areas around and including the mouth and eyebrows as the most important for facial image reconstruction. These regions overlapped with but interestingly were not identical to areas of maximum pixel-value variance. Here we show a system that is indifferent to stimulus content and uses the correlation between vectors in face space as a criterion, rather than just pixel-value correlation, identifies the eyebrows and mouth as important regions. This implies that the importance of the eyebrows and the mouth in dynamic face perception may depend not on the information content of the features per se but on the degree to which these regions of the face provide information about the global form of the face.

Effects of direction of facial contour and features on processing for dynamic face perception: an MEG study

Effects of direction of facial contour and features on processing for dynamic face perception: an MEG study