Holistic Face Research Articles

Potentiated early neural responses to fearful faces are not driven by specific face parts

Prioritized processing of fearful compared to neutral faces is reflected in increased amplitudes of components of the event-related potential (ERP). It is unknown whether specific face parts drive these modulations. Here, we investigated the contributions of face parts on ERPs to task-irrelevant fearful and neutral faces using an ERP-dependent facial decoding technique and a large sample of participants (N = 83). Classical ERP analyses showed typical and robust increases of N170 and EPN amplitudes by fearful relative to neutral faces. Facial decoding further showed that the absolute amplitude of these components, as well as the P1, was driven by the low-frequency contrast of specific face parts. However, the difference between fearful and neutral faces was not driven by any specific face part, as supported by Bayesian statistics. Furthermore, there were no correlations between trait anxiety and main effects or interactions. These results suggest that increased N170 and EPN amplitudes to task-irrelevant fearful compared to neutral faces are not driven by specific facial regions but represent a holistic face processing effect.

Robust holistic face processing in early childhood during the COVID-19 pandemic

The timing of the developmental emergence of holistic face processing and its sensitivity to experience in early childhood are somewhat controversial topics. To investigate holistic face perception in early childhood, we used an online testing platform and administered a two-alternative forced-choice task to 4-, 5-, and 6-year-old children. The children saw pairs of composite faces and needed to decide whether the faces were the same or different. To determine whether experience with masked faces may have negatively affected holistic processing, we also administered a parental questionnaire to assess the children’s exposure to masked faces during the COVID-19 pandemic. We found that all three age groups performed holistic face processing when the faces were upright (Experiment 1) but not when the faces were inverted (Experiment 2), that response accuracy increased with age, and that response accuracy was not related to degree of exposure to masked faces. These results indicate that holistic face processing is relatively robust in early childhood and that short-term exposure to partially visible faces does not negatively affect young children’s holistic face perception.

Looking at the upper facial half enlarges the range of holistic face processing

Previous studies suggested that upper and lower facial halves might be involved in the human holistic face processing differently. In this study, we replicated and extended the finding above. In Experiment 1, we used the standard composite-face task to measure holistic face processing when participants made judgements on the upper and lower facial halves separately. Results showed that the composite-face effect was stronger for the upper facial half compared to the lower half. In Experiment 2, we investigated how facial information was integrated when participants focused on different features, using the perceptual field paradigm. Results showed that: (1) more “peripheral faces” were chosen when participants fixated at the eyes than when they fixated at the mouth; (2) less “peripheral faces” were chosen for inverted faces regardless of the fixated features. Findings from both experiments together indicate that more peripheral facial information were integrated when participants focused on the upper facial half, highlighting the significance of focusing on the upper facial half in face processing.

The face inversion effect does not provide a pure measure of holistic face processing.

It is widely held that upright faces are processed more holistically than inverted faces and that this difference is reflected in the face inversion effect. It is not clear, however, how the inversion effect can best be measured, whether it is task specific, or even whether it specifically correlates with processing of upright faces. We examined these questions in a large sample (N = 420) who provided data on processing of upright and inverted stimuli in two different tasks with faces and one with objects. We find that the inversion effects are task dependent, and that they do not correlate better among face processing tasks than they do across face and object processing tasks. These findings were obtained regardless of whether inversion effects were measured by means of difference scores or regression. In comparison, only inversion effects based on regression predicted performance with upright faces in tasks other than those the inversion effects were derived from. Critically, however, inversion effects based on regression also predicted performance with inverted faces to a similar degree as they predicted performance with upright faces. Consequently, and contrary to what is commonly assumed, inversion effects do not seem to capture effects specific to holistic processing of upright faces. While the present findings do not bring us closer to an understanding of which changes in cognitive processing are induced by inversion, they do suggest that inversion effects do not reflect a unitary construct; an implicit assumption that seems to characterize much of the research regarding face processing.

Joint Holistic and Masked Face Recognition

With the widespread use of face masks due to the COVID-19 pandemic, accurate masked face recognition has become more crucial than ever. While several studies have investigated masked face recognition using convolutional neural networks (CNNs), there is a paucity of research exploring the use of plain Vision Transformers (ViTs) for this task. Unlike ViT models used in image classification, object detection, and semantic segmentation, the model trained by modern face recognition losses struggles to converge when trained from scratch. To this end, this paper initializes the model parameters via a proxy task of patch reconstruction and observes that the ViT backbone exhibits improved training stability with satisfactory performance for face recognition. Beyond the training stability, two strategies based on prompts are proposed to integrate holistic and masked face recognition in a single framework, namely FaceT. Along with popular holistic face recognition benchmarks, several open-sourced masked face recognition benchmarks are collected for evaluation. Our extensive experiments demonstrate that the proposed FaceT performs on par or better than state-of-the-art CNNs on both holistic and masked face recognition benchmarks. Codes will be made available at https://github.com/zyainfal/Joint-Holistic-and-Masked-Face-Recognition.

Happy face advantage disappears in Chinese context: The constancy of holistic processing for emotional faces

It is well documented that facial identity is processed in a holistic manner. Chen & Cheung (2021) recently found that happy faces evoked a larger holistic face processing than angry faces in Caucasian context. Here we adopted the complete composite face paradigm to investigate how facial emotion influenced holistic processing of facial identity in Chinese context. In Experiment 1, we used Chinese faces with happy, neutral and surprise emotion. Participants judged whether the top halves of the two successively presented composite faces were same or different, while their facial emotion was always consistent in a trial. We found equal composite face effects (CFE) for the three emotional faces. In Experiment 2, either top halves or bottom halves were judged, and angry, happy, fearful and neutral faces were compared. Results showed that although judging the top half induced a larger CFE than judging the bottom half, the CFE was equal among four emotional faces. In Experiment 3, angry and happy faces with high/low emotional intensity were further compared. We also found equal CFE for angry and happy faces, which was not modulated by the emotional intensity. These results indicated that the holistic processing of facial identity is robust and constant regardless of emotion categories and emotion intensity in Chinese context. The relation between cultural difference and holistic processing of emotional faces should be considered in the future.

Individual differences in classification images of Mooney faces.

Human face recognition is robust even under conditions of extreme lighting and in situations where there is high noise and uncertainty. Mooney faces are a canonical example of this: Mooney faces are two-tone shadow-defined images that are readily and holistically recognized despite lacking easily segmented face features. Face perception in such impoverished situations-and Mooney face perception in particular-is often thought to be supported by comparing encountered faces to stored templates. Here, we used a classification image approach to measure the templates that observers use to recognize Mooney faces. Visualizing these templates reveals the regions and structures of the image that best predict individual observer recognition, and they reflect the underlying internal representation of faces. Using this approach, we tested whether there are classification images that are consistent from session to session, whether the classification images are observer-specific, and whether they allow for pattern completion of holistic representations even in the absence of an underlying signal. We found that classification images of Mooney faces were indeed non-random (i.e., consistent session from session) within each observer, but they were different between observers. This result is in line with previously proposed existence of face templates that support face recognition, and further suggests that these templates may be unique to each observer and could drive idiosyncratic individual differences in holistic face recognition. Moreover, we found classification images that reflected information within the blank regions of the original Mooney faces, suggesting that observers may fill in missing information using idiosyncratic internal information about faces.

The mechanisms supporting holistic perception of words and faces are not independent.

The question of whether word and face recognition rely on overlapping or dissociable neural and cognitive mechanisms received considerable attention in the literature. In the present work, we presented words (aligned or misaligned) superimposed on faces (aligned or misaligned) and tested the interference from the unattended stimulus category on holistic processing of the attended category. In Experiment 1, we found that holistic face processing is reduced when a face was overlaid with an unattended, aligned word (processed holistically). In Experiment 2, we found a similar reduction of holistic processing for words when a word was superimposed on an unattended, aligned face (processed holistically). This reciprocal interference effect indicates a trade-off in holistic processing of the two stimuli, consistent with the idea that word and face recognition may rely on non-independent, overlapping mechanisms.

Face inversion does not affect the reversed congruency effect of gaze.

In a spatial Stroop task, the eye-gaze target produces the reversed congruency effect-responses become shorter when the gaze direction and its location are incongruent than when they are congruent. The present study examined the face inversion effect on the gaze spatial Stroop task to clarify whether the holistic face processing or part-based processing of the eyes is responsible for the reversed congruency effect. In Experiment 1, participants judged the gaze direction of the upright or inverted face with a neutral expression presented either in the left or right visual field. In Experiment 2, we examined whether face inversion interacted with facial expressions (i.e., angry, happy, neutral, and sad). Face inversion disrupted holistic face processing, slowing down the overall performance relative to the performance with upright faces. However, face inversion did not affect the reversed congruency effect. These results further support the parts-based processing account and suggest that while faces are processed holistically, the reversed congruency effect, relying on the extracted local features (i.e., eyes), may be processed in a part-based manner.

The impact of COVID-19 on memory: Recognition for masked and unmasked faces.

Considering the current state of the worldwide pandemic, it is still common to encounter people wearing face protection masks. Although a safety measure against COVID-19, face masks might be compromising our capacity for face recognition. We conducted an online study where 140 participants observed masked and unmasked faces in a within-subjects design and then performed a recognition memory task. The best performance was found when there were no masks either at study and test phase, i.e., at the congruent unmasked condition. The worst performance was found for faces encoded with a mask but tested without it (i.e., masked-unmasked incongruent condition), which can be explained by the disruption in holistic face processing and the violation of the encoding specificity principle. Interestingly, considering the unmasked-masked incongruent condition, performance was probably affected by the violation of the encoding specificity principle but protected by holistic processing that occurred during encoding.

Detecting a familiar person behind the surgical mask: recognition without identification among masked versus sunglasses-covered faces

Previous research has shown that even when famous people’s identities cannot be discerned from faces that have been filtered with monochromatic noise, these unidentifiable famous faces still tend to receive higher familiarity ratings than similarly filtered non-famous faces. Experiment 1 investigated whether a similar face recognition without identification effect would occur among faces whose identification was hindered through the wearing of a surgical mask. Among a mixture of famous and non-famous faces wearing surgical masks and hoods, participants rated how familiar each person seemed then attempted to identify the person. Though surgical masks significantly impaired identification of the famous faces, unidentified masked famous faces received higher familiarity ratings on average than the non-famous masked faces, suggesting that a sense of familiarity could still occur even when identification was impaired by the mask. Experiment 2 compared faces covered by surgical masks with faces covered by sunglasses. Though sunglasses impaired face identification more than surgical masks, the magnitude of the face recognition without identification effect was the same in both cases. This pattern suggests that holistic face processing is not a requirement for the sense of familiarity with a face, and that different facial feature types can contribute.

Holistic face processing in 4- and 7-month-old infants.

Previous studies found an onset of holistic face processing in the age range between 0-4 and 7months of age. To validate these studies, the present study investigated infants 4 and 7months of age with a different experimental approach. In a habituation-dishabituation experiment, the infants were tested with stereoscopic stimuli in which stripes floated above a face, thereby occluding some parts of the face (amodal completion condition), and stereoscopic stimuli in which the same face parts floated above stripes (modal completion condition). Research with adults indicates that faces are processed holistically, that is as global wholes, in the amodal, but as independent parts in the modal completion condition, resulting in superior face recognition when the occluding bars are in front of than when they are behind the visible face parts. The present study found that infants regardless of whether they are 4 or 7months old reliably recognized and differentiated the faces in the amodal but not in the modal completion condition. Moreover, the difference between the experimental conditions was statistically significant. These findings show that approximately at the age of 4-7months of life, infants begin to holistically unify disjoint face parts into a coherent whole.

Social experience drives the development of holistic face processing in paper wasps.

Most recognition is based on identifying features, but specialization for face recognition in some taxa relies on a different mechanism, termed 'holistic processing' where facial features are bound together into a gestalt which is more than the sum of its parts. Although previous work suggests that extensive experience may be required for the development of holistic processing, we lack experiments that test how age and experience interact to influence holistic processing. Here, we test how age and experience influence the development of holistic face processing in Polistes fuscatus paper wasps. Previous work has shown that P. fuscatus use facial patterns to individually identify conspecifics and wasps use holistic processing to discriminate between conspecific faces. We tested face processing in three groups of P. fuscatus: young (1-week-old), older, experienced (2-weeks-old, normal experience), and older, inexperienced (2-weeks-old, 1 week normal social experience and 1 week social isolation). Older, experienced wasps used holistic processing to discriminate between conspecific faces. In contrast, older inexperienced wasps used featural rather than holistic mechanisms to discriminate between faces. Young wasps show some evidence of holistic face processing, but this ability was less refined than older, experienced wasps. Notably, wasps only required 2 weeks of normal experience to develop holistic processing, while previous work suggests that humans may require years of experience. Overall, P. fuscatus wasps rapidly develop holistic processing for conspecific faces. Experience rather than age facilitates the transition between featural and holistic face processing mechanisms.

The relation between holistic processing as measured by three composite tasks and face processing: A latent variable modeling approach

We investigated the relationship between holistic processing and face processing using a latent variables approach. Three versions of the composite paradigm were used to measure holistic processing: Vanderbilt Holistic Face Processing Test, a sequential composite matching task, and a simultaneous composite matching task. Three tasks were used to measure face perception and face memory abilities respectively. We had three pairs of tasks such that within each pair (of memory and perception task), the stimuli involved, the requirement for matching across viewpoints, etc., are the same, such that the only difference is whether perception or memory is taxed. There were no significant correlations between the different versions of the composite task. We discovered no evidence to support a distinction between face perception and face memory, suggesting the existence of a general face processing factor. Finally, there was no evidence that holistic processing (as captured by either of the three composite tasks) is predictive of better face processing per se, casting doubts on the role of holistic processing in differentiating different levels of efficiency in face processing.

Examining the Impact of Human Face Stimulus on Shape-Contrast Effects during a Brief Presentation.

Shape-contrast effects have been introduced to the investigations into face perception with the aim of exploring face adaptation in the context of norm-based coding. Research has indicated that shape-contrast effects occur even for shapes as complex as the human face. However, whether the complexity of face stimuli alters the magnitude of shape-contrast effects needs to be examined. In this study, emoticons and realistic human faces were used with the original white circle as the test stimuli. The results revealed that the shape-contrast effect was dependent on the stimulus. However, there was no significant difference between the shape-contrast effect evoked by upright faces and that evoked by inverted ones. This suggests that the face stimuli influenced the strength of the shape-contrast effect: the mechanism of this effect involved multiple stages of the visual system related to luminance and complexity, rather than the holistic face perception.

A featural account for own-face processing? Looking for support from face inversion, composite face, and part-whole tasks.

It is widely accepted that face perception relies on holistic processing. However, this holistic advantage is not always found in the processing of the own face. Our study aimed to explore the role of holistic and featural processing in the identification of the own face, using three standard, but largely independent measures of holistic face processing: the face inversion task, the composite face task, and the part-whole task. Participants were asked to identify their face, a friend’s face, and an unfamiliar face in three different experimental blocks: (a) inverted versus upright; (b) top and bottom halves of the face aligned versus misaligned; and (c) facial features presented in isolation versus whole foil face context. Inverting a face impaired its identification, regardless of the identity. However, alignment effects were only found when identifying a friend or an unfamiliar face. In addition, a stronger feature advantage (i.e., better recognition for isolated features compared to in a whole-face context) was observed for the own face compared to the friend and unfamiliar faces. Altogether, these findings suggest that the own face is processed in a more featural manner but also relies on holistic processing. This work also highlights the importance of taking into consideration that different holistic processing paradigms could tap different forms of holistic processing.

AI based contactless attendance monitoring and management system

The traditional methods of attendance marking need to be replaced with an Artificial Intelligence based attendance monitoring system. The process of checking attendance using the existing methods is time consuming and open to easy fraud. The usage of Bio-metrics poses the threat of viruses. A contact-less Attendance Management System (CAMS) is proposed to overcome the above challenges. AI based attendance monitoring system will record the students attendance automatically while they enter the class, by recognizing their faces. The in-time and out-time of the students are also recorded on a regular basis. The Holistic Face Matching algorithm is used to compare the facial images. The total time duration, the students were present inside the class can be easily calculated. There is no chance of fake attendance as each student has to record his or her facial identity. The main objective is to achieve reliability, accessibility and security in a very efficient way. The results showed that by interfacing a face recognition system with a server, a real-time attendance system can be built and be monitored automatically. Finally, the student’s attendance details can be updated in the academic management system.

Relative tuning of holistic face processing towards the fovea

Humans quickly detect and gaze at faces in the world, which reflects their importance in cognition and may lead to tuning of face recognition toward the central visual field. Although sometimes reported, foveal selectivity in face processing is debated: brain imaging studies have found evidence for a central field bias specific to faces, but behavioral studies have found little foveal selectivity in face recognition. These conflicting results are difficult to reconcile, but they could arise from stimulus-specific differences. Recent studies, for example, suggest that individual faces vary in the degree to which they require holistic processing. Holistic processing is the perception of faces as a whole rather than as a set of separate features. We hypothesized that the dissociation between behavioral and neuroimaging studies arises because of this stimulus-specific dependence on holistic processing. Specifically, the central bias found in neuroimaging studies may be specific to holistic processing. Here, we tested whether the eccentricity-dependence of face perception is determined by the degree to which faces require holistic processing. We first measured the holistic-ness of individual Mooney faces (two-tone shadow images readily perceived as faces). In a group of independent observers, we then used a gender discrimination task to measured recognition of these Mooney faces as a function of their eccentricity. Face gender was recognized across the visual field, even at substantial eccentricities, replicating prior work. Importantly, however, holistic face gender recognition was relatively tuned—slightly, but reliably stronger in the central visual field. Our results may reconcile the debate on the eccentricity-dependance of face perception and reveal a spatial inhomogeneity specifically in the holistic representations of faces.

Face-Processing Differences Present in Grapheme-Color Synesthetes.

Grapheme‐color synesthesia is a heterogeneous neurological phenomenon whereby the experience of a grapheme automatically and involuntarily elicits an experience of color. While the majority of synesthesia research has focused on inducer‐specific influences of synesthetic associations, more recent efforts have examined potential broader differences. Based on spontaneous reports from synesthetes detailing problems with face recognition, in conjunction with the geographical proximity of neurological regions relevant to both synesthesia and face processing, we sought to examine whether synesthetes demonstrated atypical face‐processing abilities.A total of 16 grapheme‐color synesthetes and 16 age‐and‐gender matched controls (±3 years) completed the Cambridge Face Memory Test (CFMT; Duchaine & Nakayama, 2006) of face memory, the Vanderbilt Holistic Face Processing Task (VHPT‐F; Richler, Floyd, & Gauthier, 2014) of holistic face processing, as well as a standardized self‐report questionnaire the Faces and Emotions Questionnaire (Freeman, Palermo, & Brock, 2015). The results revealed significantly poorer performance in synesthete's ability to recognize faces in the CFMT that was driven by a reduction in upright advantage. Results also revealed a significant reduction in overall accuracy on the VHPT‐F for synesthetes, who despite this displayed a comparable holistic processing advantage compared to matched controls. Finally, synesthetes also rated themselves as significantly worse at face recognition. We suggest that this pattern may reflect differences in the development of individualized perceptual strategies.

Social avoidance motivation tendency linked to face processing ability among 6- to 12-year-old children

This study examined whether social motivation was associated with face recognition ability and holistic face processing strategy among typically developing children. One hundred and three 6- to 12-year-old children were instructed to complete the approach and avoidance task, which was adopted to measure social motivation tendency, as well as the recognition of face identity task and the part-whole task, which were used to assess children’s face recognition ability and holistic processing strategy, respectively. For social motivation, the result revealed that 6- to 7-year-old children showed a lower social approach tendency, whereas 8- to 11-year-old children did not show preferences for social pictures over nonsocial pictures. While a linear improvement in face recognition abilities was found from 6- to 11-year-old children, a plateau emerged in face recognition performance from age 10 to 11. In contrast to face holistic processing strategy, 10- to 11-year-old children tend to use a featural processing strategy for identifying facial features. Although the relationship between social motivation and face recognition was not moderated by children’s age, results revealed that high social avoidance tendency toward emotionally neutral and negative pictures significantly predicted faster face recognition after controlling for IQ of children, and high social avoidance tendency to neutral pictures significantly predicted faster eye identification. However, social motivation was not associated with holistic face processing. These findings highlight the importance of social motivation in the development of children’s face recognition abilities.