Face Identity Processing Research Articles

The 2024 Richardson Lecture: Prosopagnosia - A Classic Neurologic Deficit Meets the Modern Era.

Acquired prosopagnosia is a rare disorder, but it serves as a model for impairments in expert-level visual processing. This review discusses five key observations made over the past 30 years. First, there are variants, an apperceptive type linked to damage to the inferior occipitotemporal cortex and an amnestic type associated with anterior temporal lesions, both either right or bilateral. Second, these variants are clustered in syndromes with other perceptual deficits, the apperceptive type with field defects, dyschromatopsia and topographagnosia, and the amnestic type with topographagnosia and the auditory disorders of phonagnosia and acquired amusia. Third, extensive testing often shows additional problems with recognizing exemplars of other objects, especially when degrees of expertise are taken into account. Fourth, the prosopagnosic impairment does not affect all facial information. For example, the perception of expression and lip-reading likely depends on other neural substrates than those for processing facial identity. Last, face perception in prosopagnosia is not immutable but can improve with extensive training, though as yet this does not represent a cure for the condition. Continuing work with neural networks and animal models will enhance our understanding of this intriguing condition and what it tells us about how our brains process vision.

Flexible use of facial features supports face identity processing.

People prioritize diagnostic features in classification tasks. However, it is not clear whether this priority is fixed or is flexibly applied depending on the specific classification decision, or how feature use behavior contributes to individual differences in performance. Here we examined whether flexibility in features used in a face identification task supports face recognition ability. In Experiment 1, we show that the facial features most useful for identification vary-to a surprising degree-depending on the specific face identity comparison at hand. While the ears and eyes were the most diagnostic for face identification in general, they were the most diagnostic feature for just 22% and 14% of identity decisions, respectively. In three subsequent experiments, we find that flexibility in feature use contributes to an individual's face identity matching ability. Higher face identification accuracy was associated with being aware of (Experiments 2 and 4) and attending to (Experiments 3 and 4) the most diagnostic features for a specific facial comparison. This conferred an enhanced benefit relative to focusing on features that were diagnostic of face identity decisions in general (Experiment 4). We conclude that adaptability in information sampling supports face recognition ability and discuss theoretical and applied implications. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Information sampling differences supporting superior face identity processing ability.

Face recognition in humans is often cited as a model example of perceptual expertise that is characterized by an increased tendency to process faces as holistic percepts. However emerging evidence across different domains of expertise points to a critical role of feature-based processing strategies during the initial encoding of information. Here, we examined the eye-movement patterns of super-recognisers-individuals with extremely high faceidentification ability compared with the average person-using gaze-contingent "spotlight" apertures that restrict visual face information in real time around their point of fixation. As an additional contrast, we also compared their performance with that of facial examiners-highly trained individuals whose superiority has been shown to rely heavily on featural processing. Super-recognisers and facial examiners showed equivalent facematching accuracy in both spotlight aperture and natural viewing conditions, suggesting that they were equally adept at using featural information for faceidentity processing. Further, both groups sampled more information across the face than controls. Together, these results show that the active exploration of facial features is an important determinant of facerecognition ability that generalizes across different types of experts.

EXPRESS: Prosopagnosia is highly comorbid in individuals with probable developmental coordination disorder (DCD).

Developmental co-ordination disorder (DCD) is characterised by difficulties in motor control and coordination from early childhood. While problems processing facial identity are often associated with neurodevelopmental conditions, such issues have never been directly tested in adults with DCD. We tested this possibility through a range of tasks, and assessed the prevalence of developmental prosopagnosia (i.e., lifelong difficulties with faces), in a group comprising individuals who self-reported a diagnosis of, or suspected that they had, DCD. Strikingly, we found 53% of this probable DCD group met recently recommended criteria for a diagnosis of prosopagnosia, with 22% acquiring a diagnosis using traditional cognitive task-based methods. Moreover, their problems with faces were apparent on both unfamiliar and familiar face memory tests, as well as on a facial perception task (i.e., could they tell faces apart). Positive correlations were found between self-report measures assessing movement and coordination problems, and objective difficulties on experimental face identity processing tasks, suggesting widespread neurocognitive disruption in DCD. Importantly, issues in identity processing in our probable DCD group remained even after excluding participants with comorbid conditions traditionally associated with difficulties in face recognition, i.e., autism and dyslexia. We recommend that any diagnostic test for DCD should include an assessment for prosopagnosia. Given the high prevalence of prosopagnosia in our probable DCD group, and the positive correlations between DCD and prosopagnosia symptoms, there may be a stronger link between movement and facial identity abilities than previously thought.

Super recognizers: Increased sensitivity or reduced biases? Insights from serial dependence.

Super recognizers (SRs) are people that exhibit a naturally occurring superiority for processing facial identity. Despite the increase of SR research, the mechanisms underlying their exceptional abilities remain unclear. Here, we investigated whether the enhanced facial identity processing of SRs could be attributed to the lack of sequential effects, such as serial dependence. In serial dependence, perception of stimulus features is assimilated toward stimuli presented in previous trials. This constant error in visual perception has been proposed as a mechanism that promotes perceptual stability in everyday life. We hypothesized that an absence of this constant source of error in SRs could account for their superior processing-potentially in a domain-general fashion. We tested SRs (n = 17) identified via a recently proposed diagnostic framework (Ramon, 2021) and age-matched controls (n = 20) with two experiments probing serial dependence in the face and shape domains. In each experiment, observers were presented with randomly morphed face identities or shapes and were asked to adjust a face's identity or a shape to match the stimulus they saw. We found serial dependence in controls and SRs alike, with no difference in its magnitude across groups. Interestingly, we found that serial dependence impacted the performance of SRs more than that of controls. Taken together, our results show that enhanced face identity processing skills in SRs cannot be attributed to the lack of serial dependence. Rather, serial dependence, a beneficial nested error in our visual system, may in fact further stabilize the perception of SRs and thus enhance their visual processing proficiency.

Determinants of Face Recognition: The Role of Target Prevalence and Similarity.

Studies of facial identity processing typically assess perception (via matching) and/or memory (via recognition), with experimental designs differing with respect to one important aspect: Target Prevalence. Some designs include "target absent" (TA) among "target present" (TP) trials. In visual search tasks, TA trials shift an observer's decisional criterion towards a stricter one, increasing misses. However, decisional biases will differ between individuals and across an individual's decisions as well. In this way, excluding TA trials ensures comparable levels of expectation and thus a more controlled decisional bias both within and between observers by not considering correct rejections and false alarms. However, TA trials may occur, e.g., in police line-ups, where it is important to consider observers' face recognition ability net of the potential biases introduced by TA and TP trials. And, while these have been investigated in numerous other stimulus domains, their effects have not yet been extended to face recognition. We therefore sought to fill this void by testing different versions of the previously established Models Memory Test, which measures old/new recognition of experimentally learned facial identities. Our study found significant expectation effects, driven by target prevalence that persist even given prevalence changes. This implies that face recognition - even measured with naturalistic changes - is influenced by prior perceptual decisions.

Temporal dynamics of facial identity and expression processing from magnetoencephalography

Temporal dynamics of facial identity and expression processing from magnetoencephalography

Data-driven studies in face identity processing rely on the quality of the tests and data sets

There is growing interest in how data-driven approaches can help understand individual differences in face identity processing (FIP). However, researchers employ various FIP tests interchangeably, and it is unclear whether these tests 1) measure the same underlying ability/ies and processes (e.g., confirmation of identity match or elimination of identity match) 2) are reliable, 3) provide consistent performance for individuals across tests online and in laboratory. Together these factors would influence the outcomes of data-driven analyses. Here, we asked 211 participants to perform eight tests frequently reported in the literature. We used Principal Component Analysis and Agglomerative Clustering to determine factors underpinning performance. Importantly, we examined the reliability of these tests, relationships between them, and quantified participant consistency across tests. Our findings show that participants’ performance can be split into two factors (called here confirmation and elimination of an identity match) and that participants cluster according to whether they are strong on one of the factors or equally on both. We found that the reliability of these tests is at best moderate, the correlations between them are weak, and that the consistency in participant performance across tests and is low. Developing reliable and valid measures of FIP and consistently scrutinising existing ones will be key for drawing meaningful conclusions from data-driven studies.

Dissociations between face identity and face expression processing in developmental prosopagnosia

Individuals with developmental prosopagnosia (DPs) experience severe and lifelong deficits recognising faces, but whether their deficits are selective to the processing of face identity or extend to the processing of face expression remains unclear. Clarifying this issue is important for understanding DP impairments and advancing theories of face processing. We compared identity and expression processing in a large sample of DPs (N = 124) using three different matching tasks that each assessed identity and expression processing with identical experimental formats. We ran each task in upright and inverted orientations and we measured inversion effects to assess the integrity of upright-specific face processes. We report three main results. First, DPs showed large deficits at discriminating identity but only subtle deficits at discriminating expression. Second, DPs showed a reduced inversion effect for identity but a normal inversion effect for expression. Third, DPs' performance on the expression tasks were linked to autism traits, but their performance on the identity tasks were not. These results constitute several dissociations between identity and expression processing in DP, and they are consistent with the view that the core impairment in DP is highly selective to identity.

Selecting police super-recognisers.

People vary in their ability to recognise faces. These individual differences are consistent over time, heritable and associated with brain anatomy. This implies that face identity processing can be improved in applied settings by selecting high performers-'super-recognisers' (SRs)-but these selection processes are rarely available for scientific scrutiny. Here we report an 'end-to-end' selection process used to establish an SR 'unit' in a large police force. Australian police officers (n = 1600) completed 3 standardised face identification tests and we recruited 38 SRs from this cohort to complete 10 follow-up tests. As a group, SRs were 20% better than controls in lab-based tests of face memory and matching, and equalled or surpassed accuracy of forensic specialists that currently perform face identification tasks for police. Individually, SR accuracy was variable but this problem was mitigated by adopting strict selection criteria. SRs' superior abilities transferred only partially to body identity decisions where the face was not visible, and they were no better than controls at deciding which visual scene that faces had initially been encountered in. Notwithstanding these important qualifications, we conclude that super-recognisers are an effective solution to improving face identity processing in applied settings.

Improving forensic perpetrator identification with Super-Recognizers

About a decade ago, Super-Recognizers (SRs) were first described as individuals with exceptional face identity processing abilities. Since then, various tests have been developed or adapted to assess individuals' abilities and identify SRs. The extant literature suggests that SRs may be beneficial in police tasks requiring individual identification. However, in reality, the performance of SRs has never been examined using authentic forensic material. This not only limits the external validity of test procedures used to identify SRs, but also claims concerning their deployment in policing. Here, we report the first-ever investigation of SRs' ability to identify perpetrators using authentic case material. We report the data of 73 SRs and 45 control participants. These include (a) performance on three challenging tests of face identity processing recommended by Ramon (2021) for SR identification; (b) performance for perpetrator identification using four CCTV sequences depicting five perpetrators and police line-ups created for criminal investigation purposes. Our findings demonstrate that the face identity processing tests used here are valid in measuring such abilities and identifying SRs. Moreover, SRs excel at perpetrator identification relative to control participants, with more correct perpetrator identifications, the better their performance across lab tests. These results provide external validity for the recently proposed diagnostic framework and its tests used for SR identification (Ramon, 2021). This study provides the first empirical evidence that SRs identified using these measures can be beneficial for forensic perpetrator identification. We discuss theoretical and practical implications for law enforcement, whose procedures can be improved via a human-centric approach centered around individuals with superior abilities.

A correlational study investigating whether semantic knowledge facilitates face identity processing

ABSTRACT The ability to recognize faces is a fundamental skill in human social interaction. While much research has focused on the recognition of familiar faces, there is growing interest in understanding the cognitive processes underlying the recognition of unfamiliar faces. Previous studies have suggested that both semantic knowledge and physical features play a role in unfamiliar face recognition, but the nature of their relationship is not well understood. This study examines the relationship between unfamiliar face recognition ability and the encoding abilities of semantic knowledge and physical features for famous faces. Using the Gorilla platform, a large group of participants (N = 66) with a broad age range completed three tasks: a challenging unfamiliar face matching task and Famous People Recognition Tests 1 and 2 to evaluate semantic and physical feature encoding abilities, respectively. Results indicate positive correlations between encoding abilities for both semantic knowledge and physical features of familiar faces with Model Face Matching Task scores. Additionally, the encoding ability for semantic knowledge was found to be positively associated with that of physical features.

The neural dynamics of familiarity-dependent face identity representation.

Recognizing a face as belonging to a given identity is essential in our everyday life. Clearly, the correct identification of a face is only possible for familiar people, but 'familiarity' covers a wide range-from people we see every day to those we barely know. Although several studies have shown that the processing of familiar and unfamiliar faces is substantially different, little is known about how the degree of familiarity affects the neural dynamics of face identity processing. Here, we report the results of a multivariate EEG analysis, examining the representational dynamics of face identity across several familiarity levels. Participants viewed highly variable face images of 20 identities, including the participants' own face, personally familiar (PF), celebrity and unfamiliar faces. Linear discriminant classifiers were trained and tested on EEG patterns to discriminate pairs of identities of the same familiarity level. Time-resolved classification revealed that the neural representations of identity discrimination emerge around 100 ms post-stimulus onset, relatively independently of familiarity level. In contrast, identity decoding between 200 and 400 ms is determined to a large extent by familiarity: it can be recovered with higher accuracy and for a longer duration in the case of more familiar faces. In addition, we found no increased discriminability for faces of PF persons compared to those of highly familiar celebrities. One's own face benefits from processing advantages only in a relatively late time-window. Our findings provide new insights into how the brain represents face identity with various degrees of familiarity and show that the degree of familiarity modulates the available identity-specific information at a relatively early time window.

Equivalent processing of facial expression and identity by macaque visual system and task-optimized neural network

Both the primate visual system and artificial deep neural network (DNN) models show an extraordinary ability to simultaneously classify facial expression and identity. However, the neural computations underlying the two systems are unclear. Here, we developed a multi-task DNN model that optimally classified both monkey facial expressions and identities. By comparing the fMRI neural representations of the macaque visual cortex with the best-performing DNN model, we found that both systems: (1) share initial stages for processing low-level face features which segregate into separate branches at later stages for processing facial expression and identity respectively, and (2) gain more specificity for the processing of either facial expression or identity as one progresses along each branch towards higher stages. Correspondence analysis between the DNN and monkey visual areas revealed that the amygdala and anterior fundus face patch (AF) matched well with later layers of the DNN's facial expression branch, while the anterior medial face patch (AM) matched well with later layers of the DNN's facial identity branch. Our results highlight the anatomical and functional similarities between macaque visual system and DNN model, suggesting a common mechanism between the two systems.

Face masks are less effective than sunglasses in masking face identity

The effect of covering faces on face identification is recently garnering interest amid the COVID-19 pandemic. Here, we investigated how face identification performance was affected by two types of face disguise: sunglasses and face masks. Observers studied a series of faces; then judged whether a series of test faces, comprising studied and novel faces, had been studied before or not. Face stimuli were presented either without coverings (full faces), wearing sunglasses covering the upper region (eyes, eyebrows), or wearing surgical masks covering the lower region (nose, mouth, chin). We found that sunglasses led to larger reductions in sensitivity (d’) to face identity than face masks did, while both disguises increased the tendency to report faces as studied before, a bias that was absent for full faces. In addition, faces disguised during either study or test only (i.e. study disguised faces, test with full faces; and vice versa) led to further reductions in sensitivity from both studying and testing with disguised faces, suggesting that congruence between study and test is crucial for memory retrieval. These findings implied that the upper region of the face, including the eye-region features, is more diagnostic for holistic face-identity processing than the lower face region.

Stimulus size modulates idiosyncratic neural face identity discrimination.

Humans show individual differences in neural facial identity discrimination (FID) responses across viewing positions. Critically, these variations have been shown to be reliable over time and to directly relate to observers' idiosyncratic preferences in facial information sampling. This functional signature in facial identity processing might relate to observer-specific diagnostic information processing. Although these individual differences are a valuable source of information for interpreting data, they can also be difficult to isolate when it is not possible to test many conditions. To address this potential issue, we explored whether reducing stimulus size would help decrease these interindividual variations in neural FID. We manipulated the size of face stimuli (covering 3°, 5°, 6.7°, 8.5°, and 12° of visual angle), as well as the fixation location (left eye, right eye, below the nasion, nose, and mouth) while recording electrophysiological responses. Same identity faces were presented with a base frequency of 6Hz. Different identity faces were periodically inserted within this sequence to trigger an objective index of neural FID. Our data show robust and consistent individual differences in neural face identity discrimination across viewing positions for all face sizes. Nevertheless, FID was optimal for a larger number of observers when faces subtended 6.7° of visual angle and fixation was below the nasion. This condition is the most suited to reduce natural interindividual variations in neural FID patterns, defining an important benchmark to measure neural FID when it is not possible to assess and control for observers' idiosyncrasies.

Happy face advantage disappears in Chinese context: The constancy of holistic processing for emotional faces

It is well documented that facial identity is processed in a holistic manner. Chen & Cheung (2021) recently found that happy faces evoked a larger holistic face processing than angry faces in Caucasian context. Here we adopted the complete composite face paradigm to investigate how facial emotion influenced holistic processing of facial identity in Chinese context. In Experiment 1, we used Chinese faces with happy, neutral and surprise emotion. Participants judged whether the top halves of the two successively presented composite faces were same or different, while their facial emotion was always consistent in a trial. We found equal composite face effects (CFE) for the three emotional faces. In Experiment 2, either top halves or bottom halves were judged, and angry, happy, fearful and neutral faces were compared. Results showed that although judging the top half induced a larger CFE than judging the bottom half, the CFE was equal among four emotional faces. In Experiment 3, angry and happy faces with high/low emotional intensity were further compared. We also found equal CFE for angry and happy faces, which was not modulated by the emotional intensity. These results indicated that the holistic processing of facial identity is robust and constant regardless of emotion categories and emotion intensity in Chinese context. The relation between cultural difference and holistic processing of emotional faces should be considered in the future.

Brain mechanism of sight recovery

AbstractPatients who were born with total bilateral cataracts and who were treated late in life show permanent deficits in visual acuity and other basic visual functions. Some higher visual functions are more (face identity processing, coherent motion processing) and other higher order visual functions (biological motion processing, visual exploration) are less impaired than expected from the visual acuity loss. Moreover, the integration of the late available visual input with the intact sensory inputs (hearing, touch) seems to be partially altered. The present talk will focus on new behavioural, electrophysiological and both functional and structural brain imaging results aiming to uncover the neural mechanisms of sight recovery in patients who had suffered from an extended period of blindness before they underwent cataract removal surgery.

Characteristic fixation biases in Super-Recognizers.

Neurotypical observers show large and reliable individual differences in gaze behavior along several semantic object dimensions. Individual gaze behavior toward faces has been linked to face identity processing, including that of neurotypical observers. Here, we investigated potential gaze biases in Super-Recognizers (SRs), individuals with exceptional face identity processing skills. Ten SRs, identified with a novel conservative diagnostic framework, and 43 controls freely viewed 700 complex scenes depicting more than 5000 objects. First, we tested whether SRs and controls differ in fixation biases along four semantic dimensions: faces, text, objects being touched, and bodies. Second, we tested potential group differences in fixation biases toward eyes and mouths. Finally, we tested whether SRs fixate closer to the theoretical optimal fixation point for face identification. SRs showed a stronger gaze bias toward faces and away from text and touched objects, starting from the first fixation onward. Further, SRs spent a significantly smaller proportion of first fixations and dwell time toward faces on mouths but did not differ in dwell time or first fixations devoted to eyes. Face fixation of SRs also fell significantly closer to the theoretical optimal fixation point for identification, just below the eyes. Our findings suggest that reliable superiority for face identity processing is accompanied by early fixation biases toward faces and preferred saccadic landing positions close to the theoretical optimum for face identification. We discuss future directions to investigate the functional basis of individual fixation behavior and face identity processing ability.

Face recognition in police officers: Who fits the bill?

Accurate face identity processing (FIP) is a critical component of security professions. Unfortunately, however, rapid face matching as required in real-life situations such as passport controls cannot be improved via training. While in such situations accuracy is a high priority, it is neither the only, nor most important performance measure. Officers must process high-throughput information as efficiently as possible – i.e. both accurately and rapidly. In scenarios with grave public safety implications, however, efficiency is not sufficient. Suspect surveillance and mass data analysis in criminal investigations also demand processing ample sensitive material consistently over extended periods.Police agencies have sought to optimize operations through personnel selection targeting FIP abilities. Yet to date, the lab-based tests researchers have proffered neither reflect officers’ specific tasks, nor the efficiency and consistency critical to accomplishing them. Therefore, we aimed to benchmark the three most challenging FIP tests available against two work-samples — tasks developed in consultation with police practitioners to measure specific, situationally critical performance. We solicited participation from 390 police professionals from regional police and criminal investigation departments, yielding a representative sample of 114 participating Security Officers, Mass Data Analysts, and Search Unit Members who regularly employ FIP skills in their work.Data-driven analyses of officers’ FIP abilities revealed that work-sample efficiency and consistency represented most relevant dimensions of variation, and accounted for lab-test performance. Furthermore, performance on either work-sample was better predicted by performance on the other, than by lab-based test scores. This demonstrates the limitations of lab-based tests for applied settings, and stresses the need for predicting police officers’ FIP abilities through contextually and practically relevant performance measures.