Face Matching Performance Research Articles

The impact of bilateral versus unilateral anterior temporal lobe damage on face recognition, person knowledge and semantic memory.

The functional importance of the anterior temporal lobes (ATLs) has come to prominence in two active, albeit unconnected literatures-(i) face recognition and (ii) semantic memory. To generate a unified account of the ATLs, we tested the predictions from each literature and examined the effects of bilateral versus unilateral ATL damage on face recognition, person knowledge, and semantic memory. Sixteen people with bilateral ATL atrophy from semantic dementia (SD), 17 people with unilateral ATL resection for temporal lobe epilepsy (TLE; left = 10, right = 7), and 14 controls completed tasks assessing perceptual face matching, person knowledge and general semantic memory. People with SD were impaired across all semantic tasks, including person knowledge. Despite commensurate total ATL damage, unilateral resection generated mild impairments, with minimal differences between left- and right-ATL resection. Face matching performance was largely preserved but slightly reduced in SD and right TLE. All groups displayed the familiarity effect in face matching; however, it was reduced in SD and right TLE and was aligned with the level of item-specific semantic knowledge in all participants. We propose a neurocognitive framework whereby the ATLs underpin a resilient bilateral representation system that supports semantic memory, person knowledge and face recognition.

A signal detection-based confidence-similarity model of face matching.

Face matching consists of the ability to decide whether two face images (or more) belong to the same person or to different identities. Face matching is crucial for efficient face recognition and plays an important role in applied settings such as passport control and eyewitness memory. However, despite extensive research, the mechanisms that govern face-matching performance are still not well understood. Moreover, to date, many researchers hold on to the belief that match and mismatch conditions are governed by two separate systems, an assumption that likely thwarted the development of a unified model of face matching. The present study outlines a unified unequal variance confidence-similarity signal detection-based model of face-matching performance, one that facilitates the use of receiver operating characteristics (ROC) and confidence-accuracy plots to better understand the relations between match and mismatch conditions, and their relations to factors of confidence and similarity. A binomial feature-matching mechanism is developed to support this signal detection model. The model can account for the presence of both within-identities and between-identities sources of variation in face recognition and explains a myriad of face-matching phenomena, including the match-mismatch dissociation. The model is also capable of generating new predictions concerning the role of confidence and similarity and their intricate relations with accuracy. The new model was tested against six alternative competing models (some postulate discrete rather than continuous representations) in three experiments. Data analyses consisted of hierarchically nested model fitting, ROC curve analyses, and confidence-accuracy plots analyses. All of these provided substantial support in the signal detection-based confidence-similarity model. The model suggests that the accuracy of face-matching performance can be predicted by the degree of similarity/dissimilarity of the depicted faces and the level of confidence in the decision. Moreover, according to the model, confidence and similarity ratings are strongly correlated. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Face Matching as a Function of Prior Identity Information in Professional Screeners

Using a computer-based face matching task we objectively measured face matching performance (reaction time, sensitivity, accuracy) as a function of prior identity source type (Artificial Intelligence (AI), human, none), prior information accuracy (accurate, inaccurate) and task difficulty (high, low) in professional screeners. Participants were required to judge how similar they thought a pair of faces were, to decide whether the faces in each pair were the same person, and then to judge the difficulty of that decision. Professional screeners were more accurate, faster, and, more sensitive when normative task difficulty was low. Professional screeners were also more accurate, faster, and more sensitive when prior identity source information was accurate. There was no main effect of prior identity source type on performance (there was a trend-level effect). Face matching accuracy positively correlated with normative data from non-professional screeners. Professional screeners were more accurate 80.6% of the time, compared to non-professional screeners.

Simulated automated facial recognition systems as decision-aids in forensic face matching tasks.

Automated Facial Recognition Systems (AFRS) are used by governments, law enforcement agencies, and private businesses to verify the identity of individuals. Although previous research has compared the performance of AFRS and humans on tasks of one-to-one face matching, little is known about how effectively human operators can use these AFRS as decision-aids. Our aim was to investigate how the prior decision from an AFRS affects human performance on a face matching task, and to establish whether human oversight of AFRS decisions can lead to collaborative performance gains for the human-algorithm team. The identification decisions from our simulated AFRS were informed by the performance of a real, state-of-the-art, Deep Convolutional Neural Network (DCNN) AFRS on the same task. Across five pre-registered experiments, human operators used the decisions from highly accurate AFRS (> 90%) to improve their own face matching performance compared with baseline (sensitivity gain: Cohen's d = 0.71-1.28; overall accuracy gain: d = 0.73-1.46). Yet, despite this improvement, AFRS-aided human performance consistently failed to reach the level that the AFRS achieved alone. Even when the AFRS erred only on the face pairs with the highest human accuracy (> 89%), participants often failed to correct the system's errors, while also overruling many correct decisions, raising questions about the conditions under which human oversight might enhance AFRS operation. Overall, these data demonstrate that the human operator is a limiting factor in this simple model of human-AFRS teaming. These findings have implications for the "human-in-the-loop" approach to AFRS oversight in forensic face matching scenarios. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Face matching and metacognition: investigating individual differences and a training intervention.

Although researchers have begun to consider metacognitive insight during face matching, little is known about the underlying mechanism. Here, I investigated whether objective ability, as well as self-assessed ability, were able to predict metacognitive performance, that is, the ability to differentiate correct and incorrect responses in terms of confidence. In addition, I considered whether a training intervention resulted in improvements to both face matching performance and metacognitive insight. In this experiment (N = 220), participants completed a face matching task, with either a diagnostic feature training course or a control course presented at the halfway point. In addition, a second face matching task, as well as a self-report questionnaire regarding ability, were completed to provide measures of objective and self-assessed ability respectively. Higher self-assessed ability with faces, as well as higher objective ability with face matching, predicted better metacognitive performance, i.e., greater confidence in correct, in comparison with incorrect, responses. This pattern of results was evident both when objective ability was measured through performance on the same task used to measure metacognitive insight and when a different task was used. Finally, the training intervention failed to produce improvements in face matching performance and showed no evidence of altering metacognitive ability. The current work begins to address the mechanism underlying individual differences in metacognitive insight during face matching. Although support was provided for a competence-based account, where better face matchers showed greater performance on the task and were more successful in monitoring their performance, further work might focus on decoupling task performance and competence in order to more conclusively explain why some people are more insightful than others.

Contributions of face processing, social anhedonia and mentalizing to the expression of social autistic-like traits.

Quantitative autistic-like traits (QATs) are a constellation of traits that mirror those of clinical autism and are thought to share the same mechanisms as the condition. There is great interest in identifying the genetic and neurobiological basis of QATs, but progress is hindered by the composite nature of these clinically based constructs. Social QATs are defined according to the diagnostic criteria for autism, comprising multiple potential neural mechanisms that may contribute to varying degrees. The objective of this study was to decompose social QATs into more specific constructs, in line with the Research Domain Criteria (RDoC). We chose constructs with trait-like properties and known or suggested significance for autistic social function: (1) social anhedonia, (2) prosopagnosia (face blindness), and (3) mentalizing (attributing mental states to images of eyes). We hypothesized that these constructs may all contribute to observed variance in social QATs. We recruited 148 adults with a broad range of QATs (mean age 37.9 years, range 18-69; 50% female; 5.4% autistic) to an experimental behavioral study conducted online. We estimated social QATs using the social factor of the Comprehensive Autistic Traits Inventory. We used the Oxford Face Matching Task and the Reading the Mind in the Eyes Test to measure face matching ability and mentalizing, respectively. Social anhedonia traits were measured with the Anticipatory and Consummatory Interpersonal Pleasure Scale, and prosopagnosic traits with the 20-item Prosopagnosia Index. A combination of frequentist and Bayesian statistics was used to test the social constructs as predictors of social QATs. We found that social anhedonic traits, prosopagnosic traits, and face matching performance were likely predictors of social QATs, whereas mentalizing showed limited contribution. The findings support prosopagnosic and anhedonic traits, but not mentalizing deficits, as dimensional predictors of individual differences in social function across the autistic spectrum. Further, the study strongly suggests that social reward systems and face processing networks play significant and independent roles in autistic-like social function.

High-frequency transcranial random noise stimulation enhances unfamiliar face matching of high resolution and pixelated faces

Face identification is useful for social interactions and its impairment can lead to severe social and mental problems. This ability is also remarkably important in applied settings, including eyewitness identification and ID verification. Several studies have demonstrated the potential of Transcranial Random Noise Stimulation (tRNS) to enhance different cognitive skills. However, research has produced inconclusive results about the effectiveness of tRNS to improve face identification. The present study aims to further explore the effect of tRNS on face identification using an unfamiliar face matching task. Observers firstly received either high-frequency bilateral tRNS or sham stimulation for 20 min. The stimulation targeted occipitotemporal areas, which have been previously involved in face processing. In a subsequent stage, observers were asked to perform an unfamiliar face matching task consisting of unaltered and pixelated face pictures. Compared to the sham stimulation group, the high-frequency tRNS group showed better unfamiliar face matching performance with both unaltered and pixelated faces. Our results show that a single high-frequency tRNS session might suffice to improve face identification abilities. These results have important consequences for the treatment of face recognition disorders, and potential applications in those scenarios whereby the identification of faces is primordial.

Forensic facial examiners versus super-recognizers: Evaluating behavior beyond accuracy.

We evaluated the detailed, behavioral properties of face matching performance in two specialist groups: forensic facial examiners and super-recognizers. Both groups compare faces to determine identity with high accuracy and outperform the general population. Typically, facial examiners are highly trained; super-recognizers rely on natural ability. We found distinct behaviors between these two groups. Examiners used the full 7-point identity judgment scale (-3: "different"; +3: "same"). Super-recognizers' judgments clustered toward highly confident decisions. Examiners' judgments for same- and different-identities were symmetric across the scale midpoint (0); super-recognizers' judgments were not. Examiners showed higher identity judgment agreement than super-recognizers. Despite these qualitative differences, both groups showed insight into their own accuracy: more confident people and those who rated the task to be easier tended to be more accurate. Altogether, we show to better understand and interpret judgments according to the nature of someone's facial expertise, evaluations should assess more than accuracy.

Image variability and face matching.

This study investigates whether variability in perceived trait judgements disrupts our ability to match unfamiliar faces. In this preregistered study, 174 participants completed a face matching task where they were asked to indicate whether two ambient face images belonged to the same person or different people (17,748 total data points). Participants completed 51 match trials consisting of images of the same person that differed substantially on one trait (either trustworthiness, dominance or attractiveness) with minimal differences in the alternate traits. Participants also completed 51 mismatch trials which contained two photos of similar-looking individuals. We hypothesised that participants would make more errors on match trials when images differed in terms of attractiveness ratings than when they differed on trustworthiness or dominance. Contrary to expectations, images that differed in terms of attractiveness were matched most accurately, and there was no relationship between the extent of differences in attractiveness ratings and accuracy. There was some evidence that differences in perceived dominance and, to a lesser extent, trustworthiness were associated with lower face matching performance. However, these relationships were not significant when alternate traits were accounted for. The findings of our study suggest that face matching performance is largely robust against variation in trait judgements.

Masked face identification is improved by diagnostic feature training

To slow the spread of COVID-19, many people now wear face masks in public. Face masks impair our ability to identify faces, which can cause problems for professional staff who identify offenders or members of the public. Here, we investigate whether performance on a masked face matching task can be improved by training participants to compare diagnostic facial features (the ears and facial marks)—a validated training method that improves matching performance for unmasked faces. We show this brief diagnostic feature training, which takes less than two minutes to complete, improves matching performance for masked faces by approximately 5%. A control training course, which was unrelated to face identification, had no effect on matching performance. Our findings demonstrate that comparing the ears and facial marks is an effective means of improving face matching performance for masked faces. These findings have implications for professions that regularly perform face identification.

Individual differences in emotion regulation and face recognition.

Face recognition ability is highly variable among neurologically intact populations. Across three experiments, this study examined for the first time associations between individual differences in a range of adaptive versus maladaptive emotion regulation strategies and face recognition. Using an immediate face-memory paradigm, in which observers had to identify a self-paced learned unfamiliar face from a 10-face target-present/ target-absent line-up, Experiment 1 (N = 42) found high levels of expressive suppression (the ongoing efforts to inhibit emotion-expressive behaviors), but not cognitive reappraisal (the cognitive re-evaluation of emotional events to change their emotional consequences), were associated with a lower level of overall face-memory accuracy and higher rates of misidentifications and false positives. Experiment 2 (N = 53) replicated these finding using a range of face-matching tasks, where observers were asked to match pairs of same-race or different-race face images taken on the same day or during different times. Once again, high levels of expressive suppression were associated with a lower level of overall face-matching performance and higher rates of false positives, but cognitive reappraisal did not correlate with any face-matching measure. Finally, Experiment 3 (N = 52) revealed that the higher use of maladaptive cognitive emotion regulation strategies, especially catastrophizing, was associated with lower levels of overall face-matching performances and higher rates of false positives. All told, the current research provides new evidence concerning the important associations between emotion and cognition.

Surgical face masks impair human face matching performance for familiar and unfamiliar faces

In response to the COVID-19 pandemic, many governments around the world now recommend, or require, that their citizens cover the lower half of their face in public. Consequently, many people now wear surgical face masks in public. We investigated whether surgical face masks affected the performance of human observers, and a state-of-the-art face recognition system, on tasks of perceptual face matching. Participants judged whether two simultaneously presented face photographs showed the same person or two different people. We superimposed images of surgical masks over the faces, creating three different mask conditions: control (no masks), mixed (one face wearing a mask), and masked (both faces wearing masks). We found that surgical face masks have a large detrimental effect on human face matching performance, and that the degree of impairment is the same regardless of whether one or both faces in each pair are masked. Surprisingly, this impairment is similar in size for both familiar and unfamiliar faces. When matching masked faces, human observers are biased to reject unfamiliar faces as “mismatches” and to accept familiar faces as “matches”. Finally, the face recognition system showed very high classification accuracy for control and masked stimuli, even though it had not been trained to recognise masked faces. However, accuracy fell markedly when one face was masked and the other was not. Our findings demonstrate that surgical face masks impair the ability of humans, and naïve face recognition systems, to perform perceptual face matching tasks. Identification decisions for masked faces should be treated with caution.

Human face matching performance is predicted by deviation from algorithmic similarity

Human face matching performance is predicted by deviation from algorithmic similarity

Human-algorithm teaming in face recognition: How algorithm outcomes cognitively bias human decision-making.

In face recognition applications, humans often team with algorithms, reviewing algorithm results to make an identity decision. However, few studies have explicitly measured how algorithms influence human face matching performance. One study that did examine this interaction found a concerning deterioration of human accuracy in the presence of algorithm errors. We conducted an experiment to examine how prior face identity decisions influence subsequent human judgements about face similarity. 376 volunteers were asked to rate the similarity of face pairs along a scale. Volunteers performing the task were told that they were reviewing identity decisions made by different sources, either a computer or human, or were told to make their own judgement without prior information. Replicating past results, we found that prior identity decisions, presented as labels, influenced volunteers’ own identity judgements. We extend these results as follows. First, we show that the influence of identity decision labels was independent of indicated decision source (human or computer) despite volunteers’ greater distrust of human identification ability. Second, applying a signal detection theory framework, we show that prior identity decision labels did not reduce volunteers’ attention to the face pair. Discrimination performance was the same with and without the labels. Instead, prior identity decision labels altered volunteers’ internal criterion used to judge a face pair as “matching” or “non-matching”. This shifted volunteers’ face pair similarity judgements by a full step along the response scale. Our work shows how human face matching is affected by prior identity decision labels and we discuss how this may limit the total accuracy of human-algorithm teams performing face matching tasks.

Hyper-realistic Face Masks in a Live Passport-Checking Task.

Hyper-realistic face masks have been used as disguises in at least one border crossing and in numerous criminal cases. Experimental tests using these masks have shown that viewers accept them as real faces under a range of conditions. Here, we tested mask detection in a live identity verification task. Fifty-four visitors at the London Science Museum viewed a mask wearer at close range (2 m) as part of a mock passport check. They then answered a series of questions designed to assess mask detection, while the masked traveller was still in view. In the identity matching task, 8% of viewers accepted the mask as matching a real photo of someone else, and 82% accepted the match between masked person and masked photo. When asked if there was any reason to detain the traveller, only 13% of viewers mentioned a mask. A further 11% picked disguise from a list of suggested reasons. Even after reading about mask-related fraud, 10% of viewers judged that the traveller was not wearing a mask. Overall, mask detection was poor and was not predicted by unfamiliar face matching performance. We conclude that hyper-realistic face masks could go undetected during live identity checks.

Challenges to both reliability and validity of masculinity-preference measures in menstrual-cycle-effects research

Although it remains contentious, women's changeable attraction to masculine faces has been used to inform evolutionary ideas about human mating strategies. Typical experiments in this area use two-alternative-forced-choice (2afc) over a few pairs of similar images. The reliability of these measures is analysed suggesting that many studies have too few trials to be reliable. In the current experiment, fertility shifts in preferences for masculinised faces (and Africanised faces) were explored using both attractiveness ratings and a 2afc method over 80 pairs. The 2afc method showed a fertility shift in preferences whereas attractiveness ratings did not show a shift. Further, it was demonstrated how the size of the preferences shown in the 2afc tasks correlated with general face-matching performance. It is concluded that fertility is associated with improved face-processing accuracy and hence 2afc designs have poor validity as measures of masculinity preference. These issues of validity and reliability may have contributed to the contentious nature of fertility effects on preferences. Further, validity and reliability need to be considered in any study where a change in preference is identified using a comparative-preference task.

Face recognition under varying lighting conditions: improving the recognition accuracy for local descriptors based on weber-face followed by difference of Gaussians

ABSTRACTIllumination variation is among the several bottlenecks in a face recognition system because it can greatly affect the appearance of a face image, which causes a reduction in the face matching performance of the system. Using illumination preprocessing methods is an effective approach to overcome this problem. Despite the achievements made, however, each method still has its own demerits. In this paper, an efficient representation method insensitive to varying illumination based on a combination of Weber-face (WF) and the difference of Gaussians (DoG) methods is proposed for human face recognition using local descriptors. After processing by our method, the obtained image will preserve more facial features and edge information while shading effects are eliminated. To demonstrate the potential of the proposed method, two systems using nearest-neighbor and support vector machine classifiers were implemented. Experimental results for the CMU-PIE and extended Yale B face databases showed that our method could effectively eliminate the effect of uneven illumination and achieved better recognition accuracies in comparison to other state-of-the-art methods (DoG, gradient face, histogram equalization, and WF).

A grey area: how does image hue affect unfamiliar face matching?

The role of image colour in face identification has received little attention in research despite the importance of identifying people from photographs in identity documents (IDs). Here, in two experiments, we investigated whether colour congruency of two photographs, shown side by side, affects face-matching accuracy. Participants were presented with two images from the Models Face Matching Test (experiment 1) and a newly devised matching task incorporating female faces (experiment 2) and asked to decide whether they show the same person or two different people. The photographs were either both in colour, both in grayscale, or mixed (one in grayscale and one in colour). Participants were more likely to accept a pair of images as a “match”, i.e. same person, in the mixed condition, regardless of whether the identity of the pair was the same or not. This demonstrates a clear shift in bias between “congruent” colour conditions and the mixed trials. In addition, there was a small decline in accuracy in the mixed condition, relative to when the images were presented in colour. Our study provides the first evidence that the hue of document photographs matters for face-matching performance. This finding has important implications for the design and regulation of photographic ID worldwide.

Face averages and multiple images in a live matching task.

We know from previous research that unfamiliar face matching (determining whether two simultaneously presented images show the same person or not) is very error-prone. A small number of studies in laboratory settings have shown that the use of multiple images or a face average, rather than a single image, can improve face matching performance. Here, we tested 1,999 participants using four-image arrays and face averages in two separate live matching tasks. Matching a single image to a live person resulted in numerous errors (79.9% accuracy across both experiments), and neither multiple images (82.4% accuracy) nor face averages (76.9% accuracy) improved performance. These results are important when considering possible alterations which could be made to photo-ID. Although multiple images and face averages have produced measurable improvements in performance in recent laboratory studies, they do not produce benefits in a real-world live face matching context.

Smiles in face matching: Idiosyncratic information revealed through a smile improves unfamiliar face matching performance

Unfamiliar face matching is a surprisingly difficult task, yet we often rely on people's matching decisions in applied settings (e.g., border control). Most attempts to improve accuracy (including training and image manipulation) have had very limited success. In a series of studies, we demonstrate that using smiling rather than neutral pairs of images brings about significant improvements in face matching accuracy. This is true for both match and mismatch trials, implying that the information provided through a smile helps us detect images of the same identity as well as distinguishing between images of different identities. Study 1 compares matching performance when images in the face pair display either an open-mouth smile or a neutral expression. In Study 2, we add an intermediate level, closed-mouth smile, to identify the effect of teeth being exposed, and Study 3 explores face matching accuracy when only information about the lower part of the face is available. Results demonstrate that an open-mouth smile changes the face in an idiosyncratic way which aids face matching decisions. Such findings have practical implications for matching in the applied context where we typically use neutral images to represent ourselves in official documents.